2024-04-25T14:17:07.317 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T14:17:07.317 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} --owner scheduled_leonidus@teuthology --name leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi --archive /home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673495 --verbose -- /home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673495/orig.config.yaml 2024-04-25T14:17:07.377 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T14:17:07.460 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673495 branch: wip-lusov-quiescer description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} email: null first_in_suite: false job_id: '7673495' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiescer ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiescer sha1: 107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69 owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8982 sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f sleep_before_teardown: 0 subset: 200/111111 suite: fs suite_branch: wip-lusov-quiescer suite_path: /home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph suite_sha1: 107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi053.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGJM6JH6Pk6xemw5/TdSJdi6Ak3+jy57DEtD2j58Q0Di+iceZYPyedDZQkbwn0ylfTB475BKPxEYjZfAolPaKfY= smithi150.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFa/eCBrB7n0Pjo8xu/HaUXE71rX4OFlV6TbzV1r4WlItzqxBEy9mstSRPk2c8CErEC68nRDrHxYu40nVK+j0Qc= smithi183.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHKX7tMdcjyqITsVtCbAL8eIzb/rVW9tORkvJa4nMcOylbzH29W+LZT/CrxO0igJkvWpXYzf8pbEkMGbgFG3m/0= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_13:50:46 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T14:17:07.460 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa; will attempt to use it 2024-04-25T14:17:07.461 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa/tasks 2024-04-25T14:17:07.461 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T14:17:07.463 INFO:teuthology.task.internal:Checking packages... 2024-04-25T14:17:07.487 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd69f13d827952bb07bb9b7973bca14f97d764d6f' 2024-04-25T14:17:07.487 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T14:17:07.487 INFO:teuthology.packaging:ref: None 2024-04-25T14:17:07.487 INFO:teuthology.packaging:tag: None 2024-04-25T14:17:07.487 INFO:teuthology.packaging:branch: wip-lusov-quiescer 2024-04-25T14:17:07.487 INFO:teuthology.packaging:sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T14:17:07.487 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiescer 2024-04-25T14:17:07.646 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3156.gd69f13d8 2024-04-25T14:17:07.648 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T14:17:07.660 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T14:17:07.660 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T14:17:07.667 INFO:teuthology.task.internal:Saving configuration 2024-04-25T14:17:07.689 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T14:17:07.696 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T14:17:07.719 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi053.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673495', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 14:11:49.865066', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGJM6JH6Pk6xemw5/TdSJdi6Ak3+jy57DEtD2j58Q0Di+iceZYPyedDZQkbwn0ylfTB475BKPxEYjZfAolPaKfY='} 2024-04-25T14:17:07.737 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi150.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673495', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 14:11:49.866364', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFa/eCBrB7n0Pjo8xu/HaUXE71rX4OFlV6TbzV1r4WlItzqxBEy9mstSRPk2c8CErEC68nRDrHxYu40nVK+j0Qc='} 2024-04-25T14:17:07.760 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi183.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673495', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 14:11:49.863707', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHKX7tMdcjyqITsVtCbAL8eIzb/rVW9tORkvJa4nMcOylbzH29W+LZT/CrxO0igJkvWpXYzf8pbEkMGbgFG3m/0='} 2024-04-25T14:17:07.760 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T14:17:07.768 INFO:teuthology.task.internal:roles: ubuntu@smithi053.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-25T14:17:07.768 INFO:teuthology.task.internal:roles: ubuntu@smithi150.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-25T14:17:07.768 INFO:teuthology.task.internal:roles: ubuntu@smithi183.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-25T14:17:07.768 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T14:17:07.869 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd6aa5850d0>, signals=[15]) 2024-04-25T14:17:07.870 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T14:17:07.878 INFO:teuthology.task.internal:Opening connections... 2024-04-25T14:17:07.879 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi053.front.sepia.ceph.com 2024-04-25T14:17:07.882 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T14:17:07.961 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi150.front.sepia.ceph.com 2024-04-25T14:17:07.961 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T14:17:08.039 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi183.front.sepia.ceph.com 2024-04-25T14:17:08.039 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T14:17:08.114 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T14:17:08.122 DEBUG:teuthology.orchestra.run.smithi053:> uname -m 2024-04-25T14:17:08.141 INFO:teuthology.orchestra.run.smithi053.stdout:x86_64 2024-04-25T14:17:08.141 DEBUG:teuthology.orchestra.run.smithi053:> cat /etc/os-release 2024-04-25T14:17:08.199 INFO:teuthology.orchestra.run.smithi053.stdout:NAME="CentOS Stream" 2024-04-25T14:17:08.199 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION="9" 2024-04-25T14:17:08.199 INFO:teuthology.orchestra.run.smithi053.stdout:ID="centos" 2024-04-25T14:17:08.199 INFO:teuthology.orchestra.run.smithi053.stdout:ID_LIKE="rhel fedora" 2024-04-25T14:17:08.199 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_ID="9" 2024-04-25T14:17:08.199 INFO:teuthology.orchestra.run.smithi053.stdout:PLATFORM_ID="platform:el9" 2024-04-25T14:17:08.199 INFO:teuthology.orchestra.run.smithi053.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T14:17:08.200 INFO:teuthology.orchestra.run.smithi053.stdout:ANSI_COLOR="0;31" 2024-04-25T14:17:08.200 INFO:teuthology.orchestra.run.smithi053.stdout:LOGO="fedora-logo-icon" 2024-04-25T14:17:08.200 INFO:teuthology.orchestra.run.smithi053.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T14:17:08.200 INFO:teuthology.orchestra.run.smithi053.stdout:HOME_URL="https://centos.org/" 2024-04-25T14:17:08.200 INFO:teuthology.orchestra.run.smithi053.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T14:17:08.200 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T14:17:08.200 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T14:17:08.201 INFO:teuthology.lock.ops:Updating smithi053.front.sepia.ceph.com on lock server 2024-04-25T14:17:08.219 DEBUG:teuthology.orchestra.run.smithi150:> uname -m 2024-04-25T14:17:08.237 INFO:teuthology.orchestra.run.smithi150.stdout:x86_64 2024-04-25T14:17:08.237 DEBUG:teuthology.orchestra.run.smithi150:> cat /etc/os-release 2024-04-25T14:17:08.294 INFO:teuthology.orchestra.run.smithi150.stdout:NAME="CentOS Stream" 2024-04-25T14:17:08.295 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION="9" 2024-04-25T14:17:08.295 INFO:teuthology.orchestra.run.smithi150.stdout:ID="centos" 2024-04-25T14:17:08.295 INFO:teuthology.orchestra.run.smithi150.stdout:ID_LIKE="rhel fedora" 2024-04-25T14:17:08.295 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_ID="9" 2024-04-25T14:17:08.295 INFO:teuthology.orchestra.run.smithi150.stdout:PLATFORM_ID="platform:el9" 2024-04-25T14:17:08.295 INFO:teuthology.orchestra.run.smithi150.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T14:17:08.295 INFO:teuthology.orchestra.run.smithi150.stdout:ANSI_COLOR="0;31" 2024-04-25T14:17:08.295 INFO:teuthology.orchestra.run.smithi150.stdout:LOGO="fedora-logo-icon" 2024-04-25T14:17:08.295 INFO:teuthology.orchestra.run.smithi150.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T14:17:08.295 INFO:teuthology.orchestra.run.smithi150.stdout:HOME_URL="https://centos.org/" 2024-04-25T14:17:08.295 INFO:teuthology.orchestra.run.smithi150.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T14:17:08.295 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T14:17:08.295 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T14:17:08.296 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server 2024-04-25T14:17:08.318 DEBUG:teuthology.orchestra.run.smithi183:> uname -m 2024-04-25T14:17:08.336 INFO:teuthology.orchestra.run.smithi183.stdout:x86_64 2024-04-25T14:17:08.337 DEBUG:teuthology.orchestra.run.smithi183:> cat /etc/os-release 2024-04-25T14:17:08.394 INFO:teuthology.orchestra.run.smithi183.stdout:NAME="CentOS Stream" 2024-04-25T14:17:08.394 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION="9" 2024-04-25T14:17:08.394 INFO:teuthology.orchestra.run.smithi183.stdout:ID="centos" 2024-04-25T14:17:08.395 INFO:teuthology.orchestra.run.smithi183.stdout:ID_LIKE="rhel fedora" 2024-04-25T14:17:08.395 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_ID="9" 2024-04-25T14:17:08.395 INFO:teuthology.orchestra.run.smithi183.stdout:PLATFORM_ID="platform:el9" 2024-04-25T14:17:08.395 INFO:teuthology.orchestra.run.smithi183.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T14:17:08.395 INFO:teuthology.orchestra.run.smithi183.stdout:ANSI_COLOR="0;31" 2024-04-25T14:17:08.395 INFO:teuthology.orchestra.run.smithi183.stdout:LOGO="fedora-logo-icon" 2024-04-25T14:17:08.395 INFO:teuthology.orchestra.run.smithi183.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T14:17:08.396 INFO:teuthology.orchestra.run.smithi183.stdout:HOME_URL="https://centos.org/" 2024-04-25T14:17:08.396 INFO:teuthology.orchestra.run.smithi183.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T14:17:08.396 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T14:17:08.396 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T14:17:08.396 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server 2024-04-25T14:17:08.419 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T14:17:08.433 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T14:17:08.441 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T14:17:08.441 DEBUG:teuthology.orchestra.run.smithi053:> test '!' -e /home/ubuntu/cephtest 2024-04-25T14:17:08.444 DEBUG:teuthology.orchestra.run.smithi150:> test '!' -e /home/ubuntu/cephtest 2024-04-25T14:17:08.448 DEBUG:teuthology.orchestra.run.smithi183:> test '!' -e /home/ubuntu/cephtest 2024-04-25T14:17:08.464 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T14:17:08.472 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T14:17:08.473 DEBUG:teuthology.orchestra.run.smithi053:> test -z $(ls -A /var/lib/ceph) 2024-04-25T14:17:08.502 DEBUG:teuthology.orchestra.run.smithi150:> test -z $(ls -A /var/lib/ceph) 2024-04-25T14:17:08.505 DEBUG:teuthology.orchestra.run.smithi183:> test -z $(ls -A /var/lib/ceph) 2024-04-25T14:17:08.633 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T14:17:08.709 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T14:17:08.724 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-04-25T14:17:08.724 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-04-25T14:17:08.725 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-04-25T14:17:08.725 DEBUG:teuthology.orchestra.run.smithi053:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T14:17:08.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:17:08.743 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-25T14:17:09.143 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T14:17:09.143 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-25T14:17:09.143 DEBUG:teuthology.orchestra.run.smithi053:> uname -r 2024-04-25T14:17:09.161 INFO:teuthology.orchestra.run.smithi053.stdout:5.14.0-437.el9.x86_64 2024-04-25T14:17:09.162 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T14:17:09.162 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-25T14:17:09.162 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum upgrade -y linux-firmware 2024-04-25T14:17:11.473 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-25T14:17:13.590 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:01 2024-04-25T14:17:18.033 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - CRB 8.3 MB/s | 6.2 MB 00:00 2024-04-25T14:17:20.706 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - Extras packages 11 kB/s | 16 kB 00:01 2024-04-25T14:17:22.552 INFO:teuthology.orchestra.run.smithi053.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-25T14:17:27.832 INFO:teuthology.orchestra.run.smithi053.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-25T14:17:29.894 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-04-25T14:17:29.895 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-04-25T14:17:29.895 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-04-25T14:17:29.979 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-25T14:17:29.980 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T14:17:30.128 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-25T14:17:30.128 DEBUG:teuthology.orchestra.run.smithi053:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-25T14:17:30.131 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi053.front.sepia.ceph.com to complete... 2024-04-25T14:17:58.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25 14:17:58 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-25T14:17:58.059 INFO:teuthology.orchestra.run.smithi053.stderr:FINISHED --2024-04-25 14:17:58-- 2024-04-25T14:17:58.059 INFO:teuthology.orchestra.run.smithi053.stderr:Total wall clock time: 28s 2024-04-25T14:17:58.059 INFO:teuthology.orchestra.run.smithi053.stderr:Downloaded: 1 files, 341M in 27s (12.4 MB/s) 2024-04-25T14:17:58.063 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-25T14:17:58.063 DEBUG:teuthology.orchestra.run.smithi053:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-25T14:18:00.257 INFO:teuthology.orchestra.run.smithi053.stdout:Verifying... ######################################## 2024-04-25T14:18:00.390 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing... ######################################## 2024-04-25T14:18:00.510 INFO:teuthology.orchestra.run.smithi053.stdout:Updating / installing... 2024-04-25T14:18:08.819 INFO:teuthology.orchestra.run.smithi053.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-25T14:18:46.885 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi053.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-25T14:18:46.885 DEBUG:teuthology.orchestra.run.smithi053:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-25T14:18:46.968 INFO:teuthology.orchestra.run.smithi053.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-25T14:18:46.968 INFO:teuthology.orchestra.run.smithi053.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-25T14:18:46.968 INFO:teuthology.orchestra.run.smithi053.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T14:18:46.968 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-25T14:18:46.968 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-25T14:18:46.968 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-25T14:18:46.969 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-25T14:18:46.969 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-25T14:18:46.969 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-25T14:18:46.969 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-25T14:18:46.969 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-25T14:18:46.969 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-25T14:18:46.969 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-25T14:18:46.969 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-25T14:18:46.969 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-25T14:18:46.969 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-25T14:18:46.969 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-25T14:18:46.969 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-25T14:18:46.969 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-25T14:18:46.969 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-25T14:18:46.969 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-25T14:18:46.970 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-25T14:18:46.970 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-25T14:18:46.970 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-25T14:18:46.970 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-25T14:18:46.970 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-25T14:18:46.970 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-25T14:18:46.970 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-25T14:18:46.970 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-25T14:18:46.970 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-25T14:18:46.970 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-25T14:18:46.970 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-25T14:18:46.970 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-25T14:18:46.970 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-25T14:18:46.970 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-25T14:18:46.971 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-25T14:18:46.971 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-25T14:18:46.971 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-25T14:18:46.971 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-25T14:18:46.971 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-25T14:18:46.971 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-25T14:18:46.971 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-25T14:18:46.971 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-25T14:18:46.971 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-25T14:18:46.971 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-25T14:18:46.971 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-25T14:18:46.971 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-25T14:18:46.971 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-25T14:18:46.971 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-25T14:18:46.971 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-25T14:18:46.971 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-25T14:18:46.971 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-25T14:18:46.972 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-25T14:18:46.972 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-25T14:18:46.972 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-25T14:18:46.972 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-25T14:18:46.972 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-25T14:18:46.972 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-25T14:18:46.972 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-25T14:18:46.972 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-25T14:18:46.972 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-25T14:18:46.972 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-25T14:18:46.972 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-25T14:18:46.972 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-25T14:18:46.972 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-25T14:18:46.972 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-25T14:18:46.972 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-25T14:18:46.973 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-25T14:18:46.973 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-25T14:18:46.973 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-25T14:18:46.973 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-25T14:18:46.973 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-25T14:18:46.973 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-25T14:18:46.973 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-25T14:18:46.973 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-25T14:18:46.973 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-25T14:18:46.973 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-25T14:18:46.973 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-25T14:18:46.973 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-25T14:18:46.973 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-25T14:18:46.973 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-25T14:18:46.973 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-25T14:18:46.973 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-25T14:18:46.974 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-25T14:18:46.974 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-25T14:18:46.974 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-25T14:18:46.974 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-25T14:18:46.974 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-25T14:18:46.974 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-25T14:18:46.974 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-25T14:18:46.974 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-25T14:18:46.974 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-25T14:18:46.974 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-25T14:18:46.974 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-25T14:18:46.974 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-25T14:18:46.974 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-25T14:18:46.975 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-25T14:18:46.975 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-25T14:18:46.975 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-25T14:18:46.975 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-25T14:18:46.975 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-25T14:18:46.975 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-25T14:18:46.975 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-25T14:18:46.975 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-25T14:18:46.975 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-25T14:18:46.975 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-25T14:18:46.975 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-25T14:18:46.975 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-25T14:18:46.975 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-25T14:18:46.975 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-25T14:18:46.975 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-25T14:18:46.975 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-25T14:18:46.976 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-25T14:18:46.976 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-25T14:18:46.976 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-25T14:18:46.976 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-25T14:18:46.976 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-25T14:18:46.976 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-25T14:18:46.976 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-25T14:18:46.976 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-25T14:18:46.976 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-25T14:18:46.976 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-25T14:18:46.976 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-25T14:18:46.976 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-25T14:18:46.976 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-25T14:18:46.976 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-25T14:18:46.976 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-25T14:18:46.977 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-25T14:18:46.977 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-25T14:18:46.977 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-25T14:18:46.977 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-25T14:18:46.977 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-25T14:18:46.977 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-25T14:18:46.977 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-25T14:18:46.977 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-25T14:18:46.977 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-25T14:18:46.977 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-25T14:18:46.977 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-25T14:18:46.977 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-25T14:18:46.977 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-25T14:18:46.977 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-25T14:18:46.978 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-25T14:18:46.978 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-25T14:18:46.978 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-25T14:18:46.978 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-25T14:18:46.978 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-25T14:18:46.978 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-25T14:18:46.978 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-25T14:18:46.978 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-25T14:18:46.978 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-25T14:18:46.978 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-25T14:18:46.978 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-25T14:18:46.978 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-25T14:18:46.978 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-25T14:18:46.978 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-25T14:18:46.979 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-25T14:18:46.979 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-25T14:18:46.979 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-25T14:18:46.979 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-25T14:18:46.979 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-25T14:18:46.979 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-25T14:18:46.979 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-25T14:18:46.979 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-25T14:18:46.979 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-25T14:18:46.979 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-25T14:18:46.979 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-25T14:18:46.979 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-25T14:18:46.979 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-25T14:18:46.979 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-25T14:18:46.979 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-25T14:18:46.980 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-25T14:18:46.980 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-25T14:18:46.980 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-25T14:18:46.980 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-25T14:18:46.980 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-25T14:18:46.980 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-25T14:18:46.980 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-25T14:18:46.980 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-25T14:18:46.980 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-25T14:18:46.980 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-25T14:18:46.980 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-25T14:18:46.980 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-25T14:18:46.980 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-25T14:18:46.980 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-25T14:18:46.980 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-25T14:18:46.980 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-25T14:18:46.981 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-25T14:18:46.981 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-25T14:18:46.981 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-25T14:18:46.981 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-25T14:18:46.981 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-25T14:18:46.981 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-25T14:18:46.981 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-25T14:18:46.981 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-25T14:18:46.981 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-25T14:18:46.981 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-25T14:18:46.981 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-25T14:18:46.981 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-25T14:18:46.981 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-25T14:18:46.981 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-25T14:18:46.981 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-25T14:18:46.982 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-25T14:18:46.982 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-25T14:18:46.982 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-25T14:18:46.982 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-25T14:18:46.982 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-25T14:18:46.982 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-25T14:18:46.982 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-25T14:18:46.982 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-25T14:18:46.982 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-25T14:18:46.982 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-25T14:18:46.982 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-25T14:18:46.982 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-25T14:18:46.982 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-25T14:18:46.982 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-25T14:18:46.982 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-25T14:18:46.983 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-25T14:18:46.983 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-25T14:18:46.983 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-25T14:18:46.983 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-25T14:18:46.983 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-25T14:18:46.983 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-25T14:18:46.983 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-25T14:18:46.983 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-25T14:18:46.983 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-25T14:18:46.983 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-25T14:18:46.983 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-25T14:18:46.983 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-25T14:18:46.983 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-25T14:18:46.983 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-25T14:18:46.983 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-25T14:18:46.983 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-25T14:18:46.984 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-25T14:18:46.984 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-25T14:18:46.984 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-25T14:18:46.984 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-25T14:18:46.984 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-25T14:18:46.984 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-25T14:18:46.984 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-25T14:18:46.984 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-25T14:18:46.984 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-25T14:18:46.984 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-25T14:18:46.984 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-25T14:18:46.984 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-25T14:18:46.984 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-25T14:18:46.985 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-25T14:18:46.985 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-25T14:18:46.985 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-25T14:18:46.985 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-25T14:18:46.985 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-25T14:18:46.985 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-25T14:18:46.985 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-25T14:18:46.985 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-25T14:18:46.985 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-25T14:18:46.985 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-25T14:18:46.985 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-25T14:18:46.985 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-25T14:18:46.985 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-25T14:18:46.985 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-25T14:18:46.985 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-25T14:18:46.985 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-25T14:18:46.986 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-25T14:18:46.986 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-25T14:18:46.986 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-25T14:18:46.986 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-25T14:18:46.986 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-25T14:18:46.986 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-25T14:18:46.986 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-25T14:18:46.986 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-25T14:18:46.986 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-25T14:18:46.986 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-25T14:18:46.986 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-25T14:18:46.986 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-25T14:18:46.986 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-25T14:18:46.986 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-25T14:18:46.987 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-25T14:18:46.987 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-25T14:18:46.987 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-25T14:18:46.987 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-25T14:18:46.987 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-25T14:18:46.987 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-25T14:18:46.987 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-25T14:18:46.987 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-25T14:18:46.987 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-25T14:18:46.987 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-25T14:18:46.987 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-25T14:18:46.987 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-25T14:18:46.987 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-25T14:18:46.987 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-25T14:18:46.987 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-25T14:18:46.987 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-25T14:18:46.988 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-25T14:18:46.988 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-25T14:18:46.988 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-25T14:18:46.988 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-25T14:18:46.988 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-25T14:18:46.988 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-25T14:18:46.988 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-25T14:18:46.988 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-25T14:18:46.988 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-25T14:18:46.988 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-25T14:18:46.988 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-25T14:18:46.988 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-25T14:18:46.988 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-25T14:18:46.988 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-25T14:18:46.988 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-25T14:18:46.989 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-25T14:18:46.989 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-25T14:18:46.989 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-25T14:18:46.989 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-25T14:18:46.989 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-25T14:18:46.989 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-25T14:18:46.989 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-25T14:18:46.989 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-25T14:18:46.989 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-25T14:18:46.989 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-25T14:18:46.989 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-25T14:18:46.989 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-25T14:18:46.989 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-25T14:18:46.989 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-25T14:18:46.989 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-25T14:18:46.990 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-25T14:18:46.990 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-25T14:18:46.990 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-25T14:18:46.990 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-25T14:18:46.990 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-25T14:18:46.990 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-25T14:18:46.990 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-25T14:18:46.990 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-25T14:18:46.990 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-25T14:18:46.990 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-25T14:18:46.990 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-25T14:18:46.990 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-25T14:18:46.990 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-25T14:18:46.990 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-25T14:18:46.991 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-25T14:18:46.991 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-25T14:18:46.991 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-25T14:18:46.991 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-25T14:18:46.991 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-25T14:18:46.991 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-25T14:18:46.991 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-25T14:18:46.991 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-25T14:18:46.991 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-25T14:18:46.991 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-25T14:18:46.991 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-25T14:18:46.991 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-25T14:18:46.991 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-25T14:18:46.991 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-25T14:18:46.991 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-25T14:18:46.992 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-25T14:18:46.992 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-25T14:18:46.992 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-25T14:18:46.992 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-25T14:18:46.992 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-25T14:18:46.992 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-25T14:18:46.992 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-25T14:18:46.992 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-25T14:18:46.992 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-25T14:18:46.992 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-25T14:18:46.992 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-25T14:18:46.992 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-25T14:18:46.992 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-25T14:18:46.992 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-25T14:18:46.993 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-25T14:18:46.993 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-25T14:18:46.993 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-25T14:18:46.993 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-25T14:18:46.993 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-25T14:18:46.993 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-25T14:18:46.993 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-25T14:18:46.993 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-25T14:18:46.993 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-25T14:18:46.993 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-25T14:18:46.993 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-25T14:18:46.993 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-25T14:18:46.993 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-25T14:18:46.993 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-25T14:18:46.994 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-25T14:18:46.994 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-25T14:18:46.994 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-25T14:18:46.994 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-25T14:18:46.994 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-25T14:18:46.994 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-25T14:18:46.994 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-25T14:18:46.994 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-25T14:18:46.994 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-25T14:18:46.994 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-25T14:18:46.994 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-25T14:18:46.994 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-25T14:18:46.994 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-25T14:18:46.994 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-25T14:18:46.994 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-25T14:18:46.995 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-25T14:18:46.995 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-25T14:18:46.995 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-25T14:18:46.995 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-25T14:18:46.995 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-25T14:18:46.995 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-25T14:18:46.995 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-25T14:18:46.995 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-25T14:18:46.995 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-25T14:18:46.995 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-25T14:18:46.995 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-25T14:18:46.995 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-25T14:18:46.995 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-25T14:18:46.995 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-25T14:18:46.995 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-25T14:18:46.996 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-25T14:18:46.996 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-25T14:18:46.996 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-25T14:18:46.996 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-25T14:18:46.996 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-25T14:18:46.996 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-25T14:18:46.996 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-25T14:18:46.996 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-25T14:18:46.996 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-25T14:18:46.996 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-25T14:18:46.996 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-25T14:18:46.996 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-25T14:18:46.996 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-25T14:18:46.996 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-25T14:18:46.996 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-25T14:18:46.996 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-25T14:18:46.997 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-25T14:18:46.997 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-25T14:18:46.997 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-25T14:18:46.997 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-25T14:18:46.997 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-25T14:18:46.997 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-25T14:18:46.997 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-25T14:18:46.997 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-25T14:18:46.997 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-25T14:18:46.997 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-25T14:18:46.997 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-25T14:18:46.997 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-25T14:18:46.997 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-25T14:18:46.997 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-25T14:18:46.998 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-25T14:18:46.998 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-25T14:18:46.998 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-25T14:18:46.998 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-25T14:18:46.998 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-25T14:18:46.998 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-25T14:18:46.998 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-25T14:18:46.998 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-25T14:18:46.998 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-25T14:18:46.998 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-25T14:18:46.998 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-25T14:18:46.998 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-25T14:18:46.998 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-25T14:18:46.998 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-25T14:18:46.999 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-25T14:18:46.999 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-25T14:18:46.999 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-25T14:18:46.999 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-25T14:18:46.999 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-25T14:18:46.999 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-25T14:18:46.999 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-25T14:18:46.999 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-25T14:18:46.999 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-25T14:18:46.999 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-25T14:18:46.999 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-25T14:18:46.999 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-25T14:18:46.999 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-25T14:18:46.999 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-25T14:18:47.000 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-25T14:18:47.000 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-25T14:18:47.000 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-25T14:18:47.000 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-25T14:18:47.000 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-25T14:18:47.000 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-25T14:18:47.000 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-25T14:18:47.000 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-25T14:18:47.000 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-25T14:18:47.000 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-25T14:18:47.000 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-25T14:18:47.000 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-25T14:18:47.000 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-25T14:18:47.000 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-25T14:18:47.000 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-25T14:18:47.001 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-25T14:18:47.001 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-25T14:18:47.001 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-25T14:18:47.001 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-25T14:18:47.001 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-25T14:18:47.001 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-25T14:18:47.001 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-25T14:18:47.001 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-25T14:18:47.001 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-25T14:18:47.001 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-25T14:18:47.001 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-25T14:18:47.001 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-25T14:18:47.001 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-25T14:18:47.001 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-25T14:18:47.002 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-25T14:18:47.002 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-25T14:18:47.002 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-25T14:18:47.002 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-25T14:18:47.002 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-25T14:18:47.002 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-25T14:18:47.002 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-25T14:18:47.002 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-25T14:18:47.002 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-25T14:18:47.002 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-25T14:18:47.002 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-25T14:18:47.002 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-25T14:18:47.002 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-25T14:18:47.002 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-25T14:18:47.003 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-25T14:18:47.003 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-25T14:18:47.003 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-25T14:18:47.003 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-25T14:18:47.003 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-25T14:18:47.003 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-25T14:18:47.003 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-25T14:18:47.003 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-25T14:18:47.003 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-25T14:18:47.003 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-25T14:18:47.003 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-25T14:18:47.003 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-25T14:18:47.003 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-25T14:18:47.003 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-25T14:18:47.003 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-25T14:18:47.003 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-25T14:18:47.004 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-25T14:18:47.004 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-25T14:18:47.004 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-25T14:18:47.004 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-25T14:18:47.004 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-25T14:18:47.004 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-25T14:18:47.004 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-25T14:18:47.004 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-25T14:18:47.004 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-25T14:18:47.004 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-25T14:18:47.004 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-25T14:18:47.004 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-25T14:18:47.004 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-25T14:18:47.004 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-25T14:18:47.004 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-25T14:18:47.004 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-25T14:18:47.005 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-25T14:18:47.005 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-25T14:18:47.005 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-25T14:18:47.005 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-25T14:18:47.005 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-25T14:18:47.005 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-25T14:18:47.005 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-25T14:18:47.005 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-25T14:18:47.005 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-25T14:18:47.005 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-25T14:18:47.005 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-25T14:18:47.005 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-25T14:18:47.005 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-25T14:18:47.005 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-25T14:18:47.005 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-25T14:18:47.006 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-25T14:18:47.006 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-25T14:18:47.006 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-25T14:18:47.006 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-25T14:18:47.006 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-25T14:18:47.006 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-25T14:18:47.006 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-25T14:18:47.006 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-25T14:18:47.006 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-25T14:18:47.006 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-25T14:18:47.006 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-25T14:18:47.006 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-25T14:18:47.006 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-25T14:18:47.006 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-25T14:18:47.007 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-25T14:18:47.007 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-25T14:18:47.007 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-25T14:18:47.007 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-25T14:18:47.007 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-25T14:18:47.007 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-25T14:18:47.007 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-25T14:18:47.007 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-25T14:18:47.007 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-25T14:18:47.007 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-25T14:18:47.007 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-25T14:18:47.007 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-25T14:18:47.007 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-25T14:18:47.007 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-25T14:18:47.008 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-25T14:18:47.008 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-25T14:18:47.008 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-25T14:18:47.008 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-25T14:18:47.008 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-25T14:18:47.008 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-25T14:18:47.008 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-25T14:18:47.008 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-25T14:18:47.008 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-25T14:18:47.008 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-25T14:18:47.008 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-25T14:18:47.008 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-25T14:18:47.008 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-25T14:18:47.008 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-25T14:18:47.008 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-25T14:18:47.009 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-25T14:18:47.009 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-25T14:18:47.009 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-25T14:18:47.009 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-25T14:18:47.009 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-25T14:18:47.009 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-25T14:18:47.009 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-25T14:18:47.009 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-25T14:18:47.009 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-25T14:18:47.009 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-25T14:18:47.009 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-25T14:18:47.009 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-25T14:18:47.009 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-25T14:18:47.009 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-25T14:18:47.009 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-25T14:18:47.010 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-25T14:18:47.010 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-25T14:18:47.010 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-25T14:18:47.010 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-25T14:18:47.010 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-25T14:18:47.010 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-25T14:18:47.010 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-25T14:18:47.010 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-25T14:18:47.010 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-25T14:18:47.010 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-25T14:18:47.010 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-25T14:18:47.010 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-25T14:18:47.010 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-25T14:18:47.010 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-25T14:18:47.011 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-25T14:18:47.011 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-25T14:18:47.011 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-25T14:18:47.011 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-25T14:18:47.011 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-25T14:18:47.011 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-25T14:18:47.011 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-25T14:18:47.011 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-25T14:18:47.011 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-25T14:18:47.011 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-25T14:18:47.011 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-25T14:18:47.011 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-25T14:18:47.011 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-25T14:18:47.011 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-25T14:18:47.011 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-25T14:18:47.012 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-25T14:18:47.012 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-25T14:18:47.012 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-25T14:18:47.012 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-25T14:18:47.012 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-25T14:18:47.012 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-25T14:18:47.012 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-25T14:18:47.012 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-25T14:18:47.012 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-25T14:18:47.012 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-25T14:18:47.012 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-25T14:18:47.012 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-25T14:18:47.012 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-25T14:18:47.012 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-25T14:18:47.012 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-25T14:18:47.013 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-25T14:18:47.013 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-25T14:18:47.013 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-25T14:18:47.013 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-25T14:18:47.013 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-25T14:18:47.013 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-25T14:18:47.013 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-25T14:18:47.013 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-25T14:18:47.013 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-25T14:18:47.013 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-25T14:18:47.013 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-25T14:18:47.013 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-25T14:18:47.013 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-25T14:18:47.013 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-25T14:18:47.013 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-25T14:18:47.014 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-25T14:18:47.014 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-25T14:18:47.014 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-25T14:18:47.014 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-25T14:18:47.014 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-25T14:18:47.014 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-25T14:18:47.014 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-25T14:18:47.014 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-25T14:18:47.014 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-25T14:18:47.014 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-25T14:18:47.014 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-25T14:18:47.014 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-25T14:18:47.014 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-25T14:18:47.014 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-25T14:18:47.015 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-25T14:18:47.015 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-25T14:18:47.015 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-25T14:18:47.015 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-25T14:18:47.015 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-25T14:18:47.015 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-25T14:18:47.015 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-25T14:18:47.015 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-25T14:18:47.015 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-25T14:18:47.015 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-25T14:18:47.015 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-25T14:18:47.015 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-25T14:18:47.015 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-25T14:18:47.015 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-25T14:18:47.016 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-25T14:18:47.016 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-25T14:18:47.016 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-25T14:18:47.016 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-25T14:18:47.016 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-25T14:18:47.016 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-25T14:18:47.016 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-25T14:18:47.016 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-25T14:18:47.016 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-25T14:18:47.016 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-25T14:18:47.016 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-25T14:18:47.016 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-25T14:18:47.016 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-25T14:18:47.016 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-25T14:18:47.016 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-25T14:18:47.017 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-25T14:18:47.017 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-25T14:18:47.017 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-25T14:18:47.017 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-25T14:18:47.017 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-25T14:18:47.017 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-25T14:18:47.017 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-25T14:18:47.017 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-25T14:18:47.017 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-25T14:18:47.017 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-25T14:18:47.017 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-25T14:18:47.017 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-25T14:18:47.017 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-25T14:18:47.017 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-25T14:18:47.017 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-25T14:18:47.018 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-25T14:18:47.018 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-25T14:18:47.018 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-25T14:18:47.018 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-25T14:18:47.018 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-25T14:18:47.018 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-25T14:18:47.018 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-25T14:18:47.018 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-25T14:18:47.018 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-25T14:18:47.018 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-25T14:18:47.018 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-25T14:18:47.018 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-25T14:18:47.018 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-25T14:18:47.018 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-25T14:18:47.019 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-25T14:18:47.019 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-25T14:18:47.019 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-25T14:18:47.019 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-25T14:18:47.019 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-25T14:18:47.019 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-25T14:18:47.019 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-25T14:18:47.019 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-25T14:18:47.019 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-25T14:18:47.019 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-25T14:18:47.019 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-25T14:18:47.019 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-25T14:18:47.019 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-25T14:18:47.019 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-25T14:18:47.020 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-25T14:18:47.020 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-25T14:18:47.020 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-25T14:18:47.020 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-25T14:18:47.020 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-25T14:18:47.020 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-25T14:18:47.020 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-25T14:18:47.020 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-25T14:18:47.020 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-25T14:18:47.020 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-25T14:18:47.020 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-25T14:18:47.020 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-25T14:18:47.020 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-25T14:18:47.020 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-25T14:18:47.020 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-25T14:18:47.021 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-25T14:18:47.021 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-25T14:18:47.021 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-25T14:18:47.021 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-25T14:18:47.021 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-25T14:18:47.021 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-25T14:18:47.021 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-25T14:18:47.021 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-25T14:18:47.021 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-25T14:18:47.021 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-25T14:18:47.021 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-25T14:18:47.021 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-25T14:18:47.021 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-25T14:18:47.021 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-25T14:18:47.022 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-25T14:18:47.022 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-25T14:18:47.022 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-25T14:18:47.022 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-25T14:18:47.022 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-25T14:18:47.022 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-25T14:18:47.022 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-25T14:18:47.022 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-25T14:18:47.022 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-25T14:18:47.022 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-25T14:18:47.022 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-25T14:18:47.022 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-25T14:18:47.022 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-25T14:18:47.022 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-25T14:18:47.023 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-25T14:18:47.023 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-25T14:18:47.023 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-25T14:18:47.023 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-25T14:18:47.023 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-25T14:18:47.023 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-25T14:18:47.023 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-25T14:18:47.023 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-25T14:18:47.023 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-25T14:18:47.023 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-25T14:18:47.023 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-25T14:18:47.023 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-25T14:18:47.023 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-25T14:18:47.023 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-25T14:18:47.024 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-25T14:18:47.024 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-25T14:18:47.024 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-25T14:18:47.024 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-25T14:18:47.024 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-25T14:18:47.024 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-25T14:18:47.024 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-25T14:18:47.024 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-25T14:18:47.024 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-25T14:18:47.024 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-25T14:18:47.024 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-25T14:18:47.024 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-25T14:18:47.024 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-25T14:18:47.024 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-25T14:18:47.025 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-25T14:18:47.025 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-25T14:18:47.025 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-25T14:18:47.025 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-25T14:18:47.025 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-25T14:18:47.025 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-25T14:18:47.025 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-25T14:18:47.025 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-25T14:18:47.025 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-25T14:18:47.025 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-25T14:18:47.025 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-25T14:18:47.025 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-25T14:18:47.025 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-25T14:18:47.025 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-25T14:18:47.026 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-25T14:18:47.026 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-25T14:18:47.026 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-25T14:18:47.026 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-25T14:18:47.026 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-25T14:18:47.026 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-25T14:18:47.026 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-25T14:18:47.026 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-25T14:18:47.026 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-25T14:18:47.026 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-25T14:18:47.026 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-25T14:18:47.026 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-25T14:18:47.026 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-25T14:18:47.026 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-25T14:18:47.027 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-25T14:18:47.027 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-25T14:18:47.027 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-25T14:18:47.027 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-25T14:18:47.027 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-25T14:18:47.027 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-25T14:18:47.027 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-25T14:18:47.027 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-25T14:18:47.027 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-25T14:18:47.027 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-25T14:18:47.027 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-25T14:18:47.027 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-25T14:18:47.027 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-25T14:18:47.027 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-25T14:18:47.028 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-25T14:18:47.028 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-25T14:18:47.028 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-25T14:18:47.028 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-25T14:18:47.028 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-25T14:18:47.028 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-25T14:18:47.028 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-25T14:18:47.028 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-25T14:18:47.028 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-25T14:18:47.028 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-25T14:18:47.028 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-25T14:18:47.028 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-25T14:18:47.028 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-25T14:18:47.028 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-25T14:18:47.028 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-25T14:18:47.029 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-25T14:18:47.029 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-25T14:18:47.029 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-25T14:18:47.029 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-25T14:18:47.029 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-25T14:18:47.029 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-25T14:18:47.029 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-25T14:18:47.029 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-25T14:18:47.029 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-25T14:18:47.029 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-25T14:18:47.029 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-25T14:18:47.029 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-25T14:18:47.029 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-25T14:18:47.029 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-25T14:18:47.030 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-25T14:18:47.030 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-25T14:18:47.030 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-25T14:18:47.030 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-25T14:18:47.030 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-25T14:18:47.030 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-25T14:18:47.030 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-25T14:18:47.030 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-25T14:18:47.030 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-25T14:18:47.030 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-25T14:18:47.030 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-25T14:18:47.030 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-25T14:18:47.030 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-25T14:18:47.030 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-25T14:18:47.030 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-25T14:18:47.031 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-25T14:18:47.031 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-25T14:18:47.031 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-25T14:18:47.031 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-25T14:18:47.031 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-25T14:18:47.031 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-25T14:18:47.031 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-25T14:18:47.031 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-25T14:18:47.031 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-25T14:18:47.031 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-25T14:18:47.031 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-25T14:18:47.031 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-25T14:18:47.031 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-25T14:18:47.031 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-25T14:18:47.031 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-25T14:18:47.032 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-25T14:18:47.032 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-25T14:18:47.032 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-25T14:18:47.032 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-25T14:18:47.032 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-25T14:18:47.032 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-25T14:18:47.032 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-25T14:18:47.032 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-25T14:18:47.032 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-25T14:18:47.032 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-25T14:18:47.032 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-25T14:18:47.032 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-25T14:18:47.032 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-25T14:18:47.032 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-25T14:18:47.033 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-25T14:18:47.033 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-25T14:18:47.033 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-25T14:18:47.033 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-25T14:18:47.033 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-25T14:18:47.033 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-25T14:18:47.033 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-25T14:18:47.033 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-25T14:18:47.033 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-25T14:18:47.033 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-25T14:18:47.033 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-25T14:18:47.033 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-25T14:18:47.033 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-25T14:18:47.033 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-25T14:18:47.033 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-25T14:18:47.034 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-25T14:18:47.034 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-25T14:18:47.034 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-25T14:18:47.034 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-25T14:18:47.034 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-25T14:18:47.034 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-25T14:18:47.034 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-25T14:18:47.034 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-25T14:18:47.034 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-25T14:18:47.034 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-25T14:18:47.034 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-25T14:18:47.034 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-25T14:18:47.034 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-25T14:18:47.035 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-25T14:18:47.035 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-25T14:18:47.035 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-25T14:18:47.035 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-25T14:18:47.035 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-25T14:18:47.035 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-25T14:18:47.035 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-25T14:18:47.035 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-25T14:18:47.035 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-25T14:18:47.035 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-25T14:18:47.035 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-25T14:18:47.035 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-25T14:18:47.035 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-25T14:18:47.035 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-25T14:18:47.036 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-25T14:18:47.036 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-25T14:18:47.036 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-25T14:18:47.036 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-25T14:18:47.036 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-25T14:18:47.036 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-25T14:18:47.036 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-25T14:18:47.036 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-25T14:18:47.036 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-25T14:18:47.036 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-25T14:18:47.036 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-25T14:18:47.036 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-25T14:18:47.036 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-25T14:18:47.036 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-25T14:18:47.037 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-25T14:18:47.037 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-25T14:18:47.037 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-25T14:18:47.037 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-25T14:18:47.037 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-25T14:18:47.037 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-25T14:18:47.037 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-25T14:18:47.037 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-25T14:18:47.037 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-25T14:18:47.037 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-25T14:18:47.037 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-25T14:18:47.037 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-25T14:18:47.037 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-25T14:18:47.037 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-25T14:18:47.037 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-25T14:18:47.038 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-25T14:18:47.038 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-25T14:18:47.038 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-25T14:18:47.038 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-25T14:18:47.038 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-25T14:18:47.038 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-25T14:18:47.038 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-25T14:18:47.038 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-25T14:18:47.038 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-25T14:18:47.038 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-25T14:18:47.038 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-25T14:18:47.038 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-25T14:18:47.038 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-25T14:18:47.038 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-25T14:18:47.039 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-25T14:18:47.039 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-25T14:18:47.039 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-25T14:18:47.039 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-25T14:18:47.039 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-25T14:18:47.039 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-25T14:18:47.039 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-25T14:18:47.039 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-25T14:18:47.039 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-25T14:18:47.039 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-25T14:18:47.039 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-25T14:18:47.039 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-25T14:18:47.039 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-25T14:18:47.039 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-25T14:18:47.039 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-25T14:18:47.040 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-25T14:18:47.040 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-25T14:18:47.040 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-25T14:18:47.040 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-25T14:18:47.040 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-25T14:18:47.040 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-25T14:18:47.040 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-25T14:18:47.040 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-25T14:18:47.040 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-25T14:18:47.040 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-25T14:18:47.040 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-25T14:18:47.040 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-25T14:18:47.040 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-25T14:18:47.040 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-25T14:18:47.040 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-25T14:18:47.041 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-25T14:18:47.041 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-25T14:18:47.041 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-25T14:18:47.041 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-25T14:18:47.041 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-25T14:18:47.041 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-25T14:18:47.041 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-25T14:18:47.041 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-25T14:18:47.041 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-25T14:18:47.041 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-25T14:18:47.041 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-25T14:18:47.041 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-25T14:18:47.041 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-25T14:18:47.041 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-25T14:18:47.041 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-25T14:18:47.042 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-25T14:18:47.042 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-25T14:18:47.042 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-25T14:18:47.042 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-25T14:18:47.042 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-25T14:18:47.042 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-25T14:18:47.042 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-25T14:18:47.042 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-25T14:18:47.042 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-25T14:18:47.042 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-25T14:18:47.042 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-25T14:18:47.042 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-25T14:18:47.042 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-25T14:18:47.042 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-25T14:18:47.043 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-25T14:18:47.043 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-25T14:18:47.043 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-25T14:18:47.043 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-25T14:18:47.043 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-25T14:18:47.043 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-25T14:18:47.043 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-25T14:18:47.043 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-25T14:18:47.043 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-25T14:18:47.043 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-25T14:18:47.043 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-25T14:18:47.043 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-25T14:18:47.043 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-25T14:18:47.043 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-25T14:18:47.044 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-25T14:18:47.044 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-25T14:18:47.044 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-25T14:18:47.044 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-25T14:18:47.044 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-25T14:18:47.044 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-25T14:18:47.044 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-25T14:18:47.044 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-25T14:18:47.044 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-25T14:18:47.044 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-25T14:18:47.044 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-25T14:18:47.044 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-25T14:18:47.044 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-25T14:18:47.044 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-25T14:18:47.044 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-25T14:18:47.045 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-25T14:18:47.045 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-25T14:18:47.045 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-25T14:18:47.045 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-25T14:18:47.045 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-25T14:18:47.045 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-25T14:18:47.045 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-25T14:18:47.045 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-25T14:18:47.045 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-25T14:18:47.045 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-25T14:18:47.045 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-25T14:18:47.045 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-25T14:18:47.045 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-25T14:18:47.045 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-25T14:18:47.045 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-25T14:18:47.046 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-25T14:18:47.046 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-25T14:18:47.046 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-25T14:18:47.046 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-25T14:18:47.046 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-25T14:18:47.046 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-25T14:18:47.046 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-25T14:18:47.046 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-25T14:18:47.046 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-25T14:18:47.046 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-25T14:18:47.046 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-25T14:18:47.046 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-25T14:18:47.046 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-25T14:18:47.046 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-25T14:18:47.047 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-25T14:18:47.047 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-25T14:18:47.047 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-25T14:18:47.047 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-25T14:18:47.047 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-25T14:18:47.047 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-25T14:18:47.047 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-25T14:18:47.047 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-25T14:18:47.047 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-25T14:18:47.047 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-25T14:18:47.047 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-25T14:18:47.047 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-25T14:18:47.047 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-25T14:18:47.047 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-25T14:18:47.047 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-25T14:18:47.048 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-25T14:18:47.048 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-25T14:18:47.048 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-25T14:18:47.048 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-25T14:18:47.048 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-25T14:18:47.048 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-25T14:18:47.048 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-25T14:18:47.048 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-25T14:18:47.048 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-25T14:18:47.048 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-25T14:18:47.048 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-25T14:18:47.048 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-25T14:18:47.048 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-25T14:18:47.048 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-25T14:18:47.048 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-25T14:18:47.049 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-25T14:18:47.049 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-25T14:18:47.049 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-25T14:18:47.049 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-25T14:18:47.049 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-25T14:18:47.049 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-25T14:18:47.049 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-25T14:18:47.049 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-25T14:18:47.049 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-25T14:18:47.049 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-25T14:18:47.049 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-25T14:18:47.049 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-25T14:18:47.049 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-25T14:18:47.049 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-25T14:18:47.049 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-25T14:18:47.050 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-25T14:18:47.050 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-25T14:18:47.050 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-25T14:18:47.050 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-25T14:18:47.050 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-25T14:18:47.050 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-25T14:18:47.050 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-25T14:18:47.050 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-25T14:18:47.050 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-25T14:18:47.050 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-25T14:18:47.050 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-25T14:18:47.050 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-25T14:18:47.050 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-25T14:18:47.050 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-25T14:18:47.051 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-25T14:18:47.051 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-25T14:18:47.051 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-25T14:18:47.051 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-25T14:18:47.051 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-25T14:18:47.051 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-25T14:18:47.051 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-25T14:18:47.051 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-25T14:18:47.051 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-25T14:18:47.051 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-25T14:18:47.051 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-25T14:18:47.051 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-25T14:18:47.051 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-25T14:18:47.051 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-25T14:18:47.051 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-25T14:18:47.052 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-25T14:18:47.052 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-25T14:18:47.052 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-25T14:18:47.052 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-25T14:18:47.052 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-25T14:18:47.052 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-25T14:18:47.052 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-25T14:18:47.052 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-25T14:18:47.052 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-25T14:18:47.052 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-25T14:18:47.052 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-25T14:18:47.052 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-25T14:18:47.052 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-25T14:18:47.052 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-25T14:18:47.052 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-25T14:18:47.053 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-25T14:18:47.053 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-25T14:18:47.053 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-25T14:18:47.053 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-25T14:18:47.053 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-25T14:18:47.053 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-25T14:18:47.053 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-25T14:18:47.053 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-25T14:18:47.053 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-25T14:18:47.053 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-25T14:18:47.053 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-25T14:18:47.053 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-25T14:18:47.053 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-25T14:18:47.053 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-25T14:18:47.054 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-25T14:18:47.054 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-25T14:18:47.054 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-25T14:18:47.054 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-25T14:18:47.054 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-25T14:18:47.054 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-25T14:18:47.054 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-25T14:18:47.054 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-25T14:18:47.054 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-25T14:18:47.054 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-25T14:18:47.054 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-25T14:18:47.054 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-25T14:18:47.054 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-25T14:18:47.054 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-25T14:18:47.055 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-25T14:18:47.055 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-25T14:18:47.055 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-25T14:18:47.055 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-25T14:18:47.055 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-25T14:18:47.055 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-25T14:18:47.055 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-25T14:18:47.055 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-25T14:18:47.055 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-25T14:18:47.055 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-25T14:18:47.055 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-25T14:18:47.055 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-25T14:18:47.055 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-25T14:18:47.055 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-25T14:18:47.055 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-25T14:18:47.056 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-25T14:18:47.056 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-25T14:18:47.056 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-25T14:18:47.056 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-25T14:18:47.056 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-25T14:18:47.056 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-25T14:18:47.056 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-25T14:18:47.056 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-25T14:18:47.056 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-25T14:18:47.056 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-25T14:18:47.056 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-25T14:18:47.056 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-25T14:18:47.056 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-25T14:18:47.056 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-25T14:18:47.056 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-25T14:18:47.057 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-25T14:18:47.057 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-25T14:18:47.057 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-25T14:18:47.057 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-25T14:18:47.057 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-25T14:18:47.057 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-25T14:18:47.057 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-25T14:18:47.057 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-25T14:18:47.057 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-25T14:18:47.057 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-25T14:18:47.057 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-25T14:18:47.057 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-25T14:18:47.057 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-25T14:18:47.057 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-25T14:18:47.057 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-25T14:18:47.058 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-25T14:18:47.058 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-25T14:18:47.058 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-25T14:18:47.058 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-25T14:18:47.058 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-25T14:18:47.058 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-25T14:18:47.058 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-25T14:18:47.058 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-25T14:18:47.058 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-25T14:18:47.058 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-25T14:18:47.058 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-25T14:18:47.058 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-25T14:18:47.058 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-25T14:18:47.058 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-25T14:18:47.059 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-25T14:18:47.059 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-25T14:18:47.059 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-25T14:18:47.059 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-25T14:18:47.059 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-25T14:18:47.059 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-25T14:18:47.059 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-25T14:18:47.059 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-25T14:18:47.059 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-25T14:18:47.059 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-25T14:18:47.059 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-25T14:18:47.059 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-25T14:18:47.059 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-25T14:18:47.059 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-25T14:18:47.060 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-25T14:18:47.060 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-25T14:18:47.060 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-25T14:18:47.060 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-25T14:18:47.060 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-25T14:18:47.060 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-25T14:18:47.060 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-25T14:18:47.060 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-25T14:18:47.060 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-25T14:18:47.060 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-25T14:18:47.060 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-25T14:18:47.060 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-25T14:18:47.060 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-25T14:18:47.060 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-25T14:18:47.060 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-25T14:18:47.061 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-25T14:18:47.061 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-25T14:18:47.061 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-25T14:18:47.061 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-25T14:18:47.061 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-25T14:18:47.061 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-25T14:18:47.061 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-25T14:18:47.061 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-25T14:18:47.061 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-25T14:18:47.061 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-25T14:18:47.061 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-25T14:18:47.061 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-25T14:18:47.061 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-25T14:18:47.061 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-25T14:18:47.062 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-25T14:18:47.062 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-25T14:18:47.062 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-25T14:18:47.062 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-25T14:18:47.062 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-25T14:18:47.062 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-25T14:18:47.062 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-25T14:18:47.062 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-25T14:18:47.062 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-25T14:18:47.062 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-25T14:18:47.062 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-25T14:18:47.062 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-25T14:18:47.062 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-25T14:18:47.062 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-25T14:18:47.063 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-25T14:18:47.063 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-25T14:18:47.063 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-25T14:18:47.063 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-25T14:18:47.063 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-25T14:18:47.063 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-25T14:18:47.063 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-25T14:18:47.063 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-25T14:18:47.063 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-25T14:18:47.063 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-25T14:18:47.063 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-25T14:18:47.063 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-25T14:18:47.063 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-25T14:18:47.063 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-25T14:18:47.063 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-25T14:18:47.064 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-25T14:18:47.064 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-25T14:18:47.064 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-25T14:18:47.064 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-25T14:18:47.064 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-25T14:18:47.064 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-25T14:18:47.064 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-25T14:18:47.064 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-25T14:18:47.064 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-25T14:18:47.064 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-25T14:18:47.064 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-25T14:18:47.064 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-25T14:18:47.064 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-25T14:18:47.064 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-25T14:18:47.065 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-25T14:18:47.065 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-25T14:18:47.065 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-25T14:18:47.065 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-25T14:18:47.065 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-25T14:18:47.065 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-25T14:18:47.065 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-25T14:18:47.065 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-25T14:18:47.065 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-25T14:18:47.065 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-25T14:18:47.065 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-25T14:18:47.065 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-25T14:18:47.065 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-25T14:18:47.065 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-25T14:18:47.065 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-25T14:18:47.066 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-25T14:18:47.066 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-25T14:18:47.066 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-25T14:18:47.066 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-25T14:18:47.066 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-25T14:18:47.066 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-25T14:18:47.066 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-25T14:18:47.066 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-25T14:18:47.066 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-25T14:18:47.066 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-25T14:18:47.066 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-25T14:18:47.066 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-25T14:18:47.066 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-25T14:18:47.067 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-25T14:18:47.067 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-25T14:18:47.067 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-25T14:18:47.067 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-25T14:18:47.067 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-25T14:18:47.067 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-25T14:18:47.067 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-25T14:18:47.067 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-25T14:18:47.067 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-25T14:18:47.067 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-25T14:18:47.067 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-25T14:18:47.067 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-25T14:18:47.067 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-25T14:18:47.067 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-25T14:18:47.068 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-25T14:18:47.068 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-25T14:18:47.068 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-25T14:18:47.068 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-25T14:18:47.068 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-25T14:18:47.068 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-25T14:18:47.068 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-25T14:18:47.068 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-25T14:18:47.068 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-25T14:18:47.068 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-25T14:18:47.068 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-25T14:18:47.068 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-25T14:18:47.068 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-25T14:18:47.068 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-25T14:18:47.068 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-25T14:18:47.069 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-25T14:18:47.069 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-25T14:18:47.069 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-25T14:18:47.069 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-25T14:18:47.069 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-25T14:18:47.069 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-25T14:18:47.069 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-25T14:18:47.069 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-25T14:18:47.069 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-25T14:18:47.069 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-25T14:18:47.069 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-25T14:18:47.069 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-25T14:18:47.069 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-25T14:18:47.069 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-25T14:18:47.069 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-25T14:18:47.069 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-25T14:18:47.070 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-25T14:18:47.070 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-25T14:18:47.070 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-25T14:18:47.070 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-25T14:18:47.070 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-25T14:18:47.070 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-25T14:18:47.070 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-25T14:18:47.070 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-25T14:18:47.070 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-25T14:18:47.070 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-25T14:18:47.070 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-25T14:18:47.070 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-25T14:18:47.070 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-25T14:18:47.070 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-25T14:18:47.071 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-25T14:18:47.071 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-25T14:18:47.071 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-25T14:18:47.071 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-25T14:18:47.071 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-25T14:18:47.071 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-25T14:18:47.071 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-25T14:18:47.071 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-25T14:18:47.071 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-25T14:18:47.071 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-25T14:18:47.071 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-25T14:18:47.071 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-25T14:18:47.071 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-25T14:18:47.071 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-25T14:18:47.071 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-25T14:18:47.071 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-25T14:18:47.072 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-25T14:18:47.072 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-25T14:18:47.072 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-25T14:18:47.072 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-25T14:18:47.072 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-25T14:18:47.072 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-25T14:18:47.072 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-25T14:18:47.072 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-25T14:18:47.072 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-25T14:18:47.072 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-25T14:18:47.072 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-25T14:18:47.072 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-25T14:18:47.072 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-25T14:18:47.072 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-25T14:18:47.072 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-25T14:18:47.072 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-25T14:18:47.073 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-25T14:18:47.073 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-25T14:18:47.073 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-25T14:18:47.073 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-25T14:18:47.073 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-25T14:18:47.073 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-25T14:18:47.073 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-25T14:18:47.073 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-25T14:18:47.073 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-25T14:18:47.073 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-25T14:18:47.073 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-25T14:18:47.073 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-25T14:18:47.073 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-25T14:18:47.073 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-25T14:18:47.073 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-25T14:18:47.073 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-25T14:18:47.073 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-25T14:18:47.074 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-25T14:18:47.074 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-25T14:18:47.074 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-25T14:18:47.074 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-25T14:18:47.074 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-25T14:18:47.074 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-25T14:18:47.074 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-25T14:18:47.074 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-25T14:18:47.074 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-25T14:18:47.074 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-25T14:18:47.074 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-25T14:18:47.074 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-25T14:18:47.074 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-25T14:18:47.074 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-25T14:18:47.075 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-25T14:18:47.075 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-25T14:18:47.075 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-25T14:18:47.075 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-25T14:18:47.075 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-25T14:18:47.075 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-25T14:18:47.075 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-25T14:18:47.075 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-25T14:18:47.075 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-25T14:18:47.075 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-25T14:18:47.075 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-25T14:18:47.075 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-25T14:18:47.075 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-25T14:18:47.075 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-25T14:18:47.075 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-25T14:18:47.075 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-25T14:18:47.076 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-25T14:18:47.076 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-25T14:18:47.076 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-25T14:18:47.076 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-25T14:18:47.076 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-25T14:18:47.076 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-25T14:18:47.076 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-25T14:18:47.076 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-25T14:18:47.076 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-25T14:18:47.076 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-25T14:18:47.076 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-25T14:18:47.076 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-25T14:18:47.076 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-25T14:18:47.076 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-25T14:18:47.077 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-25T14:18:47.077 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-25T14:18:47.077 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-25T14:18:47.077 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-25T14:18:47.077 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-25T14:18:47.077 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-25T14:18:47.077 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-25T14:18:47.077 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-25T14:18:47.077 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-25T14:18:47.077 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-25T14:18:47.077 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-25T14:18:47.077 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-25T14:18:47.077 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-25T14:18:47.077 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-25T14:18:47.077 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-25T14:18:47.077 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-25T14:18:47.078 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-25T14:18:47.078 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-25T14:18:47.078 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-25T14:18:47.078 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-25T14:18:47.078 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-25T14:18:47.078 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-25T14:18:47.078 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-25T14:18:47.078 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-25T14:18:47.078 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-25T14:18:47.078 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-25T14:18:47.078 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-25T14:18:47.078 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-25T14:18:47.078 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-25T14:18:47.079 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-25T14:18:47.079 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-25T14:18:47.079 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-25T14:18:47.079 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-25T14:18:47.079 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-25T14:18:47.079 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-25T14:18:47.079 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-25T14:18:47.079 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-25T14:18:47.079 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-25T14:18:47.079 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-25T14:18:47.079 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-25T14:18:47.079 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-25T14:18:47.079 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-25T14:18:47.079 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-25T14:18:47.079 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-25T14:18:47.080 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-25T14:18:47.080 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-25T14:18:47.080 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-25T14:18:47.080 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-25T14:18:47.080 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-25T14:18:47.080 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-25T14:18:47.080 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-25T14:18:47.080 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-25T14:18:47.080 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-25T14:18:47.080 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-25T14:18:47.080 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-25T14:18:47.080 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-25T14:18:47.080 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-25T14:18:47.080 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-25T14:18:47.080 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-25T14:18:47.081 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-25T14:18:47.081 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-25T14:18:47.081 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-25T14:18:47.081 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-25T14:18:47.081 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-25T14:18:47.081 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-25T14:18:47.081 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-25T14:18:47.081 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-25T14:18:47.081 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-25T14:18:47.081 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-25T14:18:47.081 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-25T14:18:47.081 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-25T14:18:47.081 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-25T14:18:47.081 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-25T14:18:47.082 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-25T14:18:47.082 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-25T14:18:47.082 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-25T14:18:47.082 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-25T14:18:47.082 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-25T14:18:47.082 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-25T14:18:47.082 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-25T14:18:47.082 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-25T14:18:47.082 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-25T14:18:47.082 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-25T14:18:47.082 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-25T14:18:47.082 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-25T14:18:47.082 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-25T14:18:47.082 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-25T14:18:47.082 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-25T14:18:47.083 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-25T14:18:47.083 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-25T14:18:47.083 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-25T14:18:47.083 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-25T14:18:47.083 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-25T14:18:47.083 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-25T14:18:47.083 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-25T14:18:47.083 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-25T14:18:47.083 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-25T14:18:47.083 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-25T14:18:47.083 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-25T14:18:47.083 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-25T14:18:47.083 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-25T14:18:47.083 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-25T14:18:47.084 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-25T14:18:47.084 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-25T14:18:47.084 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-25T14:18:47.084 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-25T14:18:47.084 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-25T14:18:47.084 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-25T14:18:47.084 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-25T14:18:47.084 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-25T14:18:47.084 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-25T14:18:47.084 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-25T14:18:47.084 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-25T14:18:47.084 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-25T14:18:47.084 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-25T14:18:47.084 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-25T14:18:47.085 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-25T14:18:47.085 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-25T14:18:47.085 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-25T14:18:47.085 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-25T14:18:47.085 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-25T14:18:47.085 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-25T14:18:47.085 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-25T14:18:47.085 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-25T14:18:47.085 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-25T14:18:47.085 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-25T14:18:47.085 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-25T14:18:47.085 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-25T14:18:47.085 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-25T14:18:47.085 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-25T14:18:47.085 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-25T14:18:47.085 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-25T14:18:47.086 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-25T14:18:47.086 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-25T14:18:47.086 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-25T14:18:47.086 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-25T14:18:47.086 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-25T14:18:47.086 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-25T14:18:47.086 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-25T14:18:47.086 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-25T14:18:47.086 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-25T14:18:47.086 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-25T14:18:47.086 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-25T14:18:47.086 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-25T14:18:47.086 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-25T14:18:47.086 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-25T14:18:47.087 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-25T14:18:47.087 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-25T14:18:47.087 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-25T14:18:47.087 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-25T14:18:47.087 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-25T14:18:47.087 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-25T14:18:47.087 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-25T14:18:47.087 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-25T14:18:47.087 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-25T14:18:47.087 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-25T14:18:47.087 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-25T14:18:47.087 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-25T14:18:47.087 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-25T14:18:47.087 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-25T14:18:47.087 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-25T14:18:47.088 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-25T14:18:47.088 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-25T14:18:47.088 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-25T14:18:47.088 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-25T14:18:47.088 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-25T14:18:47.088 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-25T14:18:47.088 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-25T14:18:47.088 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-25T14:18:47.088 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-25T14:18:47.088 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-25T14:18:47.088 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-25T14:18:47.088 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-25T14:18:47.088 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-25T14:18:47.088 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-25T14:18:47.088 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-25T14:18:47.089 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-25T14:18:47.089 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-25T14:18:47.089 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-25T14:18:47.089 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-25T14:18:47.089 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-25T14:18:47.089 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-25T14:18:47.089 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-25T14:18:47.089 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-25T14:18:47.089 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-25T14:18:47.089 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-25T14:18:47.089 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-25T14:18:47.089 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-25T14:18:47.089 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-25T14:18:47.089 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-25T14:18:47.089 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-25T14:18:47.090 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-25T14:18:47.090 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-25T14:18:47.090 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-25T14:18:47.090 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-25T14:18:47.090 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-25T14:18:47.090 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-25T14:18:47.090 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-25T14:18:47.090 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-25T14:18:47.090 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-25T14:18:47.090 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-25T14:18:47.090 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-25T14:18:47.090 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-25T14:18:47.090 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-25T14:18:47.090 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-25T14:18:47.091 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-25T14:18:47.091 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-25T14:18:47.091 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-25T14:18:47.091 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-25T14:18:47.091 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-25T14:18:47.091 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-25T14:18:47.091 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-25T14:18:47.091 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-25T14:18:47.091 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-25T14:18:47.091 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-25T14:18:47.091 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-25T14:18:47.091 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-25T14:18:47.091 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-25T14:18:47.091 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-25T14:18:47.092 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-25T14:18:47.092 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-25T14:18:47.092 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-25T14:18:47.092 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-25T14:18:47.092 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-25T14:18:47.092 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-25T14:18:47.092 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-25T14:18:47.092 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-25T14:18:47.092 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-25T14:18:47.092 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-25T14:18:47.092 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-25T14:18:47.092 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-25T14:18:47.092 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-25T14:18:47.092 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-25T14:18:47.092 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-25T14:18:47.092 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-25T14:18:47.093 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-25T14:18:47.093 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-25T14:18:47.093 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-25T14:18:47.093 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-25T14:18:47.093 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-25T14:18:47.093 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-25T14:18:47.093 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-25T14:18:47.093 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-25T14:18:47.093 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-25T14:18:47.093 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-25T14:18:47.093 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-25T14:18:47.093 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-25T14:18:47.093 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-25T14:18:47.093 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-25T14:18:47.094 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-25T14:18:47.094 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-25T14:18:47.094 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-25T14:18:47.094 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-25T14:18:47.094 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-25T14:18:47.094 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-25T14:18:47.094 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-25T14:18:47.094 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-25T14:18:47.094 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-25T14:18:47.094 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-25T14:18:47.094 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-25T14:18:47.094 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-25T14:18:47.094 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-25T14:18:47.094 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-25T14:18:47.095 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-25T14:18:47.095 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-25T14:18:47.095 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-25T14:18:47.095 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-25T14:18:47.095 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-25T14:18:47.095 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-25T14:18:47.095 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-25T14:18:47.095 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-25T14:18:47.095 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-25T14:18:47.095 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-25T14:18:47.095 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-25T14:18:47.095 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-25T14:18:47.095 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-25T14:18:47.095 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-25T14:18:47.095 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-25T14:18:47.096 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-25T14:18:47.096 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-25T14:18:47.096 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-25T14:18:47.096 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-25T14:18:47.096 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-25T14:18:47.096 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-25T14:18:47.096 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-25T14:18:47.096 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-25T14:18:47.096 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-25T14:18:47.096 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-25T14:18:47.096 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-25T14:18:47.096 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-25T14:18:47.096 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-25T14:18:47.096 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-25T14:18:47.096 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-25T14:18:47.097 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-25T14:18:47.097 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-25T14:18:47.097 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-25T14:18:47.097 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-25T14:18:47.097 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-25T14:18:47.097 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-25T14:18:47.097 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-25T14:18:47.097 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-25T14:18:47.097 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-25T14:18:47.097 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-25T14:18:47.097 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-25T14:18:47.097 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-25T14:18:47.097 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-25T14:18:47.097 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-25T14:18:47.097 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-25T14:18:47.098 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-25T14:18:47.098 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-25T14:18:47.098 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-25T14:18:47.098 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-25T14:18:47.098 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-25T14:18:47.098 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-25T14:18:47.098 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-25T14:18:47.098 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-25T14:18:47.098 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-25T14:18:47.098 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-25T14:18:47.098 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-25T14:18:47.098 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-25T14:18:47.098 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-25T14:18:47.098 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-25T14:18:47.099 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-25T14:18:47.099 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-25T14:18:47.099 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-25T14:18:47.099 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-25T14:18:47.099 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-25T14:18:47.099 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-25T14:18:47.099 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-25T14:18:47.099 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-25T14:18:47.099 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-25T14:18:47.099 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-25T14:18:47.099 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-25T14:18:47.099 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-25T14:18:47.099 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-25T14:18:47.099 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-25T14:18:47.099 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-25T14:18:47.100 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-25T14:18:47.100 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-25T14:18:47.100 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-25T14:18:47.100 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-25T14:18:47.100 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-25T14:18:47.100 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-25T14:18:47.100 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-25T14:18:47.100 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-25T14:18:47.100 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-25T14:18:47.100 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-25T14:18:47.100 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-25T14:18:47.100 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-25T14:18:47.100 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-25T14:18:47.100 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-25T14:18:47.100 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-25T14:18:47.101 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-25T14:18:47.101 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-25T14:18:47.101 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-25T14:18:47.101 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-25T14:18:47.101 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-25T14:18:47.101 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T14:18:47.105 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-25T14:18:47.105 DEBUG:teuthology.orchestra.run.smithi053:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-25T14:18:47.129 INFO:teuthology.orchestra.run.smithi053.stdout:postinstall scriptlet (using /bin/sh): 2024-04-25T14:18:47.130 INFO:teuthology.orchestra.run.smithi053.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T14:18:47.130 INFO:teuthology.orchestra.run.smithi053.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T14:18:47.130 INFO:teuthology.orchestra.run.smithi053.stdout:fi 2024-04-25T14:18:47.130 INFO:teuthology.orchestra.run.smithi053.stdout:for file in vmlinuz System.map config; do 2024-04-25T14:18:47.130 INFO:teuthology.orchestra.run.smithi053.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-25T14:18:47.130 INFO:teuthology.orchestra.run.smithi053.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-25T14:18:47.130 INFO:teuthology.orchestra.run.smithi053.stdout: fi 2024-04-25T14:18:47.130 INFO:teuthology.orchestra.run.smithi053.stdout:done 2024-04-25T14:18:47.130 INFO:teuthology.orchestra.run.smithi053.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-25T14:18:47.130 INFO:teuthology.orchestra.run.smithi053.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-25T14:18:47.130 INFO:teuthology.orchestra.run.smithi053.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T14:18:47.130 INFO:teuthology.orchestra.run.smithi053.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T14:18:47.131 INFO:teuthology.orchestra.run.smithi053.stdout:fi 2024-04-25T14:18:47.131 INFO:teuthology.orchestra.run.smithi053.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-25T14:18:47.131 INFO:teuthology.orchestra.run.smithi053.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-25T14:18:47.131 INFO:teuthology.orchestra.run.smithi053.stdout:fi 2024-04-25T14:18:47.131 INFO:teuthology.orchestra.run.smithi053.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-25T14:18:47.131 INFO:teuthology.orchestra.run.smithi053.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T14:18:47.131 INFO:teuthology.orchestra.run.smithi053.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-25T14:18:47.131 INFO:teuthology.orchestra.run.smithi053.stdout:fi 2024-04-25T14:18:47.132 DEBUG:teuthology.orchestra.run.smithi053:> sudo rpm -qi grub2-tools 2024-04-25T14:18:47.206 INFO:teuthology.orchestra.run.smithi053.stdout:Name : grub2-tools 2024-04-25T14:18:47.206 INFO:teuthology.orchestra.run.smithi053.stdout:Epoch : 1 2024-04-25T14:18:47.206 INFO:teuthology.orchestra.run.smithi053.stdout:Version : 2.06 2024-04-25T14:18:47.207 INFO:teuthology.orchestra.run.smithi053.stdout:Release : 68.el9 2024-04-25T14:18:47.207 INFO:teuthology.orchestra.run.smithi053.stdout:Architecture: x86_64 2024-04-25T14:18:47.207 INFO:teuthology.orchestra.run.smithi053.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-25T14:18:47.207 INFO:teuthology.orchestra.run.smithi053.stdout:Group : Unspecified 2024-04-25T14:18:47.207 INFO:teuthology.orchestra.run.smithi053.stdout:Size : 8264254 2024-04-25T14:18:47.207 INFO:teuthology.orchestra.run.smithi053.stdout:License : GPLv3+ 2024-04-25T14:18:47.207 INFO:teuthology.orchestra.run.smithi053.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-25T14:18:47.207 INFO:teuthology.orchestra.run.smithi053.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-25T14:18:47.207 INFO:teuthology.orchestra.run.smithi053.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-25T14:18:47.207 INFO:teuthology.orchestra.run.smithi053.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-25T14:18:47.207 INFO:teuthology.orchestra.run.smithi053.stdout:Packager : builder@centos.org 2024-04-25T14:18:47.207 INFO:teuthology.orchestra.run.smithi053.stdout:Vendor : CentOS 2024-04-25T14:18:47.207 INFO:teuthology.orchestra.run.smithi053.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-25T14:18:47.207 INFO:teuthology.orchestra.run.smithi053.stdout:Summary : Support tools for GRUB. 2024-04-25T14:18:47.207 INFO:teuthology.orchestra.run.smithi053.stdout:Description : 2024-04-25T14:18:47.208 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:18:47.208 INFO:teuthology.orchestra.run.smithi053.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-25T14:18:47.208 INFO:teuthology.orchestra.run.smithi053.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-25T14:18:47.208 INFO:teuthology.orchestra.run.smithi053.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-25T14:18:47.208 INFO:teuthology.orchestra.run.smithi053.stdout:hardware devices. 2024-04-25T14:18:47.208 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:18:47.208 INFO:teuthology.orchestra.run.smithi053.stdout:This subpackage provides tools for support of all platforms. 2024-04-25T14:18:47.209 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-25T14:18:47.209 INFO:teuthology.task.kernel:Updating grub on smithi053 to boot 6.9.0-rc3-g806ec36b2112 2024-04-25T14:18:47.209 DEBUG:teuthology.orchestra.run.smithi053:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-25T14:18:47.774 INFO:teuthology.orchestra.run.smithi053.stderr:Generating grub configuration file ... 2024-04-25T14:18:49.236 INFO:teuthology.orchestra.run.smithi053.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-25T14:18:49.262 INFO:teuthology.orchestra.run.smithi053.stderr:done 2024-04-25T14:18:49.265 DEBUG:teuthology.orchestra.run.smithi053:> mktemp 2024-04-25T14:18:49.311 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/tmp.5Hmb1EpUBo 2024-04-25T14:18:49.312 DEBUG:teuthology.orchestra.run.smithi053:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.5Hmb1EpUBo 2024-04-25T14:18:49.387 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 0666 /tmp/tmp.5Hmb1EpUBo 2024-04-25T14:18:49.532 DEBUG:teuthology.orchestra.remote:smithi053:/tmp/tmp.5Hmb1EpUBo is 6KB 2024-04-25T14:18:49.555 DEBUG:teuthology.orchestra.run.smithi053:> rm -fr /tmp/tmp.5Hmb1EpUBo 2024-04-25T14:18:49.575 DEBUG:teuthology.orchestra.run.smithi053:> sudo /bin/ls /boot/loader/entries || true 2024-04-25T14:18:49.640 INFO:teuthology.orchestra.run.smithi053.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-25T14:18:49.640 INFO:teuthology.orchestra.run.smithi053.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-25T14:18:49.640 INFO:teuthology.orchestra.run.smithi053.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-25T14:18:49.640 INFO:teuthology.orchestra.run.smithi053.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-25T14:18:49.641 DEBUG:teuthology.orchestra.run.smithi053:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-25T14:18:49.827 DEBUG:teuthology.orchestra.run.smithi053:> sudo shutdown -r now 2024-04-25T14:19:19.831 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-25T14:19:19.832 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-04-25T14:19:19.833 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T14:20:19.835 DEBUG:teuthology.orchestra.remote:timed out 2024-04-25T14:20:28.837 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-04-25T14:20:28.838 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T14:20:29.304 DEBUG:teuthology.orchestra.run.smithi053:> true 2024-04-25T14:20:29.928 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-04-25T14:20:29.928 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-04-25T14:20:29.929 DEBUG:teuthology.orchestra.run.smithi053:> uname -r 2024-04-25T14:20:29.982 INFO:teuthology.orchestra.run.smithi053.stdout:6.9.0-rc3-g806ec36b2112 2024-04-25T14:20:29.982 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-25T14:20:29.982 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T14:20:29.982 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-25T14:20:30.985 DEBUG:teuthology.parallel:result is None 2024-04-25T14:20:30.985 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T14:20:30.992 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T14:20:30.993 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T14:20:30.999 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T14:20:31.002 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T14:20:31.044 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T14:20:31.051 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T14:20:31.060 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T14:20:31.061 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T14:20:31.094 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T14:20:31.096 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T14:20:31.160 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T14:20:31.168 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T14:20:31.168 DEBUG:teuthology.orchestra.run.smithi150:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T14:20:31.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:20:31.184 DEBUG:teuthology.orchestra.run.smithi183:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T14:20:31.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:20:31.201 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T14:20:31.205 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T14:20:31.228 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T14:20:31.273 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T14:20:31.293 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T14:20:31.299 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T14:20:31.318 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T14:20:31.337 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T14:20:31.441 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T14:20:31.446 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T14:20:31.455 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T14:20:31.456 DEBUG:teuthology.orchestra.run.smithi053:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T14:20:31.460 DEBUG:teuthology.orchestra.run.smithi150:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T14:20:31.462 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T14:20:31.524 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T14:20:31.533 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T14:20:31.533 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T14:20:31.537 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T14:20:31.540 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T14:20:31.576 DEBUG:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T14:20:31.697 DEBUG:teuthology.orchestra.run.smithi053:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T14:20:31.791 DEBUG:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T14:20:31.859 DEBUG:teuthology.orchestra.run.smithi053:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T14:20:31.947 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-25T14:20:31.947 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T14:20:32.022 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T14:20:32.063 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T14:20:32.120 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T14:20:32.201 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T14:20:32.248 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T14:20:32.248 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T14:20:32.314 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T14:20:32.359 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T14:20:32.419 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T14:20:32.463 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T14:20:32.505 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-25T14:20:32.505 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T14:20:32.571 DEBUG:teuthology.orchestra.run.smithi053:> sudo service rsyslog restart 2024-04-25T14:20:32.576 DEBUG:teuthology.orchestra.run.smithi150:> sudo service rsyslog restart 2024-04-25T14:20:32.578 DEBUG:teuthology.orchestra.run.smithi183:> sudo service rsyslog restart 2024-04-25T14:20:32.615 INFO:teuthology.orchestra.run.smithi150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T14:20:32.645 INFO:teuthology.orchestra.run.smithi183.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T14:20:32.671 INFO:teuthology.orchestra.run.smithi053.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T14:20:33.003 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T14:20:33.013 INFO:teuthology.task.internal:Starting timer... 2024-04-25T14:20:33.013 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T14:20:33.065 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T14:20:33.111 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-25T14:20:33.111 DEBUG:teuthology.orchestra.run.smithi053:> sudo service auditd rotate 2024-04-25T14:20:33.214 INFO:teuthology.orchestra.run.smithi053.stdout:Rotating logs: 2024-04-25T14:20:33.219 DEBUG:teuthology.orchestra.run.smithi150:> sudo service auditd rotate 2024-04-25T14:20:33.297 INFO:teuthology.orchestra.run.smithi150.stdout:Rotating logs: 2024-04-25T14:20:33.299 DEBUG:teuthology.orchestra.run.smithi183:> sudo service auditd rotate 2024-04-25T14:20:33.351 INFO:teuthology.orchestra.run.smithi183.stdout:Rotating logs: 2024-04-25T14:20:33.353 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T14:20:33.353 DEBUG:teuthology.orchestra.run.smithi053:> /usr/sbin/getenforce 2024-04-25T14:20:33.407 INFO:teuthology.orchestra.run.smithi053.stdout:Permissive 2024-04-25T14:20:33.408 DEBUG:teuthology.orchestra.run.smithi150:> /usr/sbin/getenforce 2024-04-25T14:20:33.429 INFO:teuthology.orchestra.run.smithi150.stdout:Permissive 2024-04-25T14:20:33.429 DEBUG:teuthology.orchestra.run.smithi183:> /usr/sbin/getenforce 2024-04-25T14:20:33.449 INFO:teuthology.orchestra.run.smithi183.stdout:Permissive 2024-04-25T14:20:33.450 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi053.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi150.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi183.front.sepia.ceph.com': 'permissive'} 2024-04-25T14:20:33.450 DEBUG:teuthology.orchestra.run.smithi053:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T14:20:33.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:20:33.520 DEBUG:teuthology.orchestra.run.smithi150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T14:20:33.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:20:33.546 DEBUG:teuthology.orchestra.run.smithi183:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T14:20:33.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:20:33.571 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T14:20:33.571 DEBUG:teuthology.orchestra.run.smithi053:> sudo /usr/sbin/setenforce permissive 2024-04-25T14:20:33.664 DEBUG:teuthology.orchestra.run.smithi150:> sudo /usr/sbin/setenforce permissive 2024-04-25T14:20:33.689 DEBUG:teuthology.orchestra.run.smithi183:> sudo /usr/sbin/setenforce permissive 2024-04-25T14:20:33.713 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T14:20:33.722 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-25T14:20:33.727 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-25T14:20:33.905 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T14:20:33.934 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T14:20:33.935 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi053.front.sepia.ceph.com,smithi150.front.sepia.ceph.com,smithi183.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T14:25:44.930 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi053.front.sepia.ceph.com'), Remote(name='ubuntu@smithi150.front.sepia.ceph.com'), Remote(name='ubuntu@smithi183.front.sepia.ceph.com')] 2024-04-25T14:25:44.931 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-04-25T14:25:44.932 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T14:25:45.046 DEBUG:teuthology.orchestra.run.smithi053:> true 2024-04-25T14:25:45.205 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-04-25T14:25:45.205 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-04-25T14:25:45.206 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T14:25:45.282 DEBUG:teuthology.orchestra.run.smithi150:> true 2024-04-25T14:25:45.362 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-04-25T14:25:45.362 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-04-25T14:25:45.363 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T14:25:45.437 DEBUG:teuthology.orchestra.run.smithi183:> true 2024-04-25T14:25:45.521 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-04-25T14:25:45.521 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T14:25:45.533 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T14:25:45.533 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T14:25:45.533 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T14:25:45.542 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T14:25:45.542 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T14:25:45.544 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T14:25:45.544 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T14:25:45.575 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T14:25:45.589 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T14:25:45.592 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T14:25:45.607 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T14:25:45.614 INFO:teuthology.orchestra.run.smithi150.stderr:sudo: ntpd: command not found 2024-04-25T14:25:45.625 INFO:teuthology.orchestra.run.smithi150.stdout:506 Cannot talk to daemon 2024-04-25T14:25:45.629 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T14:25:45.631 INFO:teuthology.orchestra.run.smithi183.stderr:sudo: ntpd: command not found 2024-04-25T14:25:45.639 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T14:25:45.642 INFO:teuthology.orchestra.run.smithi183.stdout:506 Cannot talk to daemon 2024-04-25T14:25:45.653 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T14:25:45.655 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T14:25:45.668 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T14:25:45.673 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T14:25:45.711 INFO:teuthology.orchestra.run.smithi150.stderr:bash: line 1: ntpq: command not found 2024-04-25T14:25:45.715 INFO:teuthology.orchestra.run.smithi150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T14:25:45.715 INFO:teuthology.orchestra.run.smithi150.stdout:=============================================================================== 2024-04-25T14:25:45.715 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:25:45.715 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:25:45.715 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:25:45.715 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:25:45.722 INFO:teuthology.orchestra.run.smithi183.stderr:bash: line 1: ntpq: command not found 2024-04-25T14:25:45.726 INFO:teuthology.orchestra.run.smithi183.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T14:25:45.726 INFO:teuthology.orchestra.run.smithi183.stdout:=============================================================================== 2024-04-25T14:25:45.726 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:25:45.726 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:25:45.726 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:25:45.726 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:25:45.753 INFO:teuthology.orchestra.run.smithi053.stderr:sudo: ntpd: command not found 2024-04-25T14:25:45.789 INFO:teuthology.orchestra.run.smithi053.stdout:506 Cannot talk to daemon 2024-04-25T14:25:45.835 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T14:25:45.881 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T14:25:46.022 INFO:teuthology.orchestra.run.smithi053.stderr:bash: line 1: ntpq: command not found 2024-04-25T14:25:46.030 INFO:teuthology.orchestra.run.smithi053.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T14:25:46.030 INFO:teuthology.orchestra.run.smithi053.stdout:=============================================================================== 2024-04-25T14:25:46.030 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:25:46.030 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:25:46.030 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:25:46.030 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:25:46.031 INFO:teuthology.run_tasks:Running task install... 2024-04-25T14:25:46.040 DEBUG:teuthology.task.install:project ceph 2024-04-25T14:25:46.040 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'}} 2024-04-25T14:25:46.040 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'} 2024-04-25T14:25:46.040 INFO:teuthology.task.install:Using flavor: default 2024-04-25T14:25:46.046 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T14:25:46.046 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T14:25:46.046 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T14:25:46.047 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T14:25:46.049 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T14:25:46.049 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T14:25:46.050 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T14:25:46.050 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T14:25:46.276 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T14:25:46.276 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T14:25:46.287 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T14:25:46.287 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T14:25:46.291 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T14:25:46.291 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T14:25:46.391 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T14:25:46.391 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-25T14:25:46.391 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T14:25:46.402 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T14:25:46.403 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-25T14:25:46.403 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T14:25:46.430 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T14:25:46.431 DEBUG:teuthology.orchestra.run.smithi183:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T14:25:46.488 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T14:25:46.488 DEBUG:teuthology.orchestra.run.smithi053:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T14:25:46.498 DEBUG:teuthology.orchestra.run.smithi183:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T14:25:46.501 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T14:25:46.501 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T14:25:46.501 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T14:25:46.530 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T14:25:46.530 DEBUG:teuthology.orchestra.run.smithi150:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T14:25:46.575 DEBUG:teuthology.orchestra.run.smithi183:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T14:25:46.599 DEBUG:teuthology.orchestra.run.smithi150:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T14:25:46.622 DEBUG:teuthology.orchestra.run.smithi053:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T14:25:46.639 INFO:teuthology.orchestra.run.smithi183.stdout:check_obsoletes = 1 2024-04-25T14:25:46.641 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum clean all 2024-04-25T14:25:46.674 DEBUG:teuthology.orchestra.run.smithi150:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T14:25:46.739 DEBUG:teuthology.orchestra.run.smithi053:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T14:25:46.740 INFO:teuthology.orchestra.run.smithi150.stdout:check_obsoletes = 1 2024-04-25T14:25:46.741 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum clean all 2024-04-25T14:25:46.821 INFO:teuthology.orchestra.run.smithi053.stdout:check_obsoletes = 1 2024-04-25T14:25:46.825 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum clean all 2024-04-25T14:25:46.869 INFO:teuthology.orchestra.run.smithi183.stdout:48 files removed 2024-04-25T14:25:46.897 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T14:25:46.969 INFO:teuthology.orchestra.run.smithi150.stdout:48 files removed 2024-04-25T14:25:46.996 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T14:25:47.150 INFO:teuthology.orchestra.run.smithi053.stdout:48 files removed 2024-04-25T14:25:47.189 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T14:25:47.579 INFO:teuthology.orchestra.run.smithi183.stdout:ceph packages for x86_64 197 kB/s | 83 kB 00:00 2024-04-25T14:25:47.625 INFO:teuthology.orchestra.run.smithi150.stdout:ceph packages for x86_64 227 kB/s | 83 kB 00:00 2024-04-25T14:25:47.913 INFO:teuthology.orchestra.run.smithi183.stdout:ceph noarch packages 40 kB/s | 12 kB 00:00 2024-04-25T14:25:48.084 INFO:teuthology.orchestra.run.smithi150.stdout:ceph noarch packages 28 kB/s | 12 kB 00:00 2024-04-25T14:25:48.132 INFO:teuthology.orchestra.run.smithi053.stdout:ceph packages for x86_64 171 kB/s | 83 kB 00:00 2024-04-25T14:25:48.397 INFO:teuthology.orchestra.run.smithi183.stdout:ceph source packages 4.2 kB/s | 1.9 kB 00:00 2024-04-25T14:25:48.501 INFO:teuthology.orchestra.run.smithi150.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-04-25T14:25:48.608 INFO:teuthology.orchestra.run.smithi053.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-04-25T14:25:49.175 INFO:teuthology.orchestra.run.smithi053.stdout:ceph source packages 3.9 kB/s | 1.9 kB 00:00 2024-04-25T14:25:50.333 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - BaseOS 4.2 MB/s | 8.0 MB 00:01 2024-04-25T14:25:50.478 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - BaseOS 6.6 MB/s | 8.0 MB 00:01 2024-04-25T14:25:50.587 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - BaseOS 3.9 MB/s | 8.0 MB 00:02 2024-04-25T14:25:52.475 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - AppStream 18 MB/s | 19 MB 00:01 2024-04-25T14:25:53.030 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - AppStream 14 MB/s | 19 MB 00:01 2024-04-25T14:25:53.280 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - AppStream 12 MB/s | 19 MB 00:01 2024-04-25T14:25:56.667 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-04-25T14:25:57.181 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-25T14:25:58.197 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - CRB 6.7 MB/s | 6.2 MB 00:00 2024-04-25T14:25:58.509 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - Extras packages 24 kB/s | 16 kB 00:00 2024-04-25T14:25:58.920 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - Extras packages 27 kB/s | 16 kB 00:00 2024-04-25T14:26:00.334 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - Extras packages 21 kB/s | 16 kB 00:00 2024-04-25T14:26:00.563 INFO:teuthology.orchestra.run.smithi183.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-25T14:26:02.388 INFO:teuthology.orchestra.run.smithi150.stdout:Extra Packages for Enterprise Linux 6.3 MB/s | 21 MB 00:03 2024-04-25T14:26:03.935 INFO:teuthology.orchestra.run.smithi053.stdout:Extra Packages for Enterprise Linux 6.1 MB/s | 21 MB 00:03 2024-04-25T14:26:05.593 INFO:teuthology.orchestra.run.smithi183.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-25T14:26:07.472 INFO:teuthology.orchestra.run.smithi150.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-25T14:26:07.770 INFO:teuthology.orchestra.run.smithi183.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T14:26:07.771 INFO:teuthology.orchestra.run.smithi183.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T14:26:07.781 INFO:teuthology.orchestra.run.smithi183.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T14:26:07.782 INFO:teuthology.orchestra.run.smithi183.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T14:26:07.783 INFO:teuthology.orchestra.run.smithi183.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T14:26:07.784 INFO:teuthology.orchestra.run.smithi183.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T14:26:07.785 INFO:teuthology.orchestra.run.smithi183.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T14:26:07.788 INFO:teuthology.orchestra.run.smithi183.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T14:26:07.789 INFO:teuthology.orchestra.run.smithi183.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T14:26:07.850 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-04-25T14:26:07.860 INFO:teuthology.orchestra.run.smithi183.stdout:======================================================================================= 2024-04-25T14:26:07.860 INFO:teuthology.orchestra.run.smithi183.stdout: Package Arch Version Repository Size 2024-04-25T14:26:07.860 INFO:teuthology.orchestra.run.smithi183.stdout:======================================================================================= 2024-04-25T14:26:07.860 INFO:teuthology.orchestra.run.smithi183.stdout:Installing: 2024-04-25T14:26:07.860 INFO:teuthology.orchestra.run.smithi183.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T14:26:07.860 INFO:teuthology.orchestra.run.smithi183.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T14:26:07.861 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T14:26:07.861 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T14:26:07.861 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T14:26:07.861 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T14:26:07.861 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T14:26:07.861 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T14:26:07.862 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T14:26:07.862 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T14:26:07.862 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T14:26:07.862 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T14:26:07.862 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T14:26:07.862 INFO:teuthology.orchestra.run.smithi183.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T14:26:07.862 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T14:26:07.863 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T14:26:07.863 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T14:26:07.863 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T14:26:07.863 INFO:teuthology.orchestra.run.smithi183.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T14:26:07.863 INFO:teuthology.orchestra.run.smithi183.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T14:26:07.863 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T14:26:07.864 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T14:26:07.864 INFO:teuthology.orchestra.run.smithi183.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T14:26:07.864 INFO:teuthology.orchestra.run.smithi183.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T14:26:07.864 INFO:teuthology.orchestra.run.smithi183.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T14:26:07.864 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T14:26:07.864 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T14:26:07.864 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T14:26:07.864 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T14:26:07.865 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T14:26:07.865 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T14:26:07.865 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T14:26:07.865 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T14:26:07.865 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T14:26:07.865 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T14:26:07.866 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T14:26:07.866 INFO:teuthology.orchestra.run.smithi183.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T14:26:07.866 INFO:teuthology.orchestra.run.smithi183.stdout:Upgrading: 2024-04-25T14:26:07.866 INFO:teuthology.orchestra.run.smithi183.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T14:26:07.866 INFO:teuthology.orchestra.run.smithi183.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T14:26:07.866 INFO:teuthology.orchestra.run.smithi183.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T14:26:07.866 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T14:26:07.867 INFO:teuthology.orchestra.run.smithi183.stdout:Installing dependencies: 2024-04-25T14:26:07.867 INFO:teuthology.orchestra.run.smithi183.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T14:26:07.867 INFO:teuthology.orchestra.run.smithi183.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T14:26:07.867 INFO:teuthology.orchestra.run.smithi183.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T14:26:07.867 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T14:26:07.868 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T14:26:07.868 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T14:26:07.868 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T14:26:07.868 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T14:26:07.868 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T14:26:07.868 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T14:26:07.869 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T14:26:07.869 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T14:26:07.869 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T14:26:07.869 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T14:26:07.869 INFO:teuthology.orchestra.run.smithi183.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T14:26:07.869 INFO:teuthology.orchestra.run.smithi183.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T14:26:07.869 INFO:teuthology.orchestra.run.smithi183.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T14:26:07.870 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T14:26:07.870 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T14:26:07.870 INFO:teuthology.orchestra.run.smithi183.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T14:26:07.870 INFO:teuthology.orchestra.run.smithi183.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T14:26:07.870 INFO:teuthology.orchestra.run.smithi183.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T14:26:07.870 INFO:teuthology.orchestra.run.smithi183.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T14:26:07.871 INFO:teuthology.orchestra.run.smithi183.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T14:26:07.871 INFO:teuthology.orchestra.run.smithi183.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T14:26:07.871 INFO:teuthology.orchestra.run.smithi183.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T14:26:07.871 INFO:teuthology.orchestra.run.smithi183.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T14:26:07.871 INFO:teuthology.orchestra.run.smithi183.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T14:26:07.871 INFO:teuthology.orchestra.run.smithi183.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T14:26:07.871 INFO:teuthology.orchestra.run.smithi183.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T14:26:07.872 INFO:teuthology.orchestra.run.smithi183.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T14:26:07.872 INFO:teuthology.orchestra.run.smithi183.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T14:26:07.872 INFO:teuthology.orchestra.run.smithi183.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T14:26:07.872 INFO:teuthology.orchestra.run.smithi183.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T14:26:07.872 INFO:teuthology.orchestra.run.smithi183.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T14:26:07.872 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T14:26:07.872 INFO:teuthology.orchestra.run.smithi183.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T14:26:07.873 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T14:26:07.873 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T14:26:07.873 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T14:26:07.873 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T14:26:07.873 INFO:teuthology.orchestra.run.smithi183.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T14:26:07.873 INFO:teuthology.orchestra.run.smithi183.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T14:26:07.874 INFO:teuthology.orchestra.run.smithi183.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T14:26:07.874 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T14:26:07.874 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T14:26:07.874 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T14:26:07.874 INFO:teuthology.orchestra.run.smithi183.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T14:26:07.874 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T14:26:07.875 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T14:26:07.875 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T14:26:07.875 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T14:26:07.875 INFO:teuthology.orchestra.run.smithi183.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T14:26:07.875 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T14:26:07.875 INFO:teuthology.orchestra.run.smithi183.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T14:26:07.876 INFO:teuthology.orchestra.run.smithi183.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T14:26:07.876 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T14:26:07.876 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T14:26:07.876 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T14:26:07.876 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T14:26:07.877 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T14:26:07.877 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T14:26:07.877 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T14:26:07.877 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T14:26:07.877 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T14:26:07.877 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T14:26:07.878 INFO:teuthology.orchestra.run.smithi183.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T14:26:07.878 INFO:teuthology.orchestra.run.smithi183.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T14:26:07.878 INFO:teuthology.orchestra.run.smithi183.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T14:26:07.878 INFO:teuthology.orchestra.run.smithi183.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T14:26:07.878 INFO:teuthology.orchestra.run.smithi183.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T14:26:07.878 INFO:teuthology.orchestra.run.smithi183.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T14:26:07.879 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T14:26:07.879 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T14:26:07.879 INFO:teuthology.orchestra.run.smithi183.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T14:26:07.879 INFO:teuthology.orchestra.run.smithi183.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T14:26:07.879 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T14:26:07.879 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T14:26:07.880 INFO:teuthology.orchestra.run.smithi183.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T14:26:07.880 INFO:teuthology.orchestra.run.smithi183.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T14:26:07.880 INFO:teuthology.orchestra.run.smithi183.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T14:26:07.880 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T14:26:07.880 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T14:26:07.881 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T14:26:07.881 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T14:26:07.881 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T14:26:07.881 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T14:26:07.881 INFO:teuthology.orchestra.run.smithi183.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T14:26:07.881 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T14:26:07.881 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T14:26:07.882 INFO:teuthology.orchestra.run.smithi183.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T14:26:07.882 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T14:26:07.882 INFO:teuthology.orchestra.run.smithi183.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T14:26:07.882 INFO:teuthology.orchestra.run.smithi183.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T14:26:07.882 INFO:teuthology.orchestra.run.smithi183.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T14:26:07.882 INFO:teuthology.orchestra.run.smithi183.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T14:26:07.883 INFO:teuthology.orchestra.run.smithi183.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T14:26:07.883 INFO:teuthology.orchestra.run.smithi183.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T14:26:07.883 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T14:26:07.883 INFO:teuthology.orchestra.run.smithi183.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T14:26:07.883 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T14:26:07.883 INFO:teuthology.orchestra.run.smithi183.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T14:26:07.884 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T14:26:07.884 INFO:teuthology.orchestra.run.smithi183.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T14:26:07.884 INFO:teuthology.orchestra.run.smithi183.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T14:26:07.884 INFO:teuthology.orchestra.run.smithi183.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T14:26:07.884 INFO:teuthology.orchestra.run.smithi183.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T14:26:07.884 INFO:teuthology.orchestra.run.smithi183.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T14:26:07.884 INFO:teuthology.orchestra.run.smithi183.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T14:26:07.884 INFO:teuthology.orchestra.run.smithi183.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T14:26:07.884 INFO:teuthology.orchestra.run.smithi183.stdout:Installing weak dependencies: 2024-04-25T14:26:07.885 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T14:26:07.885 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-25T14:26:07.885 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2024-04-25T14:26:07.885 INFO:teuthology.orchestra.run.smithi183.stdout:======================================================================================= 2024-04-25T14:26:07.885 INFO:teuthology.orchestra.run.smithi183.stdout:Install 148 Packages 2024-04-25T14:26:07.885 INFO:teuthology.orchestra.run.smithi183.stdout:Upgrade 4 Packages 2024-04-25T14:26:07.885 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-25T14:26:07.886 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 222 M 2024-04-25T14:26:07.886 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2024-04-25T14:26:08.620 INFO:teuthology.orchestra.run.smithi183.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 43 kB/s | 6.6 kB 00:00 2024-04-25T14:26:08.970 INFO:teuthology.orchestra.run.smithi183.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 2.5 MB/s | 890 kB 00:00 2024-04-25T14:26:09.228 INFO:teuthology.orchestra.run.smithi183.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 560 kB/s | 145 kB 00:00 2024-04-25T14:26:09.614 INFO:teuthology.orchestra.run.smithi150.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T14:26:09.615 INFO:teuthology.orchestra.run.smithi150.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T14:26:09.625 INFO:teuthology.orchestra.run.smithi150.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T14:26:09.626 INFO:teuthology.orchestra.run.smithi150.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T14:26:09.627 INFO:teuthology.orchestra.run.smithi150.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T14:26:09.628 INFO:teuthology.orchestra.run.smithi150.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T14:26:09.629 INFO:teuthology.orchestra.run.smithi150.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T14:26:09.632 INFO:teuthology.orchestra.run.smithi150.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T14:26:09.632 INFO:teuthology.orchestra.run.smithi150.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T14:26:09.692 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-04-25T14:26:09.707 INFO:teuthology.orchestra.run.smithi150.stdout:======================================================================================= 2024-04-25T14:26:09.707 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-04-25T14:26:09.707 INFO:teuthology.orchestra.run.smithi150.stdout:======================================================================================= 2024-04-25T14:26:09.707 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-04-25T14:26:09.707 INFO:teuthology.orchestra.run.smithi150.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T14:26:09.707 INFO:teuthology.orchestra.run.smithi150.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T14:26:09.707 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T14:26:09.707 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T14:26:09.707 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T14:26:09.707 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T14:26:09.708 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T14:26:09.708 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T14:26:09.708 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T14:26:09.708 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T14:26:09.708 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T14:26:09.708 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T14:26:09.708 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T14:26:09.708 INFO:teuthology.orchestra.run.smithi150.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T14:26:09.708 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T14:26:09.708 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T14:26:09.708 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T14:26:09.708 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T14:26:09.708 INFO:teuthology.orchestra.run.smithi150.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T14:26:09.708 INFO:teuthology.orchestra.run.smithi150.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T14:26:09.709 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T14:26:09.709 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T14:26:09.709 INFO:teuthology.orchestra.run.smithi150.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T14:26:09.709 INFO:teuthology.orchestra.run.smithi150.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T14:26:09.709 INFO:teuthology.orchestra.run.smithi150.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T14:26:09.709 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T14:26:09.709 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T14:26:09.709 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T14:26:09.709 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T14:26:09.709 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T14:26:09.709 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T14:26:09.709 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T14:26:09.709 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T14:26:09.709 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T14:26:09.710 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T14:26:09.710 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T14:26:09.710 INFO:teuthology.orchestra.run.smithi150.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T14:26:09.710 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrading: 2024-04-25T14:26:09.710 INFO:teuthology.orchestra.run.smithi150.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T14:26:09.710 INFO:teuthology.orchestra.run.smithi150.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T14:26:09.710 INFO:teuthology.orchestra.run.smithi150.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T14:26:09.710 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T14:26:09.710 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-04-25T14:26:09.710 INFO:teuthology.orchestra.run.smithi150.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T14:26:09.710 INFO:teuthology.orchestra.run.smithi150.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T14:26:09.710 INFO:teuthology.orchestra.run.smithi150.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T14:26:09.710 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T14:26:09.710 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T14:26:09.711 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T14:26:09.711 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T14:26:09.711 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T14:26:09.711 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T14:26:09.711 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T14:26:09.711 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T14:26:09.711 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T14:26:09.711 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T14:26:09.711 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T14:26:09.711 INFO:teuthology.orchestra.run.smithi150.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T14:26:09.711 INFO:teuthology.orchestra.run.smithi150.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T14:26:09.711 INFO:teuthology.orchestra.run.smithi150.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T14:26:09.711 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T14:26:09.711 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T14:26:09.711 INFO:teuthology.orchestra.run.smithi150.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T14:26:09.712 INFO:teuthology.orchestra.run.smithi150.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T14:26:09.712 INFO:teuthology.orchestra.run.smithi150.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T14:26:09.712 INFO:teuthology.orchestra.run.smithi150.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T14:26:09.712 INFO:teuthology.orchestra.run.smithi150.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T14:26:09.712 INFO:teuthology.orchestra.run.smithi150.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T14:26:09.712 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T14:26:09.712 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T14:26:09.712 INFO:teuthology.orchestra.run.smithi150.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T14:26:09.712 INFO:teuthology.orchestra.run.smithi150.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T14:26:09.712 INFO:teuthology.orchestra.run.smithi150.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T14:26:09.712 INFO:teuthology.orchestra.run.smithi150.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T14:26:09.712 INFO:teuthology.orchestra.run.smithi150.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T14:26:09.713 INFO:teuthology.orchestra.run.smithi150.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T14:26:09.713 INFO:teuthology.orchestra.run.smithi150.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T14:26:09.713 INFO:teuthology.orchestra.run.smithi150.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T14:26:09.713 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T14:26:09.713 INFO:teuthology.orchestra.run.smithi150.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T14:26:09.713 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T14:26:09.713 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T14:26:09.713 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T14:26:09.713 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T14:26:09.713 INFO:teuthology.orchestra.run.smithi150.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T14:26:09.713 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T14:26:09.713 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T14:26:09.713 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T14:26:09.713 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T14:26:09.714 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T14:26:09.714 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T14:26:09.714 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T14:26:09.714 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T14:26:09.714 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T14:26:09.714 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T14:26:09.714 INFO:teuthology.orchestra.run.smithi150.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T14:26:09.714 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T14:26:09.714 INFO:teuthology.orchestra.run.smithi150.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T14:26:09.714 INFO:teuthology.orchestra.run.smithi150.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T14:26:09.714 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T14:26:09.714 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T14:26:09.714 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T14:26:09.715 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T14:26:09.715 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T14:26:09.715 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T14:26:09.715 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T14:26:09.715 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T14:26:09.715 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T14:26:09.715 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T14:26:09.715 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T14:26:09.715 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T14:26:09.715 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T14:26:09.715 INFO:teuthology.orchestra.run.smithi150.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T14:26:09.715 INFO:teuthology.orchestra.run.smithi150.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T14:26:09.715 INFO:teuthology.orchestra.run.smithi150.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T14:26:09.715 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T14:26:09.715 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T14:26:09.716 INFO:teuthology.orchestra.run.smithi150.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T14:26:09.716 INFO:teuthology.orchestra.run.smithi150.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T14:26:09.716 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T14:26:09.716 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T14:26:09.716 INFO:teuthology.orchestra.run.smithi150.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T14:26:09.716 INFO:teuthology.orchestra.run.smithi150.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T14:26:09.716 INFO:teuthology.orchestra.run.smithi150.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T14:26:09.716 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T14:26:09.716 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T14:26:09.716 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T14:26:09.716 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T14:26:09.716 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T14:26:09.716 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T14:26:09.716 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T14:26:09.716 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T14:26:09.717 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T14:26:09.717 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T14:26:09.717 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T14:26:09.717 INFO:teuthology.orchestra.run.smithi150.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T14:26:09.717 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T14:26:09.717 INFO:teuthology.orchestra.run.smithi150.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T14:26:09.717 INFO:teuthology.orchestra.run.smithi150.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T14:26:09.717 INFO:teuthology.orchestra.run.smithi150.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T14:26:09.717 INFO:teuthology.orchestra.run.smithi150.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T14:26:09.717 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T14:26:09.717 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T14:26:09.717 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T14:26:09.717 INFO:teuthology.orchestra.run.smithi150.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T14:26:09.717 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T14:26:09.717 INFO:teuthology.orchestra.run.smithi150.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T14:26:09.718 INFO:teuthology.orchestra.run.smithi150.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T14:26:09.718 INFO:teuthology.orchestra.run.smithi150.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T14:26:09.718 INFO:teuthology.orchestra.run.smithi150.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T14:26:09.718 INFO:teuthology.orchestra.run.smithi150.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T14:26:09.718 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T14:26:09.718 INFO:teuthology.orchestra.run.smithi150.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T14:26:09.718 INFO:teuthology.orchestra.run.smithi150.stdout:Installing weak dependencies: 2024-04-25T14:26:09.718 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T14:26:09.718 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T14:26:09.718 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-04-25T14:26:09.718 INFO:teuthology.orchestra.run.smithi150.stdout:======================================================================================= 2024-04-25T14:26:09.718 INFO:teuthology.orchestra.run.smithi150.stdout:Install 148 Packages 2024-04-25T14:26:09.718 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrade 4 Packages 2024-04-25T14:26:09.719 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T14:26:09.719 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 222 M 2024-04-25T14:26:09.719 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-04-25T14:26:09.745 INFO:teuthology.orchestra.run.smithi183.stdout:(4/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 4.6 MB/s | 2.4 MB 00:00 2024-04-25T14:26:09.937 INFO:teuthology.orchestra.run.smithi183.stdout:(5/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 7.8 MB/s | 1.5 MB 00:00 2024-04-25T14:26:10.110 INFO:teuthology.orchestra.run.smithi053.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-04-25T14:26:10.287 INFO:teuthology.orchestra.run.smithi183.stdout:(6/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 13 MB/s | 4.7 MB 00:00 2024-04-25T14:26:11.109 INFO:teuthology.orchestra.run.smithi150.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 16 kB/s | 6.6 kB 00:00 2024-04-25T14:26:11.146 INFO:teuthology.orchestra.run.smithi183.stdout:(7/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x86 20 MB/s | 17 MB 00:00 2024-04-25T14:26:11.729 INFO:teuthology.orchestra.run.smithi183.stdout:(8/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el7 18 MB/s | 11 MB 00:00 2024-04-25T14:26:11.748 INFO:teuthology.orchestra.run.smithi150.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.4 MB/s | 890 kB 00:00 2024-04-25T14:26:11.804 INFO:teuthology.orchestra.run.smithi183.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 337 kB/s | 25 kB 00:00 2024-04-25T14:26:11.823 INFO:teuthology.orchestra.run.smithi150.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 1.9 MB/s | 145 kB 00:00 2024-04-25T14:26:12.188 INFO:teuthology.orchestra.run.smithi183.stdout:(10/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x 1.4 MB/s | 5.4 MB 00:03 2024-04-25T14:26:12.396 INFO:teuthology.orchestra.run.smithi183.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 1.1 MB/s | 225 kB 00:00 2024-04-25T14:26:12.555 INFO:teuthology.orchestra.run.smithi183.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 171 kB/s | 27 kB 00:00 2024-04-25T14:26:12.637 INFO:teuthology.orchestra.run.smithi053.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T14:26:12.638 INFO:teuthology.orchestra.run.smithi053.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T14:26:12.659 INFO:teuthology.orchestra.run.smithi053.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T14:26:12.661 INFO:teuthology.orchestra.run.smithi053.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T14:26:12.662 INFO:teuthology.orchestra.run.smithi053.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T14:26:12.663 INFO:teuthology.orchestra.run.smithi053.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T14:26:12.666 INFO:teuthology.orchestra.run.smithi053.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T14:26:12.669 INFO:teuthology.orchestra.run.smithi053.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T14:26:12.670 INFO:teuthology.orchestra.run.smithi053.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T14:26:12.697 INFO:teuthology.orchestra.run.smithi183.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 224 kB/s | 32 kB 00:00 2024-04-25T14:26:12.747 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-04-25T14:26:12.761 INFO:teuthology.orchestra.run.smithi053.stdout:======================================================================================= 2024-04-25T14:26:12.761 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-04-25T14:26:12.762 INFO:teuthology.orchestra.run.smithi053.stdout:======================================================================================= 2024-04-25T14:26:12.762 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-04-25T14:26:12.762 INFO:teuthology.orchestra.run.smithi053.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T14:26:12.762 INFO:teuthology.orchestra.run.smithi053.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T14:26:12.762 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T14:26:12.762 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T14:26:12.762 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T14:26:12.762 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T14:26:12.762 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T14:26:12.762 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T14:26:12.762 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T14:26:12.762 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T14:26:12.762 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T14:26:12.763 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T14:26:12.763 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T14:26:12.763 INFO:teuthology.orchestra.run.smithi053.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T14:26:12.763 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T14:26:12.763 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T14:26:12.763 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T14:26:12.763 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T14:26:12.763 INFO:teuthology.orchestra.run.smithi053.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T14:26:12.763 INFO:teuthology.orchestra.run.smithi053.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T14:26:12.763 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T14:26:12.763 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T14:26:12.763 INFO:teuthology.orchestra.run.smithi053.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T14:26:12.763 INFO:teuthology.orchestra.run.smithi053.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T14:26:12.764 INFO:teuthology.orchestra.run.smithi053.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T14:26:12.764 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T14:26:12.764 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T14:26:12.764 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T14:26:12.764 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T14:26:12.764 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T14:26:12.764 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T14:26:12.764 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T14:26:12.764 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T14:26:12.764 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T14:26:12.764 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T14:26:12.764 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T14:26:12.764 INFO:teuthology.orchestra.run.smithi053.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T14:26:12.764 INFO:teuthology.orchestra.run.smithi053.stdout:Upgrading: 2024-04-25T14:26:12.765 INFO:teuthology.orchestra.run.smithi053.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T14:26:12.765 INFO:teuthology.orchestra.run.smithi053.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T14:26:12.765 INFO:teuthology.orchestra.run.smithi053.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T14:26:12.765 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T14:26:12.765 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-04-25T14:26:12.765 INFO:teuthology.orchestra.run.smithi053.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T14:26:12.765 INFO:teuthology.orchestra.run.smithi053.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T14:26:12.765 INFO:teuthology.orchestra.run.smithi053.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T14:26:12.765 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T14:26:12.765 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T14:26:12.765 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T14:26:12.765 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T14:26:12.765 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T14:26:12.765 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T14:26:12.766 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T14:26:12.766 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T14:26:12.766 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T14:26:12.766 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T14:26:12.766 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T14:26:12.766 INFO:teuthology.orchestra.run.smithi053.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T14:26:12.766 INFO:teuthology.orchestra.run.smithi053.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T14:26:12.766 INFO:teuthology.orchestra.run.smithi053.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T14:26:12.766 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T14:26:12.766 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T14:26:12.766 INFO:teuthology.orchestra.run.smithi053.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T14:26:12.766 INFO:teuthology.orchestra.run.smithi053.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T14:26:12.766 INFO:teuthology.orchestra.run.smithi053.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T14:26:12.766 INFO:teuthology.orchestra.run.smithi053.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T14:26:12.767 INFO:teuthology.orchestra.run.smithi053.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T14:26:12.767 INFO:teuthology.orchestra.run.smithi053.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T14:26:12.767 INFO:teuthology.orchestra.run.smithi053.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T14:26:12.767 INFO:teuthology.orchestra.run.smithi053.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T14:26:12.767 INFO:teuthology.orchestra.run.smithi053.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T14:26:12.767 INFO:teuthology.orchestra.run.smithi053.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T14:26:12.767 INFO:teuthology.orchestra.run.smithi053.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T14:26:12.767 INFO:teuthology.orchestra.run.smithi053.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T14:26:12.767 INFO:teuthology.orchestra.run.smithi053.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T14:26:12.767 INFO:teuthology.orchestra.run.smithi053.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T14:26:12.767 INFO:teuthology.orchestra.run.smithi053.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T14:26:12.767 INFO:teuthology.orchestra.run.smithi053.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T14:26:12.767 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T14:26:12.767 INFO:teuthology.orchestra.run.smithi053.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T14:26:12.768 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T14:26:12.768 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T14:26:12.768 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T14:26:12.768 INFO:teuthology.orchestra.run.smithi053.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T14:26:12.768 INFO:teuthology.orchestra.run.smithi053.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T14:26:12.768 INFO:teuthology.orchestra.run.smithi053.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T14:26:12.768 INFO:teuthology.orchestra.run.smithi053.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T14:26:12.768 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T14:26:12.768 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T14:26:12.768 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T14:26:12.768 INFO:teuthology.orchestra.run.smithi053.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T14:26:12.768 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T14:26:12.768 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T14:26:12.768 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T14:26:12.768 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T14:26:12.769 INFO:teuthology.orchestra.run.smithi053.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T14:26:12.769 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T14:26:12.769 INFO:teuthology.orchestra.run.smithi053.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T14:26:12.769 INFO:teuthology.orchestra.run.smithi053.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T14:26:12.769 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T14:26:12.769 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T14:26:12.769 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T14:26:12.769 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T14:26:12.769 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T14:26:12.769 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T14:26:12.769 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T14:26:12.769 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T14:26:12.769 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T14:26:12.769 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T14:26:12.770 INFO:teuthology.orchestra.run.smithi053.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T14:26:12.770 INFO:teuthology.orchestra.run.smithi053.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T14:26:12.770 INFO:teuthology.orchestra.run.smithi053.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T14:26:12.770 INFO:teuthology.orchestra.run.smithi053.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T14:26:12.770 INFO:teuthology.orchestra.run.smithi053.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T14:26:12.770 INFO:teuthology.orchestra.run.smithi053.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T14:26:12.770 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T14:26:12.770 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T14:26:12.770 INFO:teuthology.orchestra.run.smithi053.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T14:26:12.770 INFO:teuthology.orchestra.run.smithi053.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T14:26:12.770 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T14:26:12.770 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T14:26:12.770 INFO:teuthology.orchestra.run.smithi053.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T14:26:12.770 INFO:teuthology.orchestra.run.smithi053.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T14:26:12.771 INFO:teuthology.orchestra.run.smithi053.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T14:26:12.771 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T14:26:12.771 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T14:26:12.771 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T14:26:12.771 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T14:26:12.771 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T14:26:12.771 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T14:26:12.771 INFO:teuthology.orchestra.run.smithi053.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T14:26:12.771 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T14:26:12.771 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T14:26:12.771 INFO:teuthology.orchestra.run.smithi053.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T14:26:12.771 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T14:26:12.771 INFO:teuthology.orchestra.run.smithi053.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T14:26:12.771 INFO:teuthology.orchestra.run.smithi053.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T14:26:12.772 INFO:teuthology.orchestra.run.smithi053.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T14:26:12.772 INFO:teuthology.orchestra.run.smithi053.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T14:26:12.772 INFO:teuthology.orchestra.run.smithi053.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T14:26:12.772 INFO:teuthology.orchestra.run.smithi053.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T14:26:12.772 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T14:26:12.772 INFO:teuthology.orchestra.run.smithi053.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T14:26:12.772 INFO:teuthology.orchestra.run.smithi053.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T14:26:12.772 INFO:teuthology.orchestra.run.smithi053.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T14:26:12.772 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T14:26:12.772 INFO:teuthology.orchestra.run.smithi053.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T14:26:12.772 INFO:teuthology.orchestra.run.smithi053.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T14:26:12.772 INFO:teuthology.orchestra.run.smithi053.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T14:26:12.772 INFO:teuthology.orchestra.run.smithi053.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T14:26:12.772 INFO:teuthology.orchestra.run.smithi053.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T14:26:12.772 INFO:teuthology.orchestra.run.smithi053.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T14:26:12.773 INFO:teuthology.orchestra.run.smithi053.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T14:26:12.773 INFO:teuthology.orchestra.run.smithi053.stdout:Installing weak dependencies: 2024-04-25T14:26:12.773 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T14:26:12.773 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:26:12.773 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-04-25T14:26:12.773 INFO:teuthology.orchestra.run.smithi053.stdout:======================================================================================= 2024-04-25T14:26:12.773 INFO:teuthology.orchestra.run.smithi053.stdout:Install 148 Packages 2024-04-25T14:26:12.773 INFO:teuthology.orchestra.run.smithi053.stdout:Upgrade 4 Packages 2024-04-25T14:26:12.773 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:26:12.773 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 222 M 2024-04-25T14:26:12.774 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-04-25T14:26:13.306 INFO:teuthology.orchestra.run.smithi183.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 1.2 MB/s | 747 kB 00:00 2024-04-25T14:26:13.440 INFO:teuthology.orchestra.run.smithi150.stdout:(4/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 1.9 MB/s | 5.4 MB 00:02 2024-04-25T14:26:13.591 INFO:teuthology.orchestra.run.smithi150.stdout:(5/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 1.3 MB/s | 2.4 MB 00:01 2024-04-25T14:26:13.723 INFO:teuthology.orchestra.run.smithi183.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 392 kB/s | 163 kB 00:00 2024-04-25T14:26:13.857 INFO:teuthology.orchestra.run.smithi183.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 951 kB/s | 126 kB 00:00 2024-04-25T14:26:13.937 INFO:teuthology.orchestra.run.smithi053.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 18 kB/s | 6.6 kB 00:00 2024-04-25T14:26:14.200 INFO:teuthology.orchestra.run.smithi150.stdout:(6/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 2.0 MB/s | 1.5 MB 00:00 2024-04-25T14:26:14.378 INFO:teuthology.orchestra.run.smithi053.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 2.0 MB/s | 890 kB 00:00 2024-04-25T14:26:14.437 INFO:teuthology.orchestra.run.smithi053.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 2.5 MB/s | 145 kB 00:00 2024-04-25T14:26:14.549 INFO:teuthology.orchestra.run.smithi183.stdout:(17/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 18 MB/s | 49 MB 00:02 2024-04-25T14:26:14.574 INFO:teuthology.orchestra.run.smithi183.stdout:(18/152): libradosstriper1-19.0.0-3156.gd69f13d 698 kB/s | 500 kB 00:00 2024-04-25T14:26:14.716 INFO:teuthology.orchestra.run.smithi183.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 318 kB/s | 45 kB 00:00 2024-04-25T14:26:15.016 INFO:teuthology.orchestra.run.smithi183.stdout:(20/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 11 MB/s | 5.4 MB 00:00 2024-04-25T14:26:15.041 INFO:teuthology.orchestra.run.smithi183.stdout:(21/152): python3-ceph-common-19.0.0-3156.gd69f 414 kB/s | 135 kB 00:00 2024-04-25T14:26:15.075 INFO:teuthology.orchestra.run.smithi183.stdout:(22/152): python3-cephfs-19.0.0-3156.gd69f13d8. 2.7 MB/s | 161 kB 00:00 2024-04-25T14:26:15.137 INFO:teuthology.orchestra.run.smithi053.stdout:(4/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 3.4 MB/s | 2.4 MB 00:00 2024-04-25T14:26:15.150 INFO:teuthology.orchestra.run.smithi183.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 3.9 MB/s | 298 kB 00:00 2024-04-25T14:26:15.200 INFO:teuthology.orchestra.run.smithi183.stdout:(24/152): python3-rados-19.0.0-3156.gd69f13d8.e 2.0 MB/s | 320 kB 00:00 2024-04-25T14:26:15.225 INFO:teuthology.orchestra.run.smithi183.stdout:(25/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.3 MB/s | 99 kB 00:00 2024-04-25T14:26:15.318 INFO:teuthology.orchestra.run.smithi183.stdout:(26/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 730 kB/s | 85 kB 00:00 2024-04-25T14:26:15.451 INFO:teuthology.orchestra.run.smithi183.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 1.3 MB/s | 170 kB 00:00 2024-04-25T14:26:15.496 INFO:teuthology.orchestra.run.smithi053.stdout:(5/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 4.2 MB/s | 1.5 MB 00:00 2024-04-25T14:26:15.577 INFO:teuthology.orchestra.run.smithi183.stdout:(28/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 8.8 MB/s | 3.1 MB 00:00 2024-04-25T14:26:15.593 INFO:teuthology.orchestra.run.smithi183.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3156.g 241 kB/s | 34 kB 00:00 2024-04-25T14:26:15.669 INFO:teuthology.orchestra.run.smithi183.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.5 MB/s | 142 kB 00:00 2024-04-25T14:26:16.233 INFO:teuthology.orchestra.run.smithi150.stdout:(7/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 1.8 MB/s | 4.7 MB 00:02 2024-04-25T14:26:16.338 INFO:teuthology.orchestra.run.smithi053.stdout:(6/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 5.6 MB/s | 4.7 MB 00:00 2024-04-25T14:26:16.744 INFO:teuthology.orchestra.run.smithi183.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 6.9 MB/s | 7.4 MB 00:01 2024-04-25T14:26:16.886 INFO:teuthology.orchestra.run.smithi183.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 1.7 MB/s | 249 kB 00:00 2024-04-25T14:26:16.978 INFO:teuthology.orchestra.run.smithi183.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 538 kB/s | 49 kB 00:00 2024-04-25T14:26:17.078 INFO:teuthology.orchestra.run.smithi183.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 164 kB/s | 16 kB 00:00 2024-04-25T14:26:17.187 INFO:teuthology.orchestra.run.smithi183.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 2.5 MB/s | 273 kB 00:00 2024-04-25T14:26:17.462 INFO:teuthology.orchestra.run.smithi183.stdout:(36/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 2.7 MB/s | 762 kB 00:00 2024-04-25T14:26:17.579 INFO:teuthology.orchestra.run.smithi183.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 157 kB/s | 18 kB 00:00 2024-04-25T14:26:17.579 INFO:teuthology.orchestra.run.smithi053.stdout:(7/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 1.3 MB/s | 5.4 MB 00:04 2024-04-25T14:26:17.638 INFO:teuthology.orchestra.run.smithi183.stdout:(38/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 1.7 MB/s | 3.5 MB 00:02 2024-04-25T14:26:17.779 INFO:teuthology.orchestra.run.smithi183.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 401 kB/s | 80 kB 00:00 2024-04-25T14:26:17.838 INFO:teuthology.orchestra.run.smithi183.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 3.9 MB/s | 803 kB 00:00 2024-04-25T14:26:17.863 INFO:teuthology.orchestra.run.smithi183.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-04-25T14:26:17.888 INFO:teuthology.orchestra.run.smithi183.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-04-25T14:26:17.947 INFO:teuthology.orchestra.run.smithi183.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-04-25T14:26:17.972 INFO:teuthology.orchestra.run.smithi183.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2024-04-25T14:26:17.988 INFO:teuthology.orchestra.run.smithi053.stdout:(8/152): ceph-common-19.0.0-3156.gd69f13d8.el7. 4.8 MB/s | 21 MB 00:04 2024-04-25T14:26:17.998 INFO:teuthology.orchestra.run.smithi183.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-25T14:26:18.023 INFO:teuthology.orchestra.run.smithi183.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-25T14:26:18.046 INFO:teuthology.orchestra.run.smithi053.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 440 kB/s | 25 kB 00:00 2024-04-25T14:26:18.048 INFO:teuthology.orchestra.run.smithi183.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-25T14:26:18.082 INFO:teuthology.orchestra.run.smithi183.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 12 MB/s | 689 kB 00:00 2024-04-25T14:26:18.115 INFO:teuthology.orchestra.run.smithi183.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 943 kB 00:00 2024-04-25T14:26:18.141 INFO:teuthology.orchestra.run.smithi183.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-04-25T14:26:18.174 INFO:teuthology.orchestra.run.smithi183.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 369 kB/s | 21 kB 00:00 2024-04-25T14:26:18.216 INFO:teuthology.orchestra.run.smithi183.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 313 kB 00:00 2024-04-25T14:26:18.250 INFO:teuthology.orchestra.run.smithi183.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 435 kB/s | 33 kB 00:00 2024-04-25T14:26:18.322 INFO:teuthology.orchestra.run.smithi053.stdout:(10/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 8.5 MB/s | 17 MB 00:01 2024-04-25T14:26:18.325 INFO:teuthology.orchestra.run.smithi183.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 243 kB/s | 18 kB 00:00 2024-04-25T14:26:18.375 INFO:teuthology.orchestra.run.smithi183.stdout:(55/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 19 MB/s | 3.0 MB 00:00 2024-04-25T14:26:18.381 INFO:teuthology.orchestra.run.smithi053.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 3.8 MB/s | 225 kB 00:00 2024-04-25T14:26:18.401 INFO:teuthology.orchestra.run.smithi183.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 150 kB/s | 11 kB 00:00 2024-04-25T14:26:18.431 INFO:teuthology.orchestra.run.smithi053.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 550 kB/s | 27 kB 00:00 2024-04-25T14:26:18.434 INFO:teuthology.orchestra.run.smithi183.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-25T14:26:18.460 INFO:teuthology.orchestra.run.smithi183.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 773 kB/s | 45 kB 00:00 2024-04-25T14:26:18.481 INFO:teuthology.orchestra.run.smithi053.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 647 kB/s | 32 kB 00:00 2024-04-25T14:26:18.483 INFO:teuthology.orchestra.run.smithi150.stdout:(8/152): ceph-common-19.0.0-3156.gd69f13d8.el7. 2.7 MB/s | 21 MB 00:07 2024-04-25T14:26:18.485 INFO:teuthology.orchestra.run.smithi183.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-04-25T14:26:18.518 INFO:teuthology.orchestra.run.smithi183.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 9.9 MB/s | 582 kB 00:00 2024-04-25T14:26:18.535 INFO:teuthology.orchestra.run.smithi183.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 518 kB/s | 25 kB 00:00 2024-04-25T14:26:18.550 INFO:teuthology.orchestra.run.smithi150.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 379 kB/s | 25 kB 00:00 2024-04-25T14:26:18.552 INFO:teuthology.orchestra.run.smithi183.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-04-25T14:26:18.578 INFO:teuthology.orchestra.run.smithi183.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.8 MB/s | 292 kB 00:00 2024-04-25T14:26:18.603 INFO:teuthology.orchestra.run.smithi183.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-25T14:26:18.623 INFO:teuthology.orchestra.run.smithi053.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 5.2 MB/s | 747 kB 00:00 2024-04-25T14:26:18.636 INFO:teuthology.orchestra.run.smithi183.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 608 kB/s | 35 kB 00:00 2024-04-25T14:26:18.698 INFO:teuthology.orchestra.run.smithi053.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 2.2 MB/s | 163 kB 00:00 2024-04-25T14:26:18.774 INFO:teuthology.orchestra.run.smithi053.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 1.7 MB/s | 126 kB 00:00 2024-04-25T14:26:18.828 INFO:teuthology.orchestra.run.smithi183.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 21 MB/s | 4.7 MB 00:00 2024-04-25T14:26:18.890 INFO:teuthology.orchestra.run.smithi053.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 4.2 MB/s | 500 kB 00:00 2024-04-25T14:26:18.954 INFO:teuthology.orchestra.run.smithi183.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-04-25T14:26:18.995 INFO:teuthology.orchestra.run.smithi183.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 9.7 MB/s | 1.6 MB 00:00 2024-04-25T14:26:19.029 INFO:teuthology.orchestra.run.smithi183.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-04-25T14:26:19.055 INFO:teuthology.orchestra.run.smithi183.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.3 MB/s | 133 kB 00:00 2024-04-25T14:26:19.272 INFO:teuthology.orchestra.run.smithi183.stdout:(71/152): postgresql-server-13.14-1.el9.x86_64. 24 MB/s | 5.8 MB 00:00 2024-04-25T14:26:19.305 INFO:teuthology.orchestra.run.smithi183.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 4.0 MB/s | 1.0 MB 00:00 2024-04-25T14:26:19.330 INFO:teuthology.orchestra.run.smithi183.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-25T14:26:19.539 INFO:teuthology.orchestra.run.smithi183.stdout:(74/152): python3-babel-2.9.1-2.el9.noarch.rpm 26 MB/s | 6.0 MB 00:00 2024-04-25T14:26:19.564 INFO:teuthology.orchestra.run.smithi183.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.0 MB/s | 244 kB 00:00 2024-04-25T14:26:19.590 INFO:teuthology.orchestra.run.smithi183.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-04-25T14:26:19.590 INFO:teuthology.orchestra.run.smithi053.stdout:(18/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 7.7 MB/s | 5.4 MB 00:00 2024-04-25T14:26:19.615 INFO:teuthology.orchestra.run.smithi183.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 515 kB/s | 26 kB 00:00 2024-04-25T14:26:19.640 INFO:teuthology.orchestra.run.smithi183.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-04-25T14:26:19.649 INFO:teuthology.orchestra.run.smithi053.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 784 kB/s | 45 kB 00:00 2024-04-25T14:26:19.666 INFO:teuthology.orchestra.run.smithi183.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-04-25T14:26:19.691 INFO:teuthology.orchestra.run.smithi183.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 699 kB/s | 35 kB 00:00 2024-04-25T14:26:19.716 INFO:teuthology.orchestra.run.smithi053.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 2.0 MB/s | 135 kB 00:00 2024-04-25T14:26:19.775 INFO:teuthology.orchestra.run.smithi053.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 2.8 MB/s | 161 kB 00:00 2024-04-25T14:26:19.850 INFO:teuthology.orchestra.run.smithi053.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 4.2 MB/s | 320 kB 00:00 2024-04-25T14:26:19.875 INFO:teuthology.orchestra.run.smithi183.stdout:(81/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 24 MB/s | 5.1 MB 00:00 2024-04-25T14:26:19.900 INFO:teuthology.orchestra.run.smithi183.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.2 MB/s | 250 kB 00:00 2024-04-25T14:26:19.925 INFO:teuthology.orchestra.run.smithi183.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 4.4 MB/s | 222 kB 00:00 2024-04-25T14:26:19.925 INFO:teuthology.orchestra.run.smithi053.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 3.9 MB/s | 298 kB 00:00 2024-04-25T14:26:19.950 INFO:teuthology.orchestra.run.smithi183.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-25T14:26:19.976 INFO:teuthology.orchestra.run.smithi053.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 2.0 MB/s | 99 kB 00:00 2024-04-25T14:26:19.976 INFO:teuthology.orchestra.run.smithi183.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 42 kB 00:00 2024-04-25T14:26:20.001 INFO:teuthology.orchestra.run.smithi183.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-04-25T14:26:20.026 INFO:teuthology.orchestra.run.smithi183.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-25T14:26:20.051 INFO:teuthology.orchestra.run.smithi183.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.5 MB/s | 279 kB 00:00 2024-04-25T14:26:20.059 INFO:teuthology.orchestra.run.smithi053.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.0 MB/s | 85 kB 00:00 2024-04-25T14:26:20.075 INFO:teuthology.orchestra.run.smithi150.stdout:(10/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 2.9 MB/s | 17 MB 00:05 2024-04-25T14:26:20.077 INFO:teuthology.orchestra.run.smithi183.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-25T14:26:20.102 INFO:teuthology.orchestra.run.smithi183.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-25T14:26:20.217 INFO:teuthology.orchestra.run.smithi150.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 1.6 MB/s | 225 kB 00:00 2024-04-25T14:26:20.292 INFO:teuthology.orchestra.run.smithi150.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 361 kB/s | 27 kB 00:00 2024-04-25T14:26:20.327 INFO:teuthology.orchestra.run.smithi183.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 186 kB/s | 42 kB 00:00 2024-04-25T14:26:20.376 INFO:teuthology.orchestra.run.smithi150.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 381 kB/s | 32 kB 00:00 2024-04-25T14:26:20.493 INFO:teuthology.orchestra.run.smithi053.stdout:(26/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 7.1 MB/s | 3.1 MB 00:00 2024-04-25T14:26:20.552 INFO:teuthology.orchestra.run.smithi183.stdout:(92/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 34 MB/s | 16 MB 00:00 2024-04-25T14:26:20.577 INFO:teuthology.orchestra.run.smithi053.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 2.0 MB/s | 170 kB 00:00 2024-04-25T14:26:20.578 INFO:teuthology.orchestra.run.smithi183.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 178 kB/s | 44 kB 00:00 2024-04-25T14:26:20.601 INFO:teuthology.orchestra.run.smithi150.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 3.2 MB/s | 747 kB 00:00 2024-04-25T14:26:20.603 INFO:teuthology.orchestra.run.smithi183.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-25T14:26:20.628 INFO:teuthology.orchestra.run.smithi183.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 720 kB/s | 36 kB 00:00 2024-04-25T14:26:20.635 INFO:teuthology.orchestra.run.smithi053.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 594 kB/s | 34 kB 00:00 2024-04-25T14:26:20.653 INFO:teuthology.orchestra.run.smithi183.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-04-25T14:26:20.676 INFO:teuthology.orchestra.run.smithi150.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 2.1 MB/s | 163 kB 00:00 2024-04-25T14:26:20.679 INFO:teuthology.orchestra.run.smithi183.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 305 kB 00:00 2024-04-25T14:26:20.704 INFO:teuthology.orchestra.run.smithi183.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-25T14:26:20.719 INFO:teuthology.orchestra.run.smithi053.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.7 MB/s | 142 kB 00:00 2024-04-25T14:26:20.729 INFO:teuthology.orchestra.run.smithi183.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-25T14:26:20.743 INFO:teuthology.orchestra.run.smithi150.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 1.9 MB/s | 126 kB 00:00 2024-04-25T14:26:20.773 INFO:teuthology.orchestra.run.smithi183.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 921 kB/s | 64 kB 00:00 2024-04-25T14:26:20.796 INFO:teuthology.orchestra.run.smithi183.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 671 kB/s | 45 kB 00:00 2024-04-25T14:26:20.822 INFO:teuthology.orchestra.run.smithi183.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 466 kB/s | 22 kB 00:00 2024-04-25T14:26:20.864 INFO:teuthology.orchestra.run.smithi183.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 13 MB/s | 863 kB 00:00 2024-04-25T14:26:20.952 INFO:teuthology.orchestra.run.smithi150.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 2.3 MB/s | 500 kB 00:00 2024-04-25T14:26:21.228 INFO:teuthology.orchestra.run.smithi053.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 7.0 MB/s | 3.5 MB 00:00 2024-04-25T14:26:21.331 INFO:teuthology.orchestra.run.smithi183.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 662 kB/s | 309 kB 00:00 2024-04-25T14:26:21.336 INFO:teuthology.orchestra.run.smithi150.stdout:(18/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 2.1 MB/s | 11 MB 00:05 2024-04-25T14:26:21.386 INFO:teuthology.orchestra.run.smithi150.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 909 kB/s | 45 kB 00:00 2024-04-25T14:26:21.389 INFO:teuthology.orchestra.run.smithi183.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 334 kB/s | 19 kB 00:00 2024-04-25T14:26:21.415 INFO:teuthology.orchestra.run.smithi183.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 927 kB/s | 548 kB 00:00 2024-04-25T14:26:21.453 INFO:teuthology.orchestra.run.smithi150.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 2.0 MB/s | 135 kB 00:00 2024-04-25T14:26:21.469 INFO:teuthology.orchestra.run.smithi053.stdout:(31/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 2.7 MB/s | 11 MB 00:03 2024-04-25T14:26:21.473 INFO:teuthology.orchestra.run.smithi183.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 446 kB/s | 26 kB 00:00 2024-04-25T14:26:21.532 INFO:teuthology.orchestra.run.smithi183.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 833 kB/s | 49 kB 00:00 2024-04-25T14:26:21.586 INFO:teuthology.orchestra.run.smithi150.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 1.2 MB/s | 161 kB 00:00 2024-04-25T14:26:21.599 INFO:teuthology.orchestra.run.smithi183.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-04-25T14:26:21.637 INFO:teuthology.orchestra.run.smithi053.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 1.5 MB/s | 249 kB 00:00 2024-04-25T14:26:21.716 INFO:teuthology.orchestra.run.smithi183.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-04-25T14:26:21.745 INFO:teuthology.orchestra.run.smithi053.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 460 kB/s | 49 kB 00:00 2024-04-25T14:26:21.787 INFO:teuthology.orchestra.run.smithi150.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 1.6 MB/s | 320 kB 00:00 2024-04-25T14:26:21.864 INFO:teuthology.orchestra.run.smithi053.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 140 kB/s | 16 kB 00:00 2024-04-25T14:26:21.970 INFO:teuthology.orchestra.run.smithi150.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 1.6 MB/s | 298 kB 00:00 2024-04-25T14:26:22.054 INFO:teuthology.orchestra.run.smithi150.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.2 MB/s | 99 kB 00:00 2024-04-25T14:26:22.062 INFO:teuthology.orchestra.run.smithi053.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 1.4 MB/s | 273 kB 00:00 2024-04-25T14:26:22.083 INFO:teuthology.orchestra.run.smithi183.stdout:(111/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.2 MB/s | 838 kB 00:00 2024-04-25T14:26:22.104 INFO:teuthology.orchestra.run.smithi150.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.7 MB/s | 85 kB 00:00 2024-04-25T14:26:22.204 INFO:teuthology.orchestra.run.smithi053.stdout:(36/152): ceph-mgr-diskprediction-local-19.0.0- 7.6 MB/s | 7.4 MB 00:00 2024-04-25T14:26:22.280 INFO:teuthology.orchestra.run.smithi053.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 247 kB/s | 18 kB 00:00 2024-04-25T14:26:22.300 INFO:teuthology.orchestra.run.smithi183.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 2.5 MB/s | 548 kB 00:00 2024-04-25T14:26:22.375 INFO:teuthology.orchestra.run.smithi183.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 580 kB/s | 43 kB 00:00 2024-04-25T14:26:22.380 INFO:teuthology.orchestra.run.smithi053.stdout:(38/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 2.4 MB/s | 762 kB 00:00 2024-04-25T14:26:22.442 INFO:teuthology.orchestra.run.smithi183.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 484 kB/s | 32 kB 00:00 2024-04-25T14:26:22.471 INFO:teuthology.orchestra.run.smithi150.stdout:(26/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 3.5 MB/s | 5.4 MB 00:01 2024-04-25T14:26:22.538 INFO:teuthology.orchestra.run.smithi150.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 2.5 MB/s | 170 kB 00:00 2024-04-25T14:26:22.550 INFO:teuthology.orchestra.run.smithi183.stdout:(115/152): python3-certifi-2021.10.8-2.el9.noar 138 kB/s | 15 kB 00:00 2024-04-25T14:26:22.555 INFO:teuthology.orchestra.run.smithi053.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 293 kB/s | 80 kB 00:00 2024-04-25T14:26:22.588 INFO:teuthology.orchestra.run.smithi150.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 687 kB/s | 34 kB 00:00 2024-04-25T14:26:22.651 INFO:teuthology.orchestra.run.smithi183.stdout:(116/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-04-25T14:26:22.655 INFO:teuthology.orchestra.run.smithi150.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 2.1 MB/s | 142 kB 00:00 2024-04-25T14:26:22.759 INFO:teuthology.orchestra.run.smithi183.stdout:(117/152): libarrow-9.0.0-10.el9.x86_64.rpm 3.2 MB/s | 4.4 MB 00:01 2024-04-25T14:26:22.809 INFO:teuthology.orchestra.run.smithi183.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 2.2 MB/s | 358 kB 00:00 2024-04-25T14:26:22.886 INFO:teuthology.orchestra.run.smithi183.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-04-25T14:26:22.889 INFO:teuthology.orchestra.run.smithi150.stdout:(30/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 3.9 MB/s | 3.1 MB 00:00 2024-04-25T14:26:22.910 INFO:teuthology.orchestra.run.smithi183.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 352 kB/s | 35 kB 00:00 2024-04-25T14:26:22.960 INFO:teuthology.orchestra.run.smithi183.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-04-25T14:26:23.035 INFO:teuthology.orchestra.run.smithi183.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-04-25T14:26:23.102 INFO:teuthology.orchestra.run.smithi183.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 160 kB/s | 11 kB 00:00 2024-04-25T14:26:23.156 INFO:teuthology.orchestra.run.smithi150.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 7.1 MB/s | 3.5 MB 00:00 2024-04-25T14:26:23.172 INFO:teuthology.orchestra.run.smithi053.stdout:(40/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 9.6 MB/s | 49 MB 00:05 2024-04-25T14:26:23.178 INFO:teuthology.orchestra.run.smithi183.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 236 kB/s | 18 kB 00:00 2024-04-25T14:26:23.206 INFO:teuthology.orchestra.run.smithi053.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 301 kB/s | 195 kB 00:00 2024-04-25T14:26:23.232 INFO:teuthology.orchestra.run.smithi150.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 3.2 MB/s | 249 kB 00:00 2024-04-25T14:26:23.245 INFO:teuthology.orchestra.run.smithi183.stdout:(125/152): python3-jaraco-collections-3.0.0-8.e 347 kB/s | 23 kB 00:00 2024-04-25T14:26:23.248 INFO:teuthology.orchestra.run.smithi053.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 803 kB 00:00 2024-04-25T14:26:23.282 INFO:teuthology.orchestra.run.smithi150.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 989 kB/s | 49 kB 00:00 2024-04-25T14:26:23.320 INFO:teuthology.orchestra.run.smithi183.stdout:(126/152): python3-jaraco-functools-3.5.0-2.el9 259 kB/s | 19 kB 00:00 2024-04-25T14:26:23.332 INFO:teuthology.orchestra.run.smithi150.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 328 kB/s | 16 kB 00:00 2024-04-25T14:26:23.340 INFO:teuthology.orchestra.run.smithi053.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 9.5 MB/s | 1.2 MB 00:00 2024-04-25T14:26:23.387 INFO:teuthology.orchestra.run.smithi183.stdout:(127/152): python3-grpcio-1.46.7-10.el9.x86_64. 4.3 MB/s | 2.0 MB 00:00 2024-04-25T14:26:23.398 INFO:teuthology.orchestra.run.smithi053.stdout:(44/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-04-25T14:26:23.399 INFO:teuthology.orchestra.run.smithi150.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 4.0 MB/s | 273 kB 00:00 2024-04-25T14:26:23.412 INFO:teuthology.orchestra.run.smithi183.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 214 kB/s | 20 kB 00:00 2024-04-25T14:26:23.424 INFO:teuthology.orchestra.run.smithi053.stdout:(45/152): python3-ply-3.11-14.el9.noarch.rpm 610 kB/s | 106 kB 00:00 2024-04-25T14:26:23.474 INFO:teuthology.orchestra.run.smithi053.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.7 MB/s | 125 kB 00:00 2024-04-25T14:26:23.496 INFO:teuthology.orchestra.run.smithi183.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 555 kB/s | 46 kB 00:00 2024-04-25T14:26:23.507 INFO:teuthology.orchestra.run.smithi053.stdout:(47/152): python3-pycparser-2.20-6.el9.noarch.r 822 kB/s | 135 kB 00:00 2024-04-25T14:26:23.532 INFO:teuthology.orchestra.run.smithi150.stdout:(36/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 5.6 MB/s | 762 kB 00:00 2024-04-25T14:26:23.533 INFO:teuthology.orchestra.run.smithi053.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-04-25T14:26:23.563 INFO:teuthology.orchestra.run.smithi183.stdout:(130/152): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2024-04-25T14:26:23.574 INFO:teuthology.orchestra.run.smithi150.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 440 kB/s | 18 kB 00:00 2024-04-25T14:26:23.621 INFO:teuthology.orchestra.run.smithi183.stdout:(131/152): python3-kubernetes-26.1.0-2.el9.noar 4.4 MB/s | 1.0 MB 00:00 2024-04-25T14:26:23.646 INFO:teuthology.orchestra.run.smithi183.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 691 kB/s | 58 kB 00:00 2024-04-25T14:26:23.675 INFO:teuthology.orchestra.run.smithi150.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 803 kB/s | 80 kB 00:00 2024-04-25T14:26:23.697 INFO:teuthology.orchestra.run.smithi183.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-04-25T14:26:23.723 INFO:teuthology.orchestra.run.smithi183.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 217 kB/s | 16 kB 00:00 2024-04-25T14:26:23.742 INFO:teuthology.orchestra.run.smithi150.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 12 MB/s | 803 kB 00:00 2024-04-25T14:26:23.764 INFO:teuthology.orchestra.run.smithi183.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-04-25T14:26:23.775 INFO:teuthology.orchestra.run.smithi150.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 5.8 MB/s | 195 kB 00:00 2024-04-25T14:26:23.790 INFO:teuthology.orchestra.run.smithi183.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 423 kB/s | 28 kB 00:00 2024-04-25T14:26:23.791 INFO:teuthology.orchestra.run.smithi053.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 416 kB/s | 107 kB 00:00 2024-04-25T14:26:23.817 INFO:teuthology.orchestra.run.smithi150.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 6.0 MB/s | 253 kB 00:00 2024-04-25T14:26:23.831 INFO:teuthology.orchestra.run.smithi183.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 460 kB/s | 31 kB 00:00 2024-04-25T14:26:23.842 INFO:teuthology.orchestra.run.smithi053.stdout:(50/152): automake-1.16.2-8.el9.noarch.rpm 1.9 MB/s | 689 kB 00:00 2024-04-25T14:26:23.865 INFO:teuthology.orchestra.run.smithi183.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-04-25T14:26:23.883 INFO:teuthology.orchestra.run.smithi053.stdout:(51/152): bison-3.7.4-5.el9.x86_64.rpm 2.6 MB/s | 943 kB 00:00 2024-04-25T14:26:23.884 INFO:teuthology.orchestra.run.smithi150.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 18 MB/s | 1.2 MB 00:00 2024-04-25T14:26:23.898 INFO:teuthology.orchestra.run.smithi183.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 885 kB/s | 59 kB 00:00 2024-04-25T14:26:23.909 INFO:teuthology.orchestra.run.smithi053.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 196 kB/s | 21 kB 00:00 2024-04-25T14:26:23.918 INFO:teuthology.orchestra.run.smithi150.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 3.1 MB/s | 106 kB 00:00 2024-04-25T14:26:23.932 INFO:teuthology.orchestra.run.smithi183.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 536 kB/s | 36 kB 00:00 2024-04-25T14:26:23.942 INFO:teuthology.orchestra.run.smithi053.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 3.2 MB/s | 313 kB 00:00 2024-04-25T14:26:23.951 INFO:teuthology.orchestra.run.smithi150.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 4.0 MB/s | 135 kB 00:00 2024-04-25T14:26:23.984 INFO:teuthology.orchestra.run.smithi053.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 459 kB/s | 33 kB 00:00 2024-04-25T14:26:23.985 INFO:teuthology.orchestra.run.smithi150.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 3.7 MB/s | 125 kB 00:00 2024-04-25T14:26:23.991 INFO:teuthology.orchestra.run.smithi183.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 554 kB/s | 51 kB 00:00 2024-04-25T14:26:24.016 INFO:teuthology.orchestra.run.smithi183.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-25T14:26:24.027 INFO:teuthology.orchestra.run.smithi150.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 5.1 MB/s | 215 kB 00:00 2024-04-25T14:26:24.077 INFO:teuthology.orchestra.run.smithi150.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 13 MB/s | 689 kB 00:00 2024-04-25T14:26:24.100 INFO:teuthology.orchestra.run.smithi053.stdout:(55/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 16 MB/s | 3.0 MB 00:00 2024-04-25T14:26:24.126 INFO:teuthology.orchestra.run.smithi053.stdout:(56/152): flexiblas-openblas-openmp-3.0.4-8.el9 109 kB/s | 18 kB 00:00 2024-04-25T14:26:24.152 INFO:teuthology.orchestra.run.smithi053.stdout:(57/152): libaio-devel-0.3.111-13.el9.x86_64.rp 73 kB/s | 11 kB 00:00 2024-04-25T14:26:24.185 INFO:teuthology.orchestra.run.smithi053.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-25T14:26:24.186 INFO:teuthology.orchestra.run.smithi150.stdout:(48/152): ceph-mgr-diskprediction-local-19.0.0- 5.7 MB/s | 7.4 MB 00:01 2024-04-25T14:26:24.210 INFO:teuthology.orchestra.run.smithi053.stdout:(59/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 549 kB/s | 45 kB 00:00 2024-04-25T14:26:24.220 INFO:teuthology.orchestra.run.smithi150.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 6.5 MB/s | 943 kB 00:00 2024-04-25T14:26:24.224 INFO:teuthology.orchestra.run.smithi183.stdout:(143/152): ceph-common-19.0.0-3156.gd69f13d8.el 1.3 MB/s | 21 MB 00:15 2024-04-25T14:26:24.245 INFO:teuthology.orchestra.run.smithi150.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-04-25T14:26:24.250 INFO:teuthology.orchestra.run.smithi183.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 347 kB/s | 90 kB 00:00 2024-04-25T14:26:24.252 INFO:teuthology.orchestra.run.smithi053.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 6.6 MB/s | 662 kB 00:00 2024-04-25T14:26:24.270 INFO:teuthology.orchestra.run.smithi150.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 430 kB/s | 21 kB 00:00 2024-04-25T14:26:24.275 INFO:teuthology.orchestra.run.smithi183.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 1.6 MB/s | 427 kB 00:00 2024-04-25T14:26:24.286 INFO:teuthology.orchestra.run.smithi053.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 8.4 MB/s | 582 kB 00:00 2024-04-25T14:26:24.303 INFO:teuthology.orchestra.run.smithi150.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 313 kB 00:00 2024-04-25T14:26:24.314 INFO:teuthology.orchestra.run.smithi053.stdout:(62/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 274 kB/s | 25 kB 00:00 2024-04-25T14:26:24.325 INFO:teuthology.orchestra.run.smithi183.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-25T14:26:24.329 INFO:teuthology.orchestra.run.smithi150.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 559 kB/s | 33 kB 00:00 2024-04-25T14:26:24.336 INFO:teuthology.orchestra.run.smithi053.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 584 kB/s | 47 kB 00:00 2024-04-25T14:26:24.370 INFO:teuthology.orchestra.run.smithi053.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.0 MB/s | 292 kB 00:00 2024-04-25T14:26:24.392 INFO:teuthology.orchestra.run.smithi183.stdout:(147/152): python3-zc-lockfile-2.0-10.el9.noarc 119 kB/s | 20 kB 00:00 2024-04-25T14:26:24.395 INFO:teuthology.orchestra.run.smithi053.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-04-25T14:26:24.404 INFO:teuthology.orchestra.run.smithi150.stdout:(54/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 30 MB/s | 3.0 MB 00:00 2024-04-25T14:26:24.420 INFO:teuthology.orchestra.run.smithi053.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 449 kB/s | 35 kB 00:00 2024-04-25T14:26:24.430 INFO:teuthology.orchestra.run.smithi150.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 185 kB/s | 18 kB 00:00 2024-04-25T14:26:24.455 INFO:teuthology.orchestra.run.smithi150.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-04-25T14:26:24.489 INFO:teuthology.orchestra.run.smithi150.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-25T14:26:24.514 INFO:teuthology.orchestra.run.smithi150.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 771 kB/s | 45 kB 00:00 2024-04-25T14:26:24.556 INFO:teuthology.orchestra.run.smithi150.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-04-25T14:26:24.589 INFO:teuthology.orchestra.run.smithi150.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 582 kB 00:00 2024-04-25T14:26:24.592 INFO:teuthology.orchestra.run.smithi183.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 5.0 MB/s | 1.6 MB 00:00 2024-04-25T14:26:24.596 INFO:teuthology.orchestra.run.smithi053.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 22 MB/s | 4.4 MB 00:00 2024-04-25T14:26:24.623 INFO:teuthology.orchestra.run.smithi150.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 385 kB/s | 25 kB 00:00 2024-04-25T14:26:24.649 INFO:teuthology.orchestra.run.smithi150.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 807 kB/s | 47 kB 00:00 2024-04-25T14:26:24.659 INFO:teuthology.orchestra.run.smithi183.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 20 MB/s | 1.3 MB 00:00 2024-04-25T14:26:24.674 INFO:teuthology.orchestra.run.smithi150.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-25T14:26:24.699 INFO:teuthology.orchestra.run.smithi150.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-25T14:26:24.704 INFO:teuthology.orchestra.run.smithi053.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 15 MB/s | 4.7 MB 00:00 2024-04-25T14:26:24.724 INFO:teuthology.orchestra.run.smithi150.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 708 kB/s | 35 kB 00:00 2024-04-25T14:26:24.726 INFO:teuthology.orchestra.run.smithi183.stdout:(150/152): librados2-19.0.0-3156.gd69f13d8.el7. 8.5 MB/s | 3.4 MB 00:00 2024-04-25T14:26:24.763 INFO:teuthology.orchestra.run.smithi053.stdout:(69/152): postgresql-13.14-1.el9.x86_64.rpm 4.9 MB/s | 1.6 MB 00:00 2024-04-25T14:26:24.810 INFO:teuthology.orchestra.run.smithi183.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 16 MB/s | 2.4 MB 00:00 2024-04-25T14:26:24.821 INFO:teuthology.orchestra.run.smithi053.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 4.3 MB/s | 911 kB 00:00 2024-04-25T14:26:24.846 INFO:teuthology.orchestra.run.smithi053.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 1.0 MB/s | 133 kB 00:00 2024-04-25T14:26:24.866 INFO:teuthology.orchestra.run.smithi150.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 28 MB/s | 4.7 MB 00:00 2024-04-25T14:26:24.992 INFO:teuthology.orchestra.run.smithi150.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-04-25T14:26:25.030 INFO:teuthology.orchestra.run.smithi053.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 4.9 MB/s | 1.0 MB 00:00 2024-04-25T14:26:25.033 INFO:teuthology.orchestra.run.smithi150.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 9.7 MB/s | 1.6 MB 00:00 2024-04-25T14:26:25.067 INFO:teuthology.orchestra.run.smithi150.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-04-25T14:26:25.092 INFO:teuthology.orchestra.run.smithi150.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.3 MB/s | 133 kB 00:00 2024-04-25T14:26:25.147 INFO:teuthology.orchestra.run.smithi053.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 18 MB/s | 5.8 MB 00:00 2024-04-25T14:26:25.193 INFO:teuthology.orchestra.run.smithi053.stdout:(74/152): python3-attrs-20.3.0-7.el9.noarch.rpm 257 kB/s | 87 kB 00:00 2024-04-25T14:26:25.226 INFO:teuthology.orchestra.run.smithi150.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 7.6 MB/s | 1.0 MB 00:00 2024-04-25T14:26:25.310 INFO:teuthology.orchestra.run.smithi150.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 24 MB/s | 5.8 MB 00:00 2024-04-25T14:26:25.335 INFO:teuthology.orchestra.run.smithi150.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 824 kB/s | 87 kB 00:00 2024-04-25T14:26:25.348 INFO:teuthology.orchestra.run.smithi053.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 19 MB/s | 6.0 MB 00:00 2024-04-25T14:26:25.373 INFO:teuthology.orchestra.run.smithi053.stdout:(76/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.5 MB/s | 244 kB 00:00 2024-04-25T14:26:25.406 INFO:teuthology.orchestra.run.smithi053.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.2 MB/s | 248 kB 00:00 2024-04-25T14:26:25.440 INFO:teuthology.orchestra.run.smithi053.stdout:(78/152): python3-jsonpatch-1.21-16.el9.noarch. 393 kB/s | 26 kB 00:00 2024-04-25T14:26:25.465 INFO:teuthology.orchestra.run.smithi053.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 212 kB/s | 19 kB 00:00 2024-04-25T14:26:25.516 INFO:teuthology.orchestra.run.smithi053.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.6 MB/s | 172 kB 00:00 2024-04-25T14:26:25.518 INFO:teuthology.orchestra.run.smithi150.stdout:(74/152): python3-babel-2.9.1-2.el9.noarch.rpm 29 MB/s | 6.0 MB 00:00 2024-04-25T14:26:25.544 INFO:teuthology.orchestra.run.smithi150.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.1 MB/s | 244 kB 00:00 2024-04-25T14:26:25.582 INFO:teuthology.orchestra.run.smithi053.stdout:(81/152): python3-markupsafe-1.1.1-12.el9.x86_6 304 kB/s | 35 kB 00:00 2024-04-25T14:26:25.602 INFO:teuthology.orchestra.run.smithi150.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-04-25T14:26:25.627 INFO:teuthology.orchestra.run.smithi150.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 307 kB/s | 26 kB 00:00 2024-04-25T14:26:25.661 INFO:teuthology.orchestra.run.smithi150.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 327 kB/s | 19 kB 00:00 2024-04-25T14:26:25.686 INFO:teuthology.orchestra.run.smithi150.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-25T14:26:25.708 INFO:teuthology.orchestra.run.smithi053.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 21 MB/s | 5.1 MB 00:00 2024-04-25T14:26:25.712 INFO:teuthology.orchestra.run.smithi150.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 696 kB/s | 35 kB 00:00 2024-04-25T14:26:25.733 INFO:teuthology.orchestra.run.smithi053.stdout:(83/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.2 MB/s | 250 kB 00:00 2024-04-25T14:26:25.767 INFO:teuthology.orchestra.run.smithi053.stdout:(84/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.3 MB/s | 222 kB 00:00 2024-04-25T14:26:25.808 INFO:teuthology.orchestra.run.smithi053.stdout:(85/152): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 77 kB 00:00 2024-04-25T14:26:25.825 INFO:teuthology.orchestra.run.smithi053.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 476 kB/s | 42 kB 00:00 2024-04-25T14:26:25.859 INFO:teuthology.orchestra.run.smithi053.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 2.9 MB/s | 269 kB 00:00 2024-04-25T14:26:25.868 INFO:teuthology.orchestra.run.smithi183.stdout:(152/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 2.1 MB/s | 3.1 MB 00:01 2024-04-25T14:26:25.874 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2024-04-25T14:26:25.874 INFO:teuthology.orchestra.run.smithi183.stdout:Total 12 MB/s | 222 MB 00:18 2024-04-25T14:26:25.887 INFO:teuthology.orchestra.run.smithi150.stdout:(81/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 25 MB/s | 5.1 MB 00:00 2024-04-25T14:26:25.892 INFO:teuthology.orchestra.run.smithi053.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-04-25T14:26:25.912 INFO:teuthology.orchestra.run.smithi150.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.2 MB/s | 250 kB 00:00 2024-04-25T14:26:25.934 INFO:teuthology.orchestra.run.smithi053.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.6 MB/s | 279 kB 00:00 2024-04-25T14:26:25.946 INFO:teuthology.orchestra.run.smithi150.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-04-25T14:26:25.951 INFO:teuthology.orchestra.run.smithi053.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 585 kB/s | 51 kB 00:00 2024-04-25T14:26:25.971 INFO:teuthology.orchestra.run.smithi150.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-25T14:26:25.976 INFO:teuthology.orchestra.run.smithi053.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 54 kB 00:00 2024-04-25T14:26:25.997 INFO:teuthology.orchestra.run.smithi150.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 42 kB 00:00 2024-04-25T14:26:26.030 INFO:teuthology.orchestra.run.smithi150.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-04-25T14:26:26.055 INFO:teuthology.orchestra.run.smithi150.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-04-25T14:26:26.097 INFO:teuthology.orchestra.run.smithi150.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-04-25T14:26:26.123 INFO:teuthology.orchestra.run.smithi150.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 764 kB/s | 51 kB 00:00 2024-04-25T14:26:26.152 INFO:teuthology.orchestra.run.smithi053.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 217 kB/s | 42 kB 00:00 2024-04-25T14:26:26.183 INFO:teuthology.orchestra.run.smithi150.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 626 kB/s | 54 kB 00:00 2024-04-25T14:26:26.327 INFO:teuthology.orchestra.run.smithi053.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 132 kB/s | 44 kB 00:00 2024-04-25T14:26:26.350 INFO:teuthology.orchestra.run.smithi150.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 253 kB/s | 42 kB 00:00 2024-04-25T14:26:26.553 INFO:teuthology.orchestra.run.smithi053.stdout:(94/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-04-25T14:26:26.578 INFO:teuthology.orchestra.run.smithi053.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 538 kB/s | 221 kB 00:00 2024-04-25T14:26:26.602 INFO:teuthology.orchestra.run.smithi053.stdout:(96/152): qatlib-service-24.02.0-1.el9.x86_64.r 138 kB/s | 36 kB 00:00 2024-04-25T14:26:26.668 INFO:teuthology.orchestra.run.smithi053.stdout:(97/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 534 kB/s | 47 kB 00:00 2024-04-25T14:26:26.703 INFO:teuthology.orchestra.run.smithi053.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 2.4 MB/s | 305 kB 00:00 2024-04-25T14:26:26.728 INFO:teuthology.orchestra.run.smithi053.stdout:(99/152): uuid-1.6.2-55.el9.x86_64.rpm 476 kB/s | 58 kB 00:00 2024-04-25T14:26:26.745 INFO:teuthology.orchestra.run.smithi053.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 2.3 MB/s | 94 kB 00:00 2024-04-25T14:26:26.770 INFO:teuthology.orchestra.run.smithi053.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 977 kB/s | 64 kB 00:00 2024-04-25T14:26:26.796 INFO:teuthology.orchestra.run.smithi053.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 693 kB/s | 45 kB 00:00 2024-04-25T14:26:26.817 INFO:teuthology.orchestra.run.smithi150.stdout:(92/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 5.9 MB/s | 49 MB 00:08 2024-04-25T14:26:26.829 INFO:teuthology.orchestra.run.smithi053.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 391 kB/s | 22 kB 00:00 2024-04-25T14:26:26.871 INFO:teuthology.orchestra.run.smithi053.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 8.6 MB/s | 863 kB 00:00 2024-04-25T14:26:27.001 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2024-04-25T14:26:27.085 INFO:teuthology.orchestra.run.smithi150.stdout:(93/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 17 MB/s | 16 MB 00:00 2024-04-25T14:26:27.101 INFO:teuthology.orchestra.run.smithi150.stdout:(94/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 59 kB/s | 44 kB 00:00 2024-04-25T14:26:27.126 INFO:teuthology.orchestra.run.smithi150.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 722 kB/s | 221 kB 00:00 2024-04-25T14:26:27.143 INFO:teuthology.orchestra.run.smithi150.stdout:(96/152): qatlib-service-24.02.0-1.el9.x86_64.r 622 kB/s | 36 kB 00:00 2024-04-25T14:26:27.144 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2024-04-25T14:26:27.144 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2024-04-25T14:26:27.160 INFO:teuthology.orchestra.run.smithi150.stdout:(97/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-04-25T14:26:27.185 INFO:teuthology.orchestra.run.smithi150.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 305 kB 00:00 2024-04-25T14:26:27.202 INFO:teuthology.orchestra.run.smithi150.stdout:(99/152): uuid-1.6.2-55.el9.x86_64.rpm 986 kB/s | 58 kB 00:00 2024-04-25T14:26:27.228 INFO:teuthology.orchestra.run.smithi150.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.4 MB/s | 94 kB 00:00 2024-04-25T14:26:27.261 INFO:teuthology.orchestra.run.smithi150.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 772 kB/s | 45 kB 00:00 2024-04-25T14:26:27.278 INFO:teuthology.orchestra.run.smithi150.stdout:(102/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 848 kB/s | 64 kB 00:00 2024-04-25T14:26:27.295 INFO:teuthology.orchestra.run.smithi150.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 332 kB/s | 22 kB 00:00 2024-04-25T14:26:27.296 INFO:teuthology.orchestra.run.smithi053.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 46 kB/s | 19 kB 00:00 2024-04-25T14:26:27.345 INFO:teuthology.orchestra.run.smithi150.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 10 MB/s | 863 kB 00:00 2024-04-25T14:26:27.489 INFO:teuthology.orchestra.run.smithi053.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 501 kB/s | 309 kB 00:00 2024-04-25T14:26:27.556 INFO:teuthology.orchestra.run.smithi053.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 731 kB/s | 548 kB 00:00 2024-04-25T14:26:27.604 INFO:teuthology.orchestra.run.smithi150.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 75 kB/s | 19 kB 00:00 2024-04-25T14:26:27.779 INFO:teuthology.orchestra.run.smithi150.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 639 kB/s | 309 kB 00:00 2024-04-25T14:26:27.889 INFO:teuthology.orchestra.run.smithi053.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 65 kB/s | 26 kB 00:00 2024-04-25T14:26:27.896 INFO:teuthology.orchestra.run.smithi150.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 223 kB/s | 26 kB 00:00 2024-04-25T14:26:27.931 INFO:teuthology.orchestra.run.smithi053.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 130 kB/s | 49 kB 00:00 2024-04-25T14:26:27.988 INFO:teuthology.orchestra.run.smithi150.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 531 kB/s | 49 kB 00:00 2024-04-25T14:26:28.063 INFO:teuthology.orchestra.run.smithi150.stdout:(109/152): abseil-cpp-20211102.0-3.el9.x86_64.r 699 kB/s | 548 kB 00:00 2024-04-25T14:26:28.082 INFO:teuthology.orchestra.run.smithi053.stdout:(110/152): libarrow-9.0.0-10.el9.x86_64.rpm 5.6 MB/s | 4.4 MB 00:00 2024-04-25T14:26:28.106 INFO:teuthology.orchestra.run.smithi053.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 332 kB/s | 67 kB 00:00 2024-04-25T14:26:28.147 INFO:teuthology.orchestra.run.smithi150.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 425 kB/s | 67 kB 00:00 2024-04-25T14:26:28.170 INFO:teuthology.orchestra.run.smithi053.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 649 kB/s | 151 kB 00:00 2024-04-25T14:26:28.240 INFO:teuthology.orchestra.run.smithi053.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 6.3 MB/s | 838 kB 00:00 2024-04-25T14:26:28.282 INFO:teuthology.orchestra.run.smithi053.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 3.1 MB/s | 548 kB 00:00 2024-04-25T14:26:28.307 INFO:teuthology.orchestra.run.smithi053.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 322 kB/s | 43 kB 00:00 2024-04-25T14:26:28.322 INFO:teuthology.orchestra.run.smithi150.stdout:(111/152): libarrow-9.0.0-10.el9.x86_64.rpm 6.2 MB/s | 4.4 MB 00:00 2024-04-25T14:26:28.333 INFO:teuthology.orchestra.run.smithi053.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 377 kB/s | 32 kB 00:00 2024-04-25T14:26:28.347 INFO:teuthology.orchestra.run.smithi150.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 532 kB/s | 151 kB 00:00 2024-04-25T14:26:28.381 INFO:teuthology.orchestra.run.smithi150.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.5 MB/s | 838 kB 00:00 2024-04-25T14:26:28.391 INFO:teuthology.orchestra.run.smithi053.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 183 kB/s | 15 kB 00:00 2024-04-25T14:26:28.414 INFO:teuthology.orchestra.run.smithi150.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 5.8 MB/s | 548 kB 00:00 2024-04-25T14:26:28.416 INFO:teuthology.orchestra.run.smithi053.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.6 MB/s | 172 kB 00:00 2024-04-25T14:26:28.440 INFO:teuthology.orchestra.run.smithi150.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 472 kB/s | 43 kB 00:00 2024-04-25T14:26:28.450 INFO:teuthology.orchestra.run.smithi053.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-04-25T14:26:28.465 INFO:teuthology.orchestra.run.smithi150.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 384 kB/s | 32 kB 00:00 2024-04-25T14:26:28.492 INFO:teuthology.orchestra.run.smithi053.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.0 MB/s | 303 kB 00:00 2024-04-25T14:26:28.499 INFO:teuthology.orchestra.run.smithi150.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 179 kB/s | 15 kB 00:00 2024-04-25T14:26:28.549 INFO:teuthology.orchestra.run.smithi150.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-04-25T14:26:28.551 INFO:teuthology.orchestra.run.smithi053.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 271 kB/s | 35 kB 00:00 2024-04-25T14:26:28.583 INFO:teuthology.orchestra.run.smithi150.stdout:(119/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.2 MB/s | 172 kB 00:00 2024-04-25T14:26:28.585 INFO:teuthology.orchestra.run.smithi053.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 1.5 MB/s | 206 kB 00:00 2024-04-25T14:26:28.608 INFO:teuthology.orchestra.run.smithi150.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-04-25T14:26:28.634 INFO:teuthology.orchestra.run.smithi150.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 419 kB/s | 35 kB 00:00 2024-04-25T14:26:28.660 INFO:teuthology.orchestra.run.smithi053.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-04-25T14:26:28.685 INFO:teuthology.orchestra.run.smithi053.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-04-25T14:26:28.710 INFO:teuthology.orchestra.run.smithi053.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 95 kB/s | 11 kB 00:00 2024-04-25T14:26:28.734 INFO:teuthology.orchestra.run.smithi150.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 16 MB/s | 2.0 MB 00:00 2024-04-25T14:26:28.744 INFO:teuthology.orchestra.run.smithi053.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 217 kB/s | 18 kB 00:00 2024-04-25T14:26:28.759 INFO:teuthology.orchestra.run.smithi150.stdout:(123/152): python3-google-auth-2.29.0-1.el9.noa 1.1 MB/s | 206 kB 00:00 2024-04-25T14:26:28.784 INFO:teuthology.orchestra.run.smithi150.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 960 kB/s | 144 kB 00:00 2024-04-25T14:26:28.794 INFO:teuthology.orchestra.run.smithi053.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 280 kB/s | 23 kB 00:00 2024-04-25T14:26:28.809 INFO:teuthology.orchestra.run.smithi150.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 141 kB/s | 11 kB 00:00 2024-04-25T14:26:28.819 INFO:teuthology.orchestra.run.smithi053.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 181 kB/s | 19 kB 00:00 2024-04-25T14:26:28.843 INFO:teuthology.orchestra.run.smithi150.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 211 kB/s | 18 kB 00:00 2024-04-25T14:26:28.845 INFO:teuthology.orchestra.run.smithi053.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 197 kB/s | 20 kB 00:00 2024-04-25T14:26:28.868 INFO:teuthology.orchestra.run.smithi150.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 276 kB/s | 23 kB 00:00 2024-04-25T14:26:28.894 INFO:teuthology.orchestra.run.smithi150.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 231 kB/s | 19 kB 00:00 2024-04-25T14:26:28.910 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2024-04-25T14:26:28.911 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2024-04-25T14:26:28.936 INFO:teuthology.orchestra.run.smithi150.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 214 kB/s | 20 kB 00:00 2024-04-25T14:26:28.945 INFO:teuthology.orchestra.run.smithi053.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 8.2 MB/s | 1.0 MB 00:00 2024-04-25T14:26:28.970 INFO:teuthology.orchestra.run.smithi053.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 311 kB/s | 46 kB 00:00 2024-04-25T14:26:28.977 INFO:teuthology.orchestra.run.smithi150.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 555 kB/s | 46 kB 00:00 2024-04-25T14:26:28.995 INFO:teuthology.orchestra.run.smithi053.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 526 kB/s | 79 kB 00:00 2024-04-25T14:26:29.019 INFO:teuthology.orchestra.run.smithi150.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 945 kB/s | 79 kB 00:00 2024-04-25T14:26:29.037 INFO:teuthology.orchestra.run.smithi053.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 641 kB/s | 58 kB 00:00 2024-04-25T14:26:29.061 INFO:teuthology.orchestra.run.smithi150.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 690 kB/s | 58 kB 00:00 2024-04-25T14:26:29.088 INFO:teuthology.orchestra.run.smithi053.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-04-25T14:26:29.112 INFO:teuthology.orchestra.run.smithi150.stdout:(133/152): python3-kubernetes-26.1.0-2.el9.noar 4.2 MB/s | 1.0 MB 00:00 2024-04-25T14:26:29.113 INFO:teuthology.orchestra.run.smithi053.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 142 kB/s | 16 kB 00:00 2024-04-25T14:26:29.130 INFO:teuthology.orchestra.run.smithi053.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 983 kB/s | 90 kB 00:00 2024-04-25T14:26:29.145 INFO:teuthology.orchestra.run.smithi150.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 197 kB/s | 16 kB 00:00 2024-04-25T14:26:29.195 INFO:teuthology.orchestra.run.smithi150.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-04-25T14:26:29.205 INFO:teuthology.orchestra.run.smithi053.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 313 kB/s | 28 kB 00:00 2024-04-25T14:26:29.221 INFO:teuthology.orchestra.run.smithi150.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 375 kB/s | 28 kB 00:00 2024-04-25T14:26:29.246 INFO:teuthology.orchestra.run.smithi150.stdout:(137/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.2 MB/s | 272 kB 00:00 2024-04-25T14:26:29.253 INFO:teuthology.orchestra.run.smithi053.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 223 kB/s | 31 kB 00:00 2024-04-25T14:26:29.271 INFO:teuthology.orchestra.run.smithi150.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-04-25T14:26:29.281 INFO:teuthology.orchestra.run.smithi053.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-04-25T14:26:29.313 INFO:teuthology.orchestra.run.smithi150.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 884 kB/s | 59 kB 00:00 2024-04-25T14:26:29.331 INFO:teuthology.orchestra.run.smithi053.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 794 kB/s | 59 kB 00:00 2024-04-25T14:26:29.347 INFO:teuthology.orchestra.run.smithi150.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 476 kB/s | 36 kB 00:00 2024-04-25T14:26:29.356 INFO:teuthology.orchestra.run.smithi053.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 360 kB/s | 36 kB 00:00 2024-04-25T14:26:29.372 INFO:teuthology.orchestra.run.smithi150.stdout:(141/152): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-04-25T14:26:29.381 INFO:teuthology.orchestra.run.smithi053.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 510 kB/s | 51 kB 00:00 2024-04-25T14:26:29.397 INFO:teuthology.orchestra.run.smithi150.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 609 kB/s | 51 kB 00:00 2024-04-25T14:26:29.431 INFO:teuthology.orchestra.run.smithi150.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-25T14:26:29.448 INFO:teuthology.orchestra.run.smithi053.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.5 MB/s | 230 kB 00:00 2024-04-25T14:26:29.456 INFO:teuthology.orchestra.run.smithi150.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-04-25T14:26:29.474 INFO:teuthology.orchestra.run.smithi053.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 778 kB/s | 90 kB 00:00 2024-04-25T14:26:29.498 INFO:teuthology.orchestra.run.smithi150.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-04-25T14:26:29.507 INFO:teuthology.orchestra.run.smithi053.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-04-25T14:26:29.515 INFO:teuthology.orchestra.run.smithi150.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 238 kB/s | 20 kB 00:00 2024-04-25T14:26:29.532 INFO:teuthology.orchestra.run.smithi053.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 256 kB/s | 20 kB 00:00 2024-04-25T14:26:29.599 INFO:teuthology.orchestra.run.smithi150.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-04-25T14:26:29.622 INFO:teuthology.orchestra.run.smithi053.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-04-25T14:26:29.624 INFO:teuthology.orchestra.run.smithi150.stdout:(148/152): re2-20211101-3.el9.x86_64.rpm 1.1 MB/s | 192 kB 00:00 2024-04-25T14:26:29.691 INFO:teuthology.orchestra.run.smithi053.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 8.7 MB/s | 1.6 MB 00:00 2024-04-25T14:26:29.707 INFO:teuthology.orchestra.run.smithi150.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 16 MB/s | 1.3 MB 00:00 2024-04-25T14:26:29.808 INFO:teuthology.orchestra.run.smithi053.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 12 MB/s | 1.3 MB 00:00 2024-04-25T14:26:29.849 INFO:teuthology.orchestra.run.smithi150.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 17 MB/s | 2.4 MB 00:00 2024-04-25T14:26:29.925 INFO:teuthology.orchestra.run.smithi053.stdout:(150/152): librados2-19.0.0-3156.gd69f13d8.el7. 8.7 MB/s | 3.4 MB 00:00 2024-04-25T14:26:30.050 INFO:teuthology.orchestra.run.smithi053.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 9.9 MB/s | 2.4 MB 00:00 2024-04-25T14:26:30.091 INFO:teuthology.orchestra.run.smithi150.stdout:(151/152): librados2-19.0.0-3156.gd69f13d8.el7. 5.9 MB/s | 3.4 MB 00:00 2024-04-25T14:26:30.142 INFO:teuthology.orchestra.run.smithi150.stdout:(152/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 5.8 MB/s | 3.1 MB 00:00 2024-04-25T14:26:30.146 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-04-25T14:26:30.146 INFO:teuthology.orchestra.run.smithi150.stdout:Total 11 MB/s | 222 MB 00:20 2024-04-25T14:26:30.759 INFO:teuthology.orchestra.run.smithi053.stdout:(152/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 2.8 MB/s | 3.1 MB 00:01 2024-04-25T14:26:30.780 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-04-25T14:26:30.780 INFO:teuthology.orchestra.run.smithi053.stdout:Total 12 MB/s | 222 MB 00:18 2024-04-25T14:26:31.276 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-04-25T14:26:31.421 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-04-25T14:26:31.421 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-04-25T14:26:33.196 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-04-25T14:26:33.197 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-04-25T14:26:33.626 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-04-25T14:26:33.883 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-04-25T14:26:33.883 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-04-25T14:26:34.858 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2024-04-25T14:26:34.938 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T14:26:34.977 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T14:26:35.030 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T14:26:35.106 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T14:26:35.131 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T14:26:35.175 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T14:26:35.555 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T14:26:35.607 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T14:26:35.729 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T14:26:35.766 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T14:26:35.858 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T14:26:35.887 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T14:26:36.040 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T14:26:36.092 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T14:26:36.120 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T14:26:36.206 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T14:26:36.230 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T14:26:36.276 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T14:26:36.297 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T14:26:36.319 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T14:26:36.505 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T14:26:36.538 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T14:26:36.645 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T14:26:36.732 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T14:26:36.787 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T14:26:36.824 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-04-25T14:26:36.826 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-04-25T14:26:36.830 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T14:26:36.895 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T14:26:36.929 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T14:26:36.973 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T14:26:37.006 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T14:26:37.033 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T14:26:37.102 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T14:26:37.155 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T14:26:37.186 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T14:26:37.222 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T14:26:37.289 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T14:26:37.324 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T14:26:37.501 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T14:26:37.621 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T14:26:37.692 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T14:26:37.736 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T14:26:37.819 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T14:26:37.956 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T14:26:38.023 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T14:26:38.556 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T14:26:38.634 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T14:26:38.663 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T14:26:38.706 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T14:26:38.744 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T14:26:38.785 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T14:26:38.944 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T14:26:38.997 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T14:26:39.051 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T14:26:39.124 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T14:26:39.156 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T14:26:39.192 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T14:26:39.224 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-04-25T14:26:39.239 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T14:26:39.275 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T14:26:39.318 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T14:26:39.354 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T14:26:39.360 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T14:26:39.496 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T14:26:39.540 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T14:26:39.552 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T14:26:39.571 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T14:26:39.606 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T14:26:39.612 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T14:26:39.655 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T14:26:39.680 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T14:26:39.706 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T14:26:39.724 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T14:26:39.737 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T14:26:39.777 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T14:26:39.803 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T14:26:39.827 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T14:26:39.902 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T14:26:40.029 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T14:26:40.060 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T14:26:40.068 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T14:26:40.096 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T14:26:40.130 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T14:26:40.169 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T14:26:40.181 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T14:26:40.201 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T14:26:40.231 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T14:26:40.265 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T14:26:40.298 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T14:26:40.302 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T14:26:40.332 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T14:26:40.340 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T14:26:40.374 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T14:26:40.423 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T14:26:40.462 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T14:26:40.556 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T14:26:40.605 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T14:26:40.615 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T14:26:40.645 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T14:26:40.666 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T14:26:40.691 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T14:26:40.695 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T14:26:40.789 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T14:26:40.798 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T14:26:40.820 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T14:26:40.838 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T14:26:40.883 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T14:26:40.905 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T14:26:40.931 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T14:26:41.122 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T14:26:41.154 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T14:26:41.261 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T14:26:41.375 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T14:26:41.412 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T14:26:41.447 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T14:26:41.512 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T14:26:41.546 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T14:26:41.560 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T14:26:41.589 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T14:26:41.609 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T14:26:41.622 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T14:26:41.643 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T14:26:41.650 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T14:26:41.720 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T14:26:41.742 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T14:26:41.772 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T14:26:41.793 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T14:26:41.802 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T14:26:41.838 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T14:26:41.880 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T14:26:41.914 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T14:26:41.920 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T14:26:41.949 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T14:26:41.992 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T14:26:42.021 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T14:26:42.092 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T14:26:42.135 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T14:26:42.135 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T14:26:42.168 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T14:26:42.211 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T14:26:42.245 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T14:26:42.346 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T14:26:42.396 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T14:26:42.889 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T14:26:42.951 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T14:26:43.007 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T14:26:43.044 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T14:26:43.071 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T14:26:43.114 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T14:26:43.152 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T14:26:43.194 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T14:26:43.360 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T14:26:43.422 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T14:26:43.477 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T14:26:43.550 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T14:26:43.581 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T14:26:43.618 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T14:26:43.664 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T14:26:43.700 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T14:26:43.726 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-04-25T14:26:43.743 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T14:26:43.786 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T14:26:43.883 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T14:26:43.923 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T14:26:43.968 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T14:26:43.994 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T14:26:44.082 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T14:26:44.188 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T14:26:44.202 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T14:26:44.234 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T14:26:44.245 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T14:26:44.297 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T14:26:44.313 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T14:26:44.341 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T14:26:44.374 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T14:26:44.376 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T14:26:44.393 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T14:26:44.429 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T14:26:44.469 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T14:26:44.503 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T14:26:44.528 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T14:26:44.603 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T14:26:44.688 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T14:26:44.729 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T14:26:44.738 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T14:26:44.768 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T14:26:44.796 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T14:26:44.819 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T14:26:44.823 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T14:26:44.830 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T14:26:44.850 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T14:26:44.869 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T14:26:44.887 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T14:26:44.910 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T14:26:44.940 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T14:26:44.943 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T14:26:44.966 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T14:26:44.999 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T14:26:45.041 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T14:26:45.092 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T14:26:45.163 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T14:26:45.188 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T14:26:45.193 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T14:26:45.230 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T14:26:45.261 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T14:26:45.309 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T14:26:45.313 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T14:26:45.366 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T14:26:45.371 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T14:26:45.398 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T14:26:45.417 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T14:26:45.469 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T14:26:45.472 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T14:26:45.491 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T14:26:45.718 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T14:26:45.785 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T14:26:45.814 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T14:26:45.969 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T14:26:45.998 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T14:26:46.092 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T14:26:46.124 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T14:26:46.185 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T14:26:46.212 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T14:26:46.260 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T14:26:46.294 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T14:26:46.396 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T14:26:46.453 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T14:26:46.491 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T14:26:46.541 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T14:26:46.541 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T14:26:46.579 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T14:26:46.730 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T14:26:46.756 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T14:26:46.793 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T14:26:46.876 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T14:26:46.880 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T14:26:46.916 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T14:26:46.966 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T14:26:47.072 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T14:26:47.144 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T14:26:47.215 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T14:26:47.269 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T14:26:47.316 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T14:26:47.442 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T14:26:47.537 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T14:26:47.552 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T14:26:47.592 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T14:26:47.653 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T14:26:47.659 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T14:26:47.759 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T14:26:47.774 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T14:26:47.795 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T14:26:47.820 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T14:26:47.871 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T14:26:47.910 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T14:26:48.034 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T14:26:48.108 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T14:26:48.222 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T14:26:48.338 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T14:26:48.391 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T14:26:48.407 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T14:26:48.482 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T14:26:48.482 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T14:26:48.482 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T14:26:48.482 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-25T14:26:48.489 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T14:26:48.508 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T14:26:48.895 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T14:26:48.981 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T14:26:49.009 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T14:26:49.042 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T14:26:49.188 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T14:26:49.353 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T14:26:49.388 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T14:26:49.400 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T14:26:49.442 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T14:26:49.462 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T14:26:49.491 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T14:26:49.493 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T14:26:49.528 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T14:26:49.538 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T14:26:49.624 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T14:26:49.907 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T14:26:49.979 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T14:26:50.006 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T14:26:50.033 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T14:26:50.062 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T14:26:50.104 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T14:26:50.134 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T14:26:50.172 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T14:26:50.210 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T14:26:50.259 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T14:26:50.330 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T14:26:50.374 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T14:26:50.464 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T14:26:50.527 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T14:26:50.730 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T14:26:50.814 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T14:26:50.853 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T14:26:50.916 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T14:26:51.056 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T14:26:51.139 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T14:26:51.195 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T14:26:51.234 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T14:26:51.267 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T14:26:51.454 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T14:26:51.652 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T14:26:51.697 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T14:26:51.741 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T14:26:51.788 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T14:26:51.851 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T14:26:51.910 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T14:26:51.951 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T14:26:51.997 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T14:26:52.046 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T14:26:52.124 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T14:26:52.178 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T14:26:52.350 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T14:26:52.392 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T14:26:52.412 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T14:26:52.453 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T14:26:52.489 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T14:26:52.527 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T14:26:52.716 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T14:26:52.784 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T14:26:53.026 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T14:26:53.068 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T14:26:53.158 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T14:26:53.158 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T14:26:53.158 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T14:26:53.158 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T14:26:53.185 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T14:26:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T14:26:53.840 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T14:26:53.881 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T14:26:54.013 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T14:26:54.085 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T14:26:54.198 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T14:26:54.249 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T14:26:54.500 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T14:26:54.538 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T14:26:54.658 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T14:26:55.576 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T14:26:55.732 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T14:26:57.410 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T14:26:57.511 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T14:26:57.563 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T14:26:57.615 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T14:26:58.047 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T14:26:58.090 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T14:26:58.213 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T14:26:58.247 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T14:26:58.305 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T14:26:58.908 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T14:26:58.943 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T14:26:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T14:26:59.094 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T14:26:59.982 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T14:26:59.982 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /sys 2024-04-25T14:26:59.983 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /proc 2024-04-25T14:26:59.983 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /mnt 2024-04-25T14:26:59.983 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /var/tmp 2024-04-25T14:26:59.983 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /home 2024-04-25T14:26:59.983 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /root 2024-04-25T14:26:59.983 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /tmp 2024-04-25T14:26:59.983 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-25T14:27:00.161 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T14:27:00.177 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T14:27:00.177 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:27:00.177 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T14:27:00.177 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-25T14:27:01.681 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T14:27:01.706 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T14:27:01.707 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:27:01.707 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T14:27:01.707 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T14:27:01.707 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T14:27:01.707 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-25T14:27:01.916 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T14:27:01.931 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T14:27:01.954 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T14:27:01.954 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:27:01.954 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T14:27:01.954 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T14:27:01.955 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T14:27:01.955 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-25T14:27:01.958 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T14:27:02.399 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T14:27:02.422 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T14:27:02.422 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:27:02.422 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T14:27:02.422 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T14:27:02.422 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T14:27:02.422 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-25T14:27:02.521 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T14:27:02.578 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T14:27:02.633 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T14:27:02.646 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T14:27:02.715 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T14:27:02.820 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T14:27:02.875 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T14:27:02.880 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T14:27:02.880 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T14:27:02.880 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T14:27:02.880 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:27:02.937 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T14:27:02.953 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T14:27:03.050 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T14:27:03.192 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T14:27:03.250 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T14:27:03.281 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T14:27:03.309 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T14:27:03.777 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T14:27:03.807 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T14:27:03.840 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T14:27:03.902 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T14:27:03.939 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T14:27:04.730 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T14:27:04.730 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /sys 2024-04-25T14:27:04.730 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /proc 2024-04-25T14:27:04.730 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /mnt 2024-04-25T14:27:04.730 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /var/tmp 2024-04-25T14:27:04.730 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /home 2024-04-25T14:27:04.730 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /root 2024-04-25T14:27:04.730 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /tmp 2024-04-25T14:27:04.730 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T14:27:04.898 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T14:27:04.915 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T14:27:04.915 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:27:04.915 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T14:27:04.915 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T14:27:04.989 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T14:27:05.032 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T14:27:06.001 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T14:27:06.032 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T14:27:06.134 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T14:27:06.306 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T14:27:06.334 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T14:27:06.358 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T14:27:06.358 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:27:06.358 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T14:27:06.358 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T14:27:06.358 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T14:27:06.358 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-25T14:27:06.402 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T14:27:06.413 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T14:27:06.418 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T14:27:06.444 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T14:27:06.444 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:27:06.444 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T14:27:06.444 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T14:27:06.444 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T14:27:06.444 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T14:27:06.510 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T14:27:06.558 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T14:27:06.668 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T14:27:06.691 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T14:27:06.691 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:27:06.691 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T14:27:06.692 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T14:27:06.692 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T14:27:06.692 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T14:27:07.127 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T14:27:07.151 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T14:27:07.151 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:27:07.151 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T14:27:07.151 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T14:27:07.151 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T14:27:07.151 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T14:27:07.258 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T14:27:07.315 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T14:27:07.370 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T14:27:07.480 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T14:27:07.513 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T14:27:07.535 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T14:27:07.536 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:27:07.536 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T14:27:07.536 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T14:27:07.536 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T14:27:07.536 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-25T14:27:07.559 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T14:27:07.579 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T14:27:07.602 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T14:27:07.602 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:27:07.602 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T14:27:07.602 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-25T14:27:07.612 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T14:27:07.655 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T14:27:07.677 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T14:27:07.677 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:27:07.678 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T14:27:07.678 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T14:27:07.678 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T14:27:07.678 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-25T14:27:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T14:27:07.788 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T14:27:07.929 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T14:27:07.970 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T14:27:07.981 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T14:27:08.002 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T14:27:08.004 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T14:27:08.004 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:27:08.004 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T14:27:08.004 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T14:27:08.004 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T14:27:08.004 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-25T14:27:08.021 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T14:27:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T14:27:08.436 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T14:27:08.469 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T14:27:08.531 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T14:27:08.568 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T14:27:09.606 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T14:27:09.644 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T14:27:10.603 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T14:27:10.636 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T14:27:10.739 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T14:27:10.907 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T14:27:10.939 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T14:27:10.962 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T14:27:10.962 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:27:10.962 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T14:27:10.963 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T14:27:10.963 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T14:27:10.963 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T14:27:11.014 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T14:27:11.026 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T14:27:11.122 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T14:27:11.175 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T14:27:12.091 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T14:27:12.126 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T14:27:12.149 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T14:27:12.149 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:27:12.149 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T14:27:12.150 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T14:27:12.150 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T14:27:12.150 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T14:27:12.194 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T14:27:12.215 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T14:27:12.215 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:27:12.215 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T14:27:12.216 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T14:27:12.260 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T14:27:12.282 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T14:27:12.283 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:27:12.283 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T14:27:12.283 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T14:27:12.283 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T14:27:12.283 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T14:27:12.594 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T14:27:12.601 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T14:27:12.617 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T14:27:12.617 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:27:12.617 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T14:27:12.617 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T14:27:12.617 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T14:27:12.617 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T14:27:12.642 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T14:27:12.691 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T14:27:12.724 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T14:27:12.750 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T14:27:12.937 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T14:27:12.980 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T14:27:13.010 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T14:27:13.600 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T14:27:13.640 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T14:27:13.660 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T14:27:14.500 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T14:27:14.595 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T14:27:14.636 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T14:27:14.720 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T14:27:14.749 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T14:27:14.749 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T14:27:14.825 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T14:27:14.857 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T14:27:14.858 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T14:27:14.933 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T14:27:14.997 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T14:27:16.213 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T14:27:16.213 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /sys 2024-04-25T14:27:16.213 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /proc 2024-04-25T14:27:16.213 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /mnt 2024-04-25T14:27:16.214 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /var/tmp 2024-04-25T14:27:16.214 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /home 2024-04-25T14:27:16.214 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /root 2024-04-25T14:27:16.214 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /tmp 2024-04-25T14:27:16.214 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:27:16.444 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T14:27:16.494 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T14:27:16.494 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:27:16.494 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T14:27:16.494 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:27:17.189 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T14:27:17.230 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T14:27:17.271 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T14:27:17.304 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T14:27:17.331 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T14:27:17.529 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T14:27:17.577 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T14:27:17.607 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T14:27:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T14:27:18.224 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T14:27:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T14:27:18.261 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:27:18.261 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T14:27:18.261 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T14:27:18.261 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T14:27:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:27:18.263 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T14:27:18.283 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T14:27:18.545 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T14:27:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T14:27:18.622 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:27:18.622 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T14:27:18.622 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T14:27:18.623 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T14:27:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:27:19.121 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T14:27:19.171 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T14:27:19.209 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T14:27:19.241 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T14:27:19.241 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:27:19.241 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T14:27:19.241 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T14:27:19.241 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T14:27:19.241 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:27:19.259 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T14:27:19.350 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T14:27:19.389 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T14:27:19.389 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T14:27:19.389 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T14:27:19.455 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T14:27:19.465 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T14:27:19.497 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T14:27:19.497 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T14:27:19.530 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T14:27:19.572 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T14:27:19.638 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T14:27:19.818 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T14:27:19.877 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T14:27:19.993 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T14:27:20.204 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T14:27:20.385 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T14:27:20.447 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T14:27:20.492 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T14:27:20.524 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T14:27:20.803 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T14:27:20.803 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T14:27:20.803 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T14:27:20.803 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T14:27:20.803 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T14:27:20.803 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T14:27:20.804 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T14:27:20.804 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T14:27:20.804 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T14:27:20.804 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T14:27:20.804 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T14:27:20.804 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T14:27:20.804 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T14:27:20.804 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T14:27:20.804 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T14:27:20.804 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T14:27:20.804 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T14:27:20.804 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T14:27:20.804 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T14:27:20.805 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T14:27:20.805 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T14:27:20.805 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T14:27:20.805 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T14:27:20.805 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T14:27:20.805 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T14:27:20.805 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T14:27:20.805 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T14:27:20.805 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T14:27:20.818 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T14:27:20.818 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T14:27:20.818 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T14:27:20.818 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T14:27:20.818 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T14:27:20.818 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T14:27:20.818 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T14:27:20.818 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T14:27:20.818 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T14:27:20.818 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T14:27:20.818 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T14:27:20.819 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T14:27:20.819 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T14:27:20.819 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T14:27:20.819 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T14:27:20.819 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T14:27:20.819 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T14:27:20.819 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T14:27:20.819 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T14:27:20.819 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T14:27:20.819 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T14:27:20.819 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T14:27:20.819 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T14:27:20.819 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T14:27:20.819 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T14:27:20.820 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T14:27:20.820 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T14:27:20.820 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T14:27:20.820 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T14:27:20.820 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T14:27:20.820 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T14:27:20.820 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T14:27:20.820 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T14:27:20.820 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T14:27:20.820 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T14:27:20.820 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T14:27:20.820 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T14:27:20.821 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T14:27:20.821 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T14:27:20.821 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T14:27:20.821 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T14:27:20.821 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T14:27:20.821 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T14:27:20.821 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T14:27:20.821 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T14:27:20.821 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T14:27:20.821 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T14:27:20.821 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T14:27:20.821 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T14:27:20.821 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T14:27:20.821 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T14:27:20.822 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T14:27:20.822 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T14:27:20.822 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T14:27:20.822 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T14:27:20.822 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T14:27:20.822 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T14:27:20.822 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T14:27:20.822 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T14:27:20.822 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T14:27:20.822 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T14:27:20.822 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T14:27:20.822 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T14:27:20.822 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T14:27:20.822 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T14:27:20.823 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T14:27:20.823 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T14:27:20.823 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T14:27:20.823 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T14:27:20.823 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T14:27:20.823 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T14:27:20.823 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T14:27:20.823 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T14:27:20.823 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T14:27:20.823 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T14:27:20.823 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T14:27:20.823 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T14:27:20.823 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T14:27:20.823 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T14:27:20.824 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T14:27:20.824 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T14:27:20.824 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T14:27:20.824 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T14:27:20.824 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T14:27:20.824 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T14:27:20.824 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T14:27:20.824 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T14:27:20.824 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T14:27:20.824 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T14:27:20.824 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T14:27:20.824 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T14:27:20.825 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T14:27:20.825 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T14:27:20.825 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T14:27:20.825 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T14:27:20.825 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T14:27:20.825 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T14:27:20.825 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T14:27:20.825 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T14:27:20.825 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T14:27:20.825 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T14:27:20.825 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T14:27:20.826 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T14:27:20.826 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T14:27:20.826 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T14:27:20.826 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T14:27:20.826 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T14:27:20.826 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T14:27:20.826 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T14:27:20.826 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T14:27:20.826 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T14:27:20.826 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T14:27:20.826 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T14:27:20.826 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T14:27:20.826 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T14:27:20.826 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T14:27:20.827 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T14:27:20.827 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T14:27:20.827 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T14:27:20.827 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T14:27:20.827 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T14:27:20.827 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T14:27:20.827 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T14:27:20.827 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T14:27:20.827 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T14:27:20.827 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T14:27:20.827 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T14:27:20.827 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T14:27:21.211 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T14:27:21.252 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T14:27:21.311 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T14:27:21.421 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T14:27:21.454 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T14:27:22.819 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T14:27:22.889 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T14:27:22.921 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T14:27:22.921 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-25T14:27:22.921 INFO:teuthology.orchestra.run.smithi183.stdout:Upgraded: 2024-04-25T14:27:22.921 INFO:teuthology.orchestra.run.smithi183.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:22.922 INFO:teuthology.orchestra.run.smithi183.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:22.922 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T14:27:22.922 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T14:27:22.922 INFO:teuthology.orchestra.run.smithi183.stdout:Installed: 2024-04-25T14:27:22.922 INFO:teuthology.orchestra.run.smithi183.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T14:27:22.922 INFO:teuthology.orchestra.run.smithi183.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T14:27:22.922 INFO:teuthology.orchestra.run.smithi183.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T14:27:22.922 INFO:teuthology.orchestra.run.smithi183.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T14:27:22.922 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:22.922 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:22.922 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:22.922 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:22.922 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:27:22.923 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:22.923 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:22.923 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:22.923 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:27:22.923 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:27:22.923 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:27:22.923 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:27:22.923 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:27:22.923 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:22.923 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:22.923 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:27:22.923 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:22.923 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:22.923 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:22.924 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:27:22.924 INFO:teuthology.orchestra.run.smithi183.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:27:22.924 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:22.924 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:22.924 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:27:22.924 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T14:27:22.924 INFO:teuthology.orchestra.run.smithi183.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T14:27:22.924 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T14:27:22.924 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T14:27:22.924 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T14:27:22.924 INFO:teuthology.orchestra.run.smithi183.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T14:27:22.924 INFO:teuthology.orchestra.run.smithi183.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T14:27:22.924 INFO:teuthology.orchestra.run.smithi183.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T14:27:22.925 INFO:teuthology.orchestra.run.smithi183.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T14:27:22.925 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T14:27:22.925 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T14:27:22.925 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:22.925 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:22.925 INFO:teuthology.orchestra.run.smithi183.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:22.925 INFO:teuthology.orchestra.run.smithi183.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T14:27:22.925 INFO:teuthology.orchestra.run.smithi183.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T14:27:22.925 INFO:teuthology.orchestra.run.smithi183.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T14:27:22.925 INFO:teuthology.orchestra.run.smithi183.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T14:27:22.925 INFO:teuthology.orchestra.run.smithi183.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T14:27:22.925 INFO:teuthology.orchestra.run.smithi183.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:22.925 INFO:teuthology.orchestra.run.smithi183.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:22.925 INFO:teuthology.orchestra.run.smithi183.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T14:27:22.926 INFO:teuthology.orchestra.run.smithi183.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:22.926 INFO:teuthology.orchestra.run.smithi183.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T14:27:22.926 INFO:teuthology.orchestra.run.smithi183.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T14:27:22.926 INFO:teuthology.orchestra.run.smithi183.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T14:27:22.926 INFO:teuthology.orchestra.run.smithi183.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T14:27:22.926 INFO:teuthology.orchestra.run.smithi183.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T14:27:22.926 INFO:teuthology.orchestra.run.smithi183.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T14:27:22.926 INFO:teuthology.orchestra.run.smithi183.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T14:27:22.926 INFO:teuthology.orchestra.run.smithi183.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T14:27:22.926 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T14:27:22.926 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T14:27:22.926 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T14:27:22.926 INFO:teuthology.orchestra.run.smithi183.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T14:27:22.926 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T14:27:22.927 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T14:27:22.927 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T14:27:22.927 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T14:27:22.927 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T14:27:22.927 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T14:27:22.927 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T14:27:22.927 INFO:teuthology.orchestra.run.smithi183.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T14:27:22.927 INFO:teuthology.orchestra.run.smithi183.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T14:27:22.927 INFO:teuthology.orchestra.run.smithi183.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T14:27:22.927 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T14:27:22.927 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:22.927 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:22.927 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:22.927 INFO:teuthology.orchestra.run.smithi183.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T14:27:22.927 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T14:27:22.928 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T14:27:22.928 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T14:27:22.928 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T14:27:22.928 INFO:teuthology.orchestra.run.smithi183.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T14:27:22.928 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T14:27:22.928 INFO:teuthology.orchestra.run.smithi183.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T14:27:22.928 INFO:teuthology.orchestra.run.smithi183.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T14:27:22.928 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T14:27:22.928 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T14:27:22.928 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T14:27:22.928 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T14:27:22.928 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T14:27:22.928 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T14:27:22.928 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T14:27:22.929 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T14:27:22.929 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T14:27:22.929 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T14:27:22.929 INFO:teuthology.orchestra.run.smithi183.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T14:27:22.929 INFO:teuthology.orchestra.run.smithi183.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T14:27:22.929 INFO:teuthology.orchestra.run.smithi183.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T14:27:22.929 INFO:teuthology.orchestra.run.smithi183.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T14:27:22.929 INFO:teuthology.orchestra.run.smithi183.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T14:27:22.929 INFO:teuthology.orchestra.run.smithi183.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T14:27:22.929 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T14:27:22.929 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T14:27:22.929 INFO:teuthology.orchestra.run.smithi183.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T14:27:22.929 INFO:teuthology.orchestra.run.smithi183.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T14:27:22.929 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T14:27:22.930 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T14:27:22.930 INFO:teuthology.orchestra.run.smithi183.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T14:27:22.930 INFO:teuthology.orchestra.run.smithi183.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T14:27:22.930 INFO:teuthology.orchestra.run.smithi183.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T14:27:22.930 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T14:27:22.930 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T14:27:22.930 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T14:27:22.930 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T14:27:22.930 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T14:27:22.930 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T14:27:22.930 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:22.930 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:22.930 INFO:teuthology.orchestra.run.smithi183.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T14:27:22.930 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T14:27:22.931 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T14:27:22.931 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:22.931 INFO:teuthology.orchestra.run.smithi183.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T14:27:22.931 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T14:27:22.931 INFO:teuthology.orchestra.run.smithi183.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T14:27:22.931 INFO:teuthology.orchestra.run.smithi183.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T14:27:22.931 INFO:teuthology.orchestra.run.smithi183.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T14:27:22.931 INFO:teuthology.orchestra.run.smithi183.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T14:27:22.931 INFO:teuthology.orchestra.run.smithi183.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T14:27:22.931 INFO:teuthology.orchestra.run.smithi183.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T14:27:22.931 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T14:27:22.931 INFO:teuthology.orchestra.run.smithi183.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T14:27:22.931 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T14:27:22.931 INFO:teuthology.orchestra.run.smithi183.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T14:27:22.931 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T14:27:22.932 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T14:27:22.932 INFO:teuthology.orchestra.run.smithi183.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T14:27:22.932 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:22.932 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:22.932 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:22.932 INFO:teuthology.orchestra.run.smithi183.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T14:27:22.932 INFO:teuthology.orchestra.run.smithi183.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T14:27:22.932 INFO:teuthology.orchestra.run.smithi183.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T14:27:22.932 INFO:teuthology.orchestra.run.smithi183.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T14:27:22.932 INFO:teuthology.orchestra.run.smithi183.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T14:27:22.932 INFO:teuthology.orchestra.run.smithi183.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T14:27:22.933 INFO:teuthology.orchestra.run.smithi183.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T14:27:22.933 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-25T14:27:22.933 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-04-25T14:27:23.299 DEBUG:teuthology.parallel:result is None 2024-04-25T14:27:24.059 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T14:27:24.089 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T14:27:24.242 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T14:27:24.430 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T14:27:24.466 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T14:27:24.539 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T14:27:24.540 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:27:24.540 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T14:27:24.540 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T14:27:24.540 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T14:27:24.540 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:27:24.600 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T14:27:24.640 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T14:27:24.754 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T14:27:24.821 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T14:27:25.735 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T14:27:25.735 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T14:27:25.735 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T14:27:25.735 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T14:27:25.735 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T14:27:25.735 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T14:27:25.736 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T14:27:25.736 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T14:27:25.736 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T14:27:25.736 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T14:27:25.736 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T14:27:25.736 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T14:27:25.736 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T14:27:25.736 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T14:27:25.736 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T14:27:25.736 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T14:27:25.736 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T14:27:25.736 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T14:27:25.736 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T14:27:25.737 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T14:27:25.737 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T14:27:25.737 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T14:27:25.737 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T14:27:25.737 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T14:27:25.737 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T14:27:25.737 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T14:27:25.737 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T14:27:25.737 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T14:27:25.737 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T14:27:25.737 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T14:27:25.749 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T14:27:25.749 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T14:27:25.749 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T14:27:25.749 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T14:27:25.749 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T14:27:25.749 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T14:27:25.749 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T14:27:25.749 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T14:27:25.749 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T14:27:25.749 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T14:27:25.749 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T14:27:25.750 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T14:27:25.750 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T14:27:25.750 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T14:27:25.750 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T14:27:25.750 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T14:27:25.750 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T14:27:25.750 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T14:27:25.750 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T14:27:25.750 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T14:27:25.750 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T14:27:25.750 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T14:27:25.750 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T14:27:25.750 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T14:27:25.750 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T14:27:25.751 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T14:27:25.751 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T14:27:25.751 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T14:27:25.751 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T14:27:25.751 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T14:27:25.751 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T14:27:25.751 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T14:27:25.751 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T14:27:25.751 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T14:27:25.751 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T14:27:25.751 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T14:27:25.751 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T14:27:25.751 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T14:27:25.751 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T14:27:25.751 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T14:27:25.752 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T14:27:25.752 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T14:27:25.752 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T14:27:25.752 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T14:27:25.752 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T14:27:25.752 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T14:27:25.752 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T14:27:25.752 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T14:27:25.752 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T14:27:25.752 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T14:27:25.752 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T14:27:25.752 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T14:27:25.752 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T14:27:25.752 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T14:27:25.753 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T14:27:25.753 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T14:27:25.753 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T14:27:25.753 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T14:27:25.753 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T14:27:25.753 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T14:27:25.753 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T14:27:25.753 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T14:27:25.753 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T14:27:25.753 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T14:27:25.753 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T14:27:25.753 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T14:27:25.753 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T14:27:25.753 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T14:27:25.753 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T14:27:25.754 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T14:27:25.754 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T14:27:25.754 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T14:27:25.754 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T14:27:25.754 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T14:27:25.754 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T14:27:25.754 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T14:27:25.754 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T14:27:25.754 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T14:27:25.754 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T14:27:25.754 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T14:27:25.754 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T14:27:25.754 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T14:27:25.754 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T14:27:25.755 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T14:27:25.755 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T14:27:25.755 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T14:27:25.755 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T14:27:25.755 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T14:27:25.755 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T14:27:25.755 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T14:27:25.755 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T14:27:25.755 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T14:27:25.755 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T14:27:25.755 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T14:27:25.755 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T14:27:25.755 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T14:27:25.755 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T14:27:25.755 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T14:27:25.756 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T14:27:25.756 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T14:27:25.756 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T14:27:25.756 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T14:27:25.756 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T14:27:25.756 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T14:27:25.756 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T14:27:25.756 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T14:27:25.756 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T14:27:25.756 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T14:27:25.756 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T14:27:25.756 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T14:27:25.756 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T14:27:25.756 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T14:27:25.756 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T14:27:25.757 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T14:27:25.757 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T14:27:25.757 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T14:27:25.757 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T14:27:25.757 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T14:27:25.757 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T14:27:25.757 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T14:27:25.757 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T14:27:25.757 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T14:27:25.757 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T14:27:25.757 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T14:27:25.757 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T14:27:25.757 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T14:27:25.881 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T14:27:25.911 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T14:27:25.982 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T14:27:25.983 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:27:25.983 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T14:27:25.983 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T14:27:25.983 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T14:27:25.983 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:27:26.028 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T14:27:26.104 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T14:27:26.105 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:27:26.105 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T14:27:26.105 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:27:26.179 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T14:27:26.257 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T14:27:26.257 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:27:26.257 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T14:27:26.257 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T14:27:26.258 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T14:27:26.258 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:27:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T14:27:26.733 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T14:27:26.733 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:27:26.733 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T14:27:26.733 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T14:27:26.733 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T14:27:26.733 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:27:27.881 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T14:27:27.882 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T14:27:27.882 INFO:teuthology.orchestra.run.smithi150.stdout:Upgraded: 2024-04-25T14:27:27.882 INFO:teuthology.orchestra.run.smithi150.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:27.882 INFO:teuthology.orchestra.run.smithi150.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:27.882 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T14:27:27.882 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T14:27:27.882 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-04-25T14:27:27.882 INFO:teuthology.orchestra.run.smithi150.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T14:27:27.882 INFO:teuthology.orchestra.run.smithi150.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T14:27:27.882 INFO:teuthology.orchestra.run.smithi150.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T14:27:27.882 INFO:teuthology.orchestra.run.smithi150.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T14:27:27.883 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:27.883 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:27.883 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:27.883 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:27.883 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:27:27.883 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:27.883 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:27.883 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:27.883 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:27:27.884 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:27:27.884 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:27:27.884 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:27:27.884 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:27:27.884 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:27.884 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:27.884 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:27:27.884 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:27.884 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:27.884 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:27.884 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:27:27.884 INFO:teuthology.orchestra.run.smithi150.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:27:27.884 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:27.884 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:27.885 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:27:27.885 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T14:27:27.885 INFO:teuthology.orchestra.run.smithi150.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T14:27:27.885 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T14:27:27.885 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T14:27:27.885 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T14:27:27.885 INFO:teuthology.orchestra.run.smithi150.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T14:27:27.885 INFO:teuthology.orchestra.run.smithi150.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T14:27:27.885 INFO:teuthology.orchestra.run.smithi150.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T14:27:27.885 INFO:teuthology.orchestra.run.smithi150.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T14:27:27.885 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T14:27:27.885 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T14:27:27.885 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:27.885 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:27.885 INFO:teuthology.orchestra.run.smithi150.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:27.886 INFO:teuthology.orchestra.run.smithi150.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T14:27:27.886 INFO:teuthology.orchestra.run.smithi150.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T14:27:27.886 INFO:teuthology.orchestra.run.smithi150.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T14:27:27.886 INFO:teuthology.orchestra.run.smithi150.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T14:27:27.886 INFO:teuthology.orchestra.run.smithi150.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T14:27:27.886 INFO:teuthology.orchestra.run.smithi150.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:27.886 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:27.886 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T14:27:27.886 INFO:teuthology.orchestra.run.smithi150.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:27.886 INFO:teuthology.orchestra.run.smithi150.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T14:27:27.886 INFO:teuthology.orchestra.run.smithi150.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T14:27:27.886 INFO:teuthology.orchestra.run.smithi150.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T14:27:27.886 INFO:teuthology.orchestra.run.smithi150.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T14:27:27.887 INFO:teuthology.orchestra.run.smithi150.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T14:27:27.887 INFO:teuthology.orchestra.run.smithi150.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T14:27:27.887 INFO:teuthology.orchestra.run.smithi150.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T14:27:27.887 INFO:teuthology.orchestra.run.smithi150.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T14:27:27.887 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T14:27:27.887 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T14:27:27.887 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T14:27:27.887 INFO:teuthology.orchestra.run.smithi150.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T14:27:27.887 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T14:27:27.887 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T14:27:27.887 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T14:27:27.887 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T14:27:27.887 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T14:27:27.888 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T14:27:27.888 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T14:27:27.888 INFO:teuthology.orchestra.run.smithi150.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T14:27:27.888 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T14:27:27.888 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T14:27:27.888 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T14:27:27.889 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:27.889 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:27.889 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:27.889 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T14:27:27.889 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T14:27:27.889 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T14:27:27.889 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T14:27:27.889 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T14:27:27.889 INFO:teuthology.orchestra.run.smithi150.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T14:27:27.889 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T14:27:27.890 INFO:teuthology.orchestra.run.smithi150.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T14:27:27.890 INFO:teuthology.orchestra.run.smithi150.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T14:27:27.890 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T14:27:27.890 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T14:27:27.890 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T14:27:27.890 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T14:27:27.890 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T14:27:27.890 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T14:27:27.890 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T14:27:27.890 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T14:27:27.890 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T14:27:27.890 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T14:27:27.890 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T14:27:27.891 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T14:27:27.891 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T14:27:27.891 INFO:teuthology.orchestra.run.smithi150.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T14:27:27.891 INFO:teuthology.orchestra.run.smithi150.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T14:27:27.891 INFO:teuthology.orchestra.run.smithi150.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T14:27:27.891 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T14:27:27.891 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T14:27:27.891 INFO:teuthology.orchestra.run.smithi150.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T14:27:27.891 INFO:teuthology.orchestra.run.smithi150.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T14:27:27.891 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T14:27:27.891 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T14:27:27.891 INFO:teuthology.orchestra.run.smithi150.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T14:27:27.891 INFO:teuthology.orchestra.run.smithi150.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T14:27:27.891 INFO:teuthology.orchestra.run.smithi150.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T14:27:27.891 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T14:27:27.892 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T14:27:27.892 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T14:27:27.892 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T14:27:27.892 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T14:27:27.892 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T14:27:27.892 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:27.892 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:27.892 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T14:27:27.892 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T14:27:27.892 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T14:27:27.892 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:27.892 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T14:27:27.892 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T14:27:27.892 INFO:teuthology.orchestra.run.smithi150.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T14:27:27.893 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T14:27:27.893 INFO:teuthology.orchestra.run.smithi150.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T14:27:27.893 INFO:teuthology.orchestra.run.smithi150.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T14:27:27.893 INFO:teuthology.orchestra.run.smithi150.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T14:27:27.893 INFO:teuthology.orchestra.run.smithi150.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T14:27:27.893 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T14:27:27.893 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T14:27:27.893 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T14:27:27.893 INFO:teuthology.orchestra.run.smithi150.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T14:27:27.893 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T14:27:27.893 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T14:27:27.894 INFO:teuthology.orchestra.run.smithi150.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T14:27:27.894 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:27.894 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:27.894 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:27.894 INFO:teuthology.orchestra.run.smithi150.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T14:27:27.894 INFO:teuthology.orchestra.run.smithi150.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T14:27:27.894 INFO:teuthology.orchestra.run.smithi150.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T14:27:27.894 INFO:teuthology.orchestra.run.smithi150.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T14:27:27.894 INFO:teuthology.orchestra.run.smithi150.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T14:27:27.894 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T14:27:27.895 INFO:teuthology.orchestra.run.smithi150.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T14:27:27.895 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T14:27:27.895 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-04-25T14:27:28.247 DEBUG:teuthology.parallel:result is None 2024-04-25T14:27:32.053 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T14:27:32.111 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T14:27:32.191 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T14:27:32.239 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T14:27:32.319 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T14:27:32.701 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T14:27:32.758 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T14:27:32.785 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T14:27:33.715 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T14:27:33.766 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T14:27:33.828 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T14:27:35.282 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T14:27:35.398 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T14:27:35.479 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T14:27:35.656 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T14:27:35.705 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T14:27:35.705 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T14:27:35.831 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T14:27:35.878 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T14:27:35.878 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T14:27:36.042 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T14:27:36.180 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T14:27:36.519 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T14:27:36.519 INFO:teuthology.orchestra.run.smithi053.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-25T14:27:36.519 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:27:42.499 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T14:27:42.520 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T14:27:42.521 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T14:27:42.521 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T14:27:42.521 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T14:27:42.521 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T14:27:42.521 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T14:27:42.521 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T14:27:42.521 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T14:27:42.521 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T14:27:42.521 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T14:27:42.521 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T14:27:42.521 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T14:27:42.521 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T14:27:42.522 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T14:27:42.522 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T14:27:42.522 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T14:27:42.522 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T14:27:42.522 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T14:27:42.522 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T14:27:42.522 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T14:27:42.522 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T14:27:42.522 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T14:27:42.522 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T14:27:42.522 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T14:27:42.522 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T14:27:42.522 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T14:27:42.523 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T14:27:42.523 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T14:27:42.523 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T14:27:42.523 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T14:27:42.523 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T14:27:42.523 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T14:27:42.523 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T14:27:42.523 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T14:27:42.523 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T14:27:42.523 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T14:27:42.523 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T14:27:42.524 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T14:27:42.524 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T14:27:42.524 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T14:27:42.524 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T14:27:42.524 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T14:27:42.524 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T14:27:42.525 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T14:27:42.525 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T14:27:42.525 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T14:27:42.525 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T14:27:42.525 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T14:27:42.525 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T14:27:42.525 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T14:27:42.525 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T14:27:42.525 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T14:27:42.525 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T14:27:42.525 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T14:27:42.525 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T14:27:42.525 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T14:27:42.525 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T14:27:42.526 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T14:27:42.526 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T14:27:42.526 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T14:27:42.526 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T14:27:42.526 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T14:27:42.526 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T14:27:42.526 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T14:27:42.526 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T14:27:42.526 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T14:27:42.526 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T14:27:42.526 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T14:27:42.526 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T14:27:42.526 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T14:27:42.526 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T14:27:42.527 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T14:27:42.527 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T14:27:42.527 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T14:27:42.527 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T14:27:42.527 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T14:27:42.527 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T14:27:42.527 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T14:27:42.527 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T14:27:42.527 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T14:27:42.527 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T14:27:42.527 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T14:27:42.527 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T14:27:42.527 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T14:27:42.527 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T14:27:42.528 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T14:27:42.528 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T14:27:42.528 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T14:27:42.528 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T14:27:42.528 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T14:27:42.528 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T14:27:42.528 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T14:27:42.528 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T14:27:42.528 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T14:27:42.528 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T14:27:42.528 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T14:27:42.528 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T14:27:42.528 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T14:27:42.528 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T14:27:42.529 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T14:27:42.529 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T14:27:42.529 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T14:27:42.529 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T14:27:42.529 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T14:27:42.529 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T14:27:42.529 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T14:27:42.529 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T14:27:42.529 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T14:27:42.529 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T14:27:42.529 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T14:27:42.529 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T14:27:42.529 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T14:27:42.529 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T14:27:42.530 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T14:27:42.530 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T14:27:42.530 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T14:27:42.530 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T14:27:42.530 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T14:27:42.530 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T14:27:42.530 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T14:27:42.530 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T14:27:42.530 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T14:27:42.530 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T14:27:42.530 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T14:27:42.530 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T14:27:42.530 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T14:27:42.530 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T14:27:42.531 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T14:27:42.531 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T14:27:42.531 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T14:27:42.531 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T14:27:42.531 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T14:27:42.531 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T14:27:42.531 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T14:27:42.531 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T14:27:42.531 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T14:27:42.531 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T14:27:42.531 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T14:27:42.531 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T14:27:42.531 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T14:27:42.531 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T14:27:42.532 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T14:27:42.532 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T14:27:42.532 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T14:27:42.532 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T14:27:42.532 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T14:27:42.532 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T14:27:42.532 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T14:27:42.532 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T14:27:42.532 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T14:27:42.532 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T14:27:42.532 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T14:27:42.532 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T14:27:42.532 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T14:27:45.068 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T14:27:45.069 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:27:45.069 INFO:teuthology.orchestra.run.smithi053.stdout:Upgraded: 2024-04-25T14:27:45.069 INFO:teuthology.orchestra.run.smithi053.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:45.069 INFO:teuthology.orchestra.run.smithi053.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:45.069 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T14:27:45.069 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T14:27:45.069 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-04-25T14:27:45.069 INFO:teuthology.orchestra.run.smithi053.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T14:27:45.069 INFO:teuthology.orchestra.run.smithi053.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T14:27:45.069 INFO:teuthology.orchestra.run.smithi053.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T14:27:45.069 INFO:teuthology.orchestra.run.smithi053.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T14:27:45.069 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:45.070 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:45.070 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:45.070 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:45.070 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:27:45.070 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:45.070 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:45.070 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:45.070 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:27:45.070 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:27:45.070 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:27:45.070 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:27:45.070 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:27:45.070 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:45.070 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:45.071 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:27:45.071 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:45.071 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:45.071 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:45.071 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:27:45.071 INFO:teuthology.orchestra.run.smithi053.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:27:45.071 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:45.071 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:45.071 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:27:45.071 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T14:27:45.071 INFO:teuthology.orchestra.run.smithi053.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T14:27:45.071 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T14:27:45.072 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T14:27:45.072 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T14:27:45.072 INFO:teuthology.orchestra.run.smithi053.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T14:27:45.072 INFO:teuthology.orchestra.run.smithi053.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T14:27:45.072 INFO:teuthology.orchestra.run.smithi053.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T14:27:45.072 INFO:teuthology.orchestra.run.smithi053.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T14:27:45.072 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T14:27:45.072 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T14:27:45.072 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:45.072 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:45.072 INFO:teuthology.orchestra.run.smithi053.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:45.072 INFO:teuthology.orchestra.run.smithi053.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T14:27:45.072 INFO:teuthology.orchestra.run.smithi053.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T14:27:45.072 INFO:teuthology.orchestra.run.smithi053.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T14:27:45.073 INFO:teuthology.orchestra.run.smithi053.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T14:27:45.073 INFO:teuthology.orchestra.run.smithi053.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T14:27:45.073 INFO:teuthology.orchestra.run.smithi053.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:45.073 INFO:teuthology.orchestra.run.smithi053.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:45.073 INFO:teuthology.orchestra.run.smithi053.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T14:27:45.073 INFO:teuthology.orchestra.run.smithi053.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:45.073 INFO:teuthology.orchestra.run.smithi053.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T14:27:45.073 INFO:teuthology.orchestra.run.smithi053.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T14:27:45.073 INFO:teuthology.orchestra.run.smithi053.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T14:27:45.073 INFO:teuthology.orchestra.run.smithi053.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T14:27:45.073 INFO:teuthology.orchestra.run.smithi053.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T14:27:45.073 INFO:teuthology.orchestra.run.smithi053.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T14:27:45.073 INFO:teuthology.orchestra.run.smithi053.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T14:27:45.073 INFO:teuthology.orchestra.run.smithi053.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T14:27:45.074 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T14:27:45.074 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T14:27:45.074 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T14:27:45.074 INFO:teuthology.orchestra.run.smithi053.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T14:27:45.074 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T14:27:45.074 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T14:27:45.074 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T14:27:45.074 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T14:27:45.074 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T14:27:45.074 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T14:27:45.074 INFO:teuthology.orchestra.run.smithi053.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T14:27:45.075 INFO:teuthology.orchestra.run.smithi053.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T14:27:45.075 INFO:teuthology.orchestra.run.smithi053.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T14:27:45.075 INFO:teuthology.orchestra.run.smithi053.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T14:27:45.075 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T14:27:45.075 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:45.075 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:45.075 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:45.075 INFO:teuthology.orchestra.run.smithi053.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T14:27:45.075 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T14:27:45.075 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T14:27:45.075 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T14:27:45.075 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T14:27:45.075 INFO:teuthology.orchestra.run.smithi053.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T14:27:45.076 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T14:27:45.076 INFO:teuthology.orchestra.run.smithi053.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T14:27:45.076 INFO:teuthology.orchestra.run.smithi053.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T14:27:45.076 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T14:27:45.076 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T14:27:45.076 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T14:27:45.076 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T14:27:45.076 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T14:27:45.076 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T14:27:45.076 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T14:27:45.076 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T14:27:45.076 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T14:27:45.076 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T14:27:45.076 INFO:teuthology.orchestra.run.smithi053.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T14:27:45.077 INFO:teuthology.orchestra.run.smithi053.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T14:27:45.077 INFO:teuthology.orchestra.run.smithi053.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T14:27:45.077 INFO:teuthology.orchestra.run.smithi053.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T14:27:45.077 INFO:teuthology.orchestra.run.smithi053.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T14:27:45.077 INFO:teuthology.orchestra.run.smithi053.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T14:27:45.077 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T14:27:45.077 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T14:27:45.077 INFO:teuthology.orchestra.run.smithi053.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T14:27:45.077 INFO:teuthology.orchestra.run.smithi053.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T14:27:45.077 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T14:27:45.077 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T14:27:45.077 INFO:teuthology.orchestra.run.smithi053.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T14:27:45.077 INFO:teuthology.orchestra.run.smithi053.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T14:27:45.077 INFO:teuthology.orchestra.run.smithi053.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T14:27:45.077 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T14:27:45.078 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T14:27:45.078 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T14:27:45.078 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T14:27:45.078 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T14:27:45.078 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T14:27:45.078 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:45.078 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:45.078 INFO:teuthology.orchestra.run.smithi053.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T14:27:45.078 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T14:27:45.078 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T14:27:45.078 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:45.078 INFO:teuthology.orchestra.run.smithi053.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T14:27:45.078 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T14:27:45.078 INFO:teuthology.orchestra.run.smithi053.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T14:27:45.079 INFO:teuthology.orchestra.run.smithi053.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T14:27:45.079 INFO:teuthology.orchestra.run.smithi053.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T14:27:45.079 INFO:teuthology.orchestra.run.smithi053.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T14:27:45.079 INFO:teuthology.orchestra.run.smithi053.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T14:27:45.079 INFO:teuthology.orchestra.run.smithi053.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T14:27:45.079 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T14:27:45.079 INFO:teuthology.orchestra.run.smithi053.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T14:27:45.079 INFO:teuthology.orchestra.run.smithi053.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T14:27:45.079 INFO:teuthology.orchestra.run.smithi053.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T14:27:45.079 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T14:27:45.079 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T14:27:45.079 INFO:teuthology.orchestra.run.smithi053.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T14:27:45.079 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:45.079 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:45.079 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:27:45.080 INFO:teuthology.orchestra.run.smithi053.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T14:27:45.080 INFO:teuthology.orchestra.run.smithi053.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T14:27:45.080 INFO:teuthology.orchestra.run.smithi053.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T14:27:45.080 INFO:teuthology.orchestra.run.smithi053.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T14:27:45.080 INFO:teuthology.orchestra.run.smithi053.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T14:27:45.080 INFO:teuthology.orchestra.run.smithi053.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T14:27:45.080 INFO:teuthology.orchestra.run.smithi053.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T14:27:45.080 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:27:45.081 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-04-25T14:27:45.748 DEBUG:teuthology.parallel:result is None 2024-04-25T14:27:45.749 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T14:27:45.894 DEBUG:teuthology.orchestra.run.smithi053:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T14:27:45.955 INFO:teuthology.orchestra.run.smithi053.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T14:27:45.957 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T14:27:45.957 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T14:27:45.960 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T14:27:46.203 DEBUG:teuthology.orchestra.run.smithi150:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T14:27:46.228 INFO:teuthology.orchestra.run.smithi150.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T14:27:46.228 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T14:27:46.229 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T14:27:46.231 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T14:27:46.469 DEBUG:teuthology.orchestra.run.smithi183:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T14:27:46.495 INFO:teuthology.orchestra.run.smithi183.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T14:27:46.495 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T14:27:46.495 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T14:27:46.497 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T14:27:46.497 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-25T14:27:46.497 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T14:27:46.588 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T14:27:46.588 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T14:27:46.619 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-25T14:27:46.620 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T14:27:46.651 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T14:27:46.651 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-25T14:27:46.651 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T14:27:46.731 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T14:27:46.838 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T14:27:46.838 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T14:27:46.867 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T14:27:46.935 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-25T14:27:46.935 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T14:27:46.964 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T14:27:47.031 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T14:27:47.031 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-25T14:27:47.031 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T14:27:47.102 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T14:27:47.208 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T14:27:47.208 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T14:27:47.232 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T14:27:47.301 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-25T14:27:47.301 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T14:27:47.326 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T14:27:47.395 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T14:27:47.395 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-25T14:27:47.395 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T14:27:47.461 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T14:27:47.567 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T14:27:47.567 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T14:27:47.592 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T14:27:47.656 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-25T14:27:47.656 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T14:27:47.681 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T14:27:47.753 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-25T14:27:47.900 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'subvols': {'create': 2}} 2024-04-25T14:27:47.900 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T14:27:47.900 INFO:tasks.cephadm:Cluster fsid is fd30e6ce-030f-11ef-bc93-c7b262605968 2024-04-25T14:27:47.901 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-25T14:27:47.901 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.53', 'mon.b': '172.21.15.150', 'mon.c': '172.21.15.183'} 2024-04-25T14:27:47.901 INFO:tasks.cephadm:First mon is mon.a on smithi053 2024-04-25T14:27:47.901 INFO:tasks.cephadm:First mgr is x 2024-04-25T14:27:47.901 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-25T14:27:47.901 DEBUG:teuthology.orchestra.run.smithi053:> sudo hostname $(hostname -s) 2024-04-25T14:27:47.969 DEBUG:teuthology.orchestra.run.smithi150:> sudo hostname $(hostname -s) 2024-04-25T14:27:47.994 DEBUG:teuthology.orchestra.run.smithi183:> sudo hostname $(hostname -s) 2024-04-25T14:27:48.018 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-25T14:27:48.018 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T14:27:48.166 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78795/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3156-gd69f13d8', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3156.gd69f13d8'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-22 12:00:06.590653', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiescer', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-25T14:27:48.303 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-lusov-quiescer, sha1 d69f13d827952bb07bb9b7973bca14f97d764d6f from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T14:27:48.304 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm 2024-04-25T14:27:48.305 DEBUG:teuthology.orchestra.run.smithi053:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T14:27:49.223 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 14:27 /home/ubuntu/cephtest/cephadm 2024-04-25T14:27:49.223 DEBUG:teuthology.orchestra.run.smithi150:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T14:27:49.764 INFO:teuthology.orchestra.run.smithi150.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 14:27 /home/ubuntu/cephtest/cephadm 2024-04-25T14:27:49.764 DEBUG:teuthology.orchestra.run.smithi183:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T14:27:50.651 INFO:teuthology.orchestra.run.smithi183.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 14:27 /home/ubuntu/cephtest/cephadm 2024-04-25T14:27:50.652 DEBUG:teuthology.orchestra.run.smithi053:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T14:27:50.701 DEBUG:teuthology.orchestra.run.smithi150:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T14:27:50.720 DEBUG:teuthology.orchestra.run.smithi183:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T14:27:50.758 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f on all hosts... 2024-04-25T14:27:50.759 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T14:27:50.765 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T14:27:50.770 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T14:27:52.314 INFO:teuthology.orchestra.run.smithi183.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T14:27:52.496 INFO:teuthology.orchestra.run.smithi150.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T14:27:52.964 INFO:teuthology.orchestra.run.smithi053.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T14:28:27.675 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-04-25T14:28:27.675 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T14:28:27.675 INFO:teuthology.orchestra.run.smithi183.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T14:28:27.675 INFO:teuthology.orchestra.run.smithi183.stdout: "repo_digests": [ 2024-04-25T14:28:27.675 INFO:teuthology.orchestra.run.smithi183.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T14:28:27.675 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-04-25T14:28:27.675 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-04-25T14:28:29.377 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-04-25T14:28:29.377 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T14:28:29.377 INFO:teuthology.orchestra.run.smithi150.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T14:28:29.377 INFO:teuthology.orchestra.run.smithi150.stdout: "repo_digests": [ 2024-04-25T14:28:29.377 INFO:teuthology.orchestra.run.smithi150.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T14:28:29.377 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-04-25T14:28:29.377 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-04-25T14:28:41.323 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-04-25T14:28:41.323 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T14:28:41.323 INFO:teuthology.orchestra.run.smithi053.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T14:28:41.323 INFO:teuthology.orchestra.run.smithi053.stdout: "repo_digests": [ 2024-04-25T14:28:41.323 INFO:teuthology.orchestra.run.smithi053.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T14:28:41.324 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-04-25T14:28:41.324 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-04-25T14:28:41.368 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /etc/ceph 2024-04-25T14:28:41.454 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /etc/ceph 2024-04-25T14:28:41.485 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /etc/ceph 2024-04-25T14:28:41.517 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 777 /etc/ceph 2024-04-25T14:28:41.612 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 777 /etc/ceph 2024-04-25T14:28:41.641 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 777 /etc/ceph 2024-04-25T14:28:41.669 INFO:tasks.cephadm:Writing seed config... 2024-04-25T14:28:41.670 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-25T14:28:41.670 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-25T14:28:41.670 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-25T14:28:41.670 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-25T14:28:41.670 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-25T14:28:41.671 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-25T14:28:41.671 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-25T14:28:41.671 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-25T14:28:41.671 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-25T14:28:41.671 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-25T14:28:41.671 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-25T14:28:41.671 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-25T14:28:41.671 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-25T14:28:41.671 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-25T14:28:41.671 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-25T14:28:41.671 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-25T14:28:41.671 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-25T14:28:41.671 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-04-25T14:28:41.671 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-25T14:28:41.671 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-25T14:28:41.671 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-25T14:28:41.671 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-25T14:28:41.672 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-25T14:28:41.672 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-25T14:28:41.672 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-25T14:28:41.672 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-25T14:28:41.672 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-25T14:28:41.672 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-25T14:28:41.672 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-25T14:28:41.672 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-25T14:28:41.672 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-25T14:28:41.672 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-25T14:28:41.672 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-25T14:28:41.672 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-25T14:28:41.672 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-25T14:28:41.672 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-25T14:28:41.672 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-25T14:28:41.672 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-25T14:28:41.673 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-25T14:28:41.673 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-25T14:28:41.673 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-25T14:28:41.673 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-25T14:28:41.673 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-25T14:28:41.673 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-25T14:28:41.673 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-25T14:28:41.673 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-25T14:28:41.673 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-25T14:28:41.673 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-25T14:28:41.673 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-25T14:28:41.674 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-25T14:28:41.674 DEBUG:teuthology.orchestra.run.smithi053:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-25T14:28:41.734 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = fd30e6ce-030f-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-25T14:28:41.734 DEBUG:teuthology.orchestra.run.smithi053:mon.a> sudo journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mon.a.service 2024-04-25T14:28:41.782 DEBUG:teuthology.orchestra.run.smithi053:mgr.x> sudo journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mgr.x.service 2024-04-25T14:28:41.829 INFO:tasks.cephadm:Bootstrapping... 2024-04-25T14:28:41.829 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f -v bootstrap --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.53 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-25T14:28:42.102 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-04-25T14:28:42.103 INFO:teuthology.orchestra.run.smithi053.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f', '-v', 'bootstrap', '--fsid', 'fd30e6ce-030f-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.53', '--skip-admin-label'] 2024-04-25T14:28:42.103 INFO:teuthology.orchestra.run.smithi053.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-25T14:28:42.103 INFO:teuthology.orchestra.run.smithi053.stdout:Verifying podman|docker is present... 2024-04-25T14:28:42.176 INFO:teuthology.orchestra.run.smithi053.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T14:28:42.177 INFO:teuthology.orchestra.run.smithi053.stdout:Verifying lvm2 is present... 2024-04-25T14:28:42.177 INFO:teuthology.orchestra.run.smithi053.stdout:Verifying time synchronization is in place... 2024-04-25T14:28:42.195 INFO:teuthology.orchestra.run.smithi053.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T14:28:42.195 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T14:28:42.211 INFO:teuthology.orchestra.run.smithi053.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T14:28:42.211 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stdout inactive 2024-04-25T14:28:42.228 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stdout enabled 2024-04-25T14:28:42.243 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stdout active 2024-04-25T14:28:42.243 INFO:teuthology.orchestra.run.smithi053.stdout:Unit chronyd.service is enabled and running 2024-04-25T14:28:42.243 INFO:teuthology.orchestra.run.smithi053.stdout:Repeating the final host check... 2024-04-25T14:28:42.302 INFO:teuthology.orchestra.run.smithi053.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T14:28:42.302 INFO:teuthology.orchestra.run.smithi053.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-25T14:28:42.303 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl is present 2024-04-25T14:28:42.303 INFO:teuthology.orchestra.run.smithi053.stdout:lvcreate is present 2024-04-25T14:28:42.325 INFO:teuthology.orchestra.run.smithi053.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T14:28:42.325 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T14:28:42.347 INFO:teuthology.orchestra.run.smithi053.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T14:28:42.347 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stdout inactive 2024-04-25T14:28:42.374 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stdout enabled 2024-04-25T14:28:42.397 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stdout active 2024-04-25T14:28:42.397 INFO:teuthology.orchestra.run.smithi053.stdout:Unit chronyd.service is enabled and running 2024-04-25T14:28:42.398 INFO:teuthology.orchestra.run.smithi053.stdout:Host looks OK 2024-04-25T14:28:42.398 INFO:teuthology.orchestra.run.smithi053.stdout:Cluster fsid: fd30e6ce-030f-11ef-bc93-c7b262605968 2024-04-25T14:28:42.398 INFO:teuthology.orchestra.run.smithi053.stdout:Acquiring lock 140021778279584 on /run/cephadm/fd30e6ce-030f-11ef-bc93-c7b262605968.lock 2024-04-25T14:28:42.399 INFO:teuthology.orchestra.run.smithi053.stdout:Lock 140021778279584 acquired on /run/cephadm/fd30e6ce-030f-11ef-bc93-c7b262605968.lock 2024-04-25T14:28:42.400 INFO:teuthology.orchestra.run.smithi053.stdout:Verifying IP 172.21.15.53 port 3300 ... 2024-04-25T14:28:42.401 INFO:teuthology.orchestra.run.smithi053.stdout:Verifying IP 172.21.15.53 port 6789 ... 2024-04-25T14:28:42.401 INFO:teuthology.orchestra.run.smithi053.stdout:Base mon IP(s) is [172.21.15.53:3300, 172.21.15.53:6789], mon addrv is [v2:172.21.15.53:3300,v1:172.21.15.53:6789] 2024-04-25T14:28:42.418 INFO:teuthology.orchestra.run.smithi053.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.53 metric 100 2024-04-25T14:28:42.418 INFO:teuthology.orchestra.run.smithi053.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.53 metric 100 2024-04-25T14:28:42.430 INFO:teuthology.orchestra.run.smithi053.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-25T14:28:42.431 INFO:teuthology.orchestra.run.smithi053.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-25T14:28:42.431 INFO:teuthology.orchestra.run.smithi053.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-25T14:28:42.443 INFO:teuthology.orchestra.run.smithi053.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-25T14:28:42.443 INFO:teuthology.orchestra.run.smithi053.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-25T14:28:42.443 INFO:teuthology.orchestra.run.smithi053.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T14:28:42.443 INFO:teuthology.orchestra.run.smithi053.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-25T14:28:42.443 INFO:teuthology.orchestra.run.smithi053.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fef8:3718/64 scope link noprefixroute 2024-04-25T14:28:42.443 INFO:teuthology.orchestra.run.smithi053.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T14:28:42.446 INFO:teuthology.orchestra.run.smithi053.stdout:Mon IP `172.21.15.53` is in CIDR network `172.21.0.0/20` 2024-04-25T14:28:42.447 INFO:teuthology.orchestra.run.smithi053.stdout:Mon IP `172.21.15.53` is in CIDR network `172.21.0.0/20` 2024-04-25T14:28:42.447 INFO:teuthology.orchestra.run.smithi053.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-25T14:28:42.449 INFO:teuthology.orchestra.run.smithi053.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-25T14:28:42.452 INFO:teuthology.orchestra.run.smithi053.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T14:28:43.069 INFO:teuthology.orchestra.run.smithi053.stdout:/bin/podman: stdout da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-25T14:28:43.069 INFO:teuthology.orchestra.run.smithi053.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T14:28:43.069 INFO:teuthology.orchestra.run.smithi053.stdout:/bin/podman: stderr Getting image source signatures 2024-04-25T14:28:43.069 INFO:teuthology.orchestra.run.smithi053.stdout:/bin/podman: stderr Copying blob sha256:7f03586bcf9c6a2b4d690c118c4aa5b771f9c6f252d75a20fb283dd07093a738 2024-04-25T14:28:43.069 INFO:teuthology.orchestra.run.smithi053.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-25T14:28:43.069 INFO:teuthology.orchestra.run.smithi053.stdout:/bin/podman: stderr Copying config sha256:da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-25T14:28:43.069 INFO:teuthology.orchestra.run.smithi053.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-25T14:28:44.534 INFO:teuthology.orchestra.run.smithi053.stdout:ceph: stdout ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-25T14:28:44.535 INFO:teuthology.orchestra.run.smithi053.stdout:Ceph version: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-25T14:28:44.535 INFO:teuthology.orchestra.run.smithi053.stdout:Extracting ceph user uid/gid from container image... 2024-04-25T14:28:45.772 INFO:teuthology.orchestra.run.smithi053.stdout:stat: stdout 167 167 2024-04-25T14:28:45.772 INFO:teuthology.orchestra.run.smithi053.stdout:Creating initial keys... 2024-04-25T14:28:46.939 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph-authtool: stdout AQCeaCpm529CDxAANSGDagni537Ka9GLivFnQQ== 2024-04-25T14:28:48.023 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph-authtool: stdout AQCfaCpm+lq9FhAATWoG8iW2+5CvH9h6QQ3mYg== 2024-04-25T14:28:49.213 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph-authtool: stdout AQCgaCpm/KLnHBAA684pT8aCV4jvARGpPgzWJQ== 2024-04-25T14:28:49.214 INFO:teuthology.orchestra.run.smithi053.stdout:Creating initial monmap... 2024-04-25T14:28:50.593 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T14:28:50.594 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-25T14:28:50.594 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to fd30e6ce-030f-11ef-bc93-c7b262605968 2024-04-25T14:28:50.594 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T14:28:50.594 INFO:teuthology.orchestra.run.smithi053.stdout:monmaptool for a [v2:172.21.15.53:3300,v1:172.21.15.53:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T14:28:50.594 INFO:teuthology.orchestra.run.smithi053.stdout:setting min_mon_release = quincy 2024-04-25T14:28:50.595 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/monmaptool: set fsid to fd30e6ce-030f-11ef-bc93-c7b262605968 2024-04-25T14:28:50.595 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T14:28:50.595 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:28:50.595 INFO:teuthology.orchestra.run.smithi053.stdout:Creating mon... 2024-04-25T14:28:51.853 INFO:teuthology.orchestra.run.smithi053.stdout:create mon.a on 2024-04-25T14:28:52.257 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-25T14:28:52.515 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-25T14:28:52.814 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-fd30e6ce-030f-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-fd30e6ce-030f-11ef-bc93-c7b262605968.target. 2024-04-25T14:28:52.814 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-fd30e6ce-030f-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-fd30e6ce-030f-11ef-bc93-c7b262605968.target. 2024-04-25T14:28:53.226 INFO:teuthology.orchestra.run.smithi053.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mon.a 2024-04-25T14:28:53.226 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stderr Failed to reset failed state of unit ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-25T14:28:53.521 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-fd30e6ce-030f-11ef-bc93-c7b262605968.target.wants/ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@.service. 2024-04-25T14:28:54.240 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:28:54 smithi053 podman[27380]: 2024-04-25 14:28:54.238554269 +0000 UTC m=+0.456482007 container init 63b7c854f04407123226cb12609b541dc2436a739e76b96046fda166fd6e7d9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mon-a, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=) 2024-04-25T14:28:54.361 INFO:teuthology.orchestra.run.smithi053.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T14:28:54.361 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stdout disabled 2024-04-25T14:28:54.379 INFO:teuthology.orchestra.run.smithi053.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T14:28:54.379 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stdout inactive 2024-04-25T14:28:54.379 INFO:teuthology.orchestra.run.smithi053.stdout:firewalld.service is not enabled 2024-04-25T14:28:54.379 INFO:teuthology.orchestra.run.smithi053.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T14:28:54.380 INFO:teuthology.orchestra.run.smithi053.stdout:Waiting for mon to start... 2024-04-25T14:28:54.380 INFO:teuthology.orchestra.run.smithi053.stdout:Waiting for mon... 2024-04-25T14:28:54.531 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:28:54 smithi053 podman[27380]: 2024-04-25 14:28:54.249555537 +0000 UTC m=+0.467483216 container start 63b7c854f04407123226cb12609b541dc2436a739e76b96046fda166fd6e7d9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mon-a, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-25T14:28:54.531 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:28:54 smithi053 bash[27380]: 63b7c854f04407123226cb12609b541dc2436a739e76b96046fda166fd6e7d9e 2024-04-25T14:28:54.531 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:28:54 smithi053 ceph-mon[27398]: mkfs fd30e6ce-030f-11ef-bc93-c7b262605968 2024-04-25T14:28:54.531 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:28:54 smithi053 systemd[1]: Started Ceph mon.a for fd30e6ce-030f-11ef-bc93-c7b262605968. 2024-04-25T14:28:54.531 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:28:54 smithi053 ceph-mon[27398]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T14:28:55.491 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:28:55 smithi053 ceph-mon[27398]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T14:28:55.491 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:28:55 smithi053 ceph-mon[27398]: monmap epoch 1 2024-04-25T14:28:55.492 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:28:55 smithi053 ceph-mon[27398]: fsid fd30e6ce-030f-11ef-bc93-c7b262605968 2024-04-25T14:28:55.492 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:28:55 smithi053 ceph-mon[27398]: last_changed 2024-04-25T14:28:49.679784+0000 2024-04-25T14:28:55.492 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:28:55 smithi053 ceph-mon[27398]: created 2024-04-25T14:28:49.679784+0000 2024-04-25T14:28:55.492 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:28:55 smithi053 ceph-mon[27398]: min_mon_release 19 (squid) 2024-04-25T14:28:55.492 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:28:55 smithi053 ceph-mon[27398]: election_strategy: 1 2024-04-25T14:28:55.492 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:28:55 smithi053 ceph-mon[27398]: 0: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.a 2024-04-25T14:28:55.492 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:28:55 smithi053 ceph-mon[27398]: fsmap 2024-04-25T14:28:55.492 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:28:55 smithi053 ceph-mon[27398]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T14:28:55.492 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:28:55 smithi053 ceph-mon[27398]: mgrmap e1: no daemons active 2024-04-25T14:28:55.492 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:28:55 smithi053 ceph-mon[27398]: from='client.? 172.21.15.53:0/664556765' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-25T14:28:55.862 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout cluster: 2024-04-25T14:28:55.863 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout id: fd30e6ce-030f-11ef-bc93-c7b262605968 2024-04-25T14:28:55.863 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-25T14:28:55.863 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 2024-04-25T14:28:55.863 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout services: 2024-04-25T14:28:55.864 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.799834s) 2024-04-25T14:28:55.864 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-25T14:28:55.864 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-25T14:28:55.864 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 2024-04-25T14:28:55.864 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout data: 2024-04-25T14:28:55.864 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-25T14:28:55.864 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-25T14:28:55.866 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-25T14:28:55.866 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout pgs: 2024-04-25T14:28:55.866 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 2024-04-25T14:28:55.866 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.033+0000 7fca25400700 1 Processor -- start 2024-04-25T14:28:55.866 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.033+0000 7fca25400700 1 -- start start 2024-04-25T14:28:55.866 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.034+0000 7fca25400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca20106c60 0x7fca20107080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:28:55.866 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.034+0000 7fca25400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fca20063cc0 con 0x7fca20106c60 2024-04-25T14:28:55.866 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.035+0000 7fca1fe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca20106c60 0x7fca20107080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:28:55.866 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.035+0000 7fca1fe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca20106c60 0x7fca20107080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33098/0 (socket says 172.21.15.53:33098) 2024-04-25T14:28:55.866 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.035+0000 7fca1fe00700 1 -- 172.21.15.53:0/1497127825 learned_addr learned my addr 172.21.15.53:0/1497127825 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:28:55.866 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.036+0000 7fca1fe00700 1 -- 172.21.15.53:0/1497127825 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca201075c0 con 0x7fca20106c60 2024-04-25T14:28:55.869 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.036+0000 7fca1fe00700 1 --2- 172.21.15.53:0/1497127825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca20106c60 0x7fca20107080 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fca100098a0 tx=0x7fca10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=878c4b9d7941ca9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:28:55.869 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.036+0000 7fca1ea00700 1 -- 172.21.15.53:0/1497127825 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca10004160 con 0x7fca20106c60 2024-04-25T14:28:55.869 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.036+0000 7fca1ea00700 1 -- 172.21.15.53:0/1497127825 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca10004300 con 0x7fca20106c60 2024-04-25T14:28:55.869 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.036+0000 7fca1ea00700 1 -- 172.21.15.53:0/1497127825 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca100036a0 con 0x7fca20106c60 2024-04-25T14:28:55.869 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.037+0000 7fca25400700 1 -- 172.21.15.53:0/1497127825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca20106c60 msgr2=0x7fca20107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:28:55.869 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.037+0000 7fca25400700 1 --2- 172.21.15.53:0/1497127825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca20106c60 0x7fca20107080 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fca100098a0 tx=0x7fca10009bb0 comp rx=0 tx=0).stop 2024-04-25T14:28:55.869 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.037+0000 7fca25400700 1 -- 172.21.15.53:0/1497127825 shutdown_connections 2024-04-25T14:28:55.869 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.037+0000 7fca25400700 1 --2- 172.21.15.53:0/1497127825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca20106c60 0x7fca20107080 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:28:55.869 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.037+0000 7fca25400700 1 -- 172.21.15.53:0/1497127825 >> 172.21.15.53:0/1497127825 conn(0x7fca20102820 msgr2=0x7fca20104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:28:55.869 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.037+0000 7fca25400700 1 -- 172.21.15.53:0/1497127825 shutdown_connections 2024-04-25T14:28:55.869 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.037+0000 7fca25400700 1 -- 172.21.15.53:0/1497127825 wait complete. 2024-04-25T14:28:55.869 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.038+0000 7fca25400700 1 Processor -- start 2024-04-25T14:28:55.870 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.039+0000 7fca25400700 1 -- start start 2024-04-25T14:28:55.870 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.039+0000 7fca25400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca20106c60 0x7fca2019dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:28:55.870 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.039+0000 7fca25400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fca20107af0 con 0x7fca20106c60 2024-04-25T14:28:55.870 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.039+0000 7fca1fe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca20106c60 0x7fca2019dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:28:55.870 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.039+0000 7fca1fe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca20106c60 0x7fca2019dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33100/0 (socket says 172.21.15.53:33100) 2024-04-25T14:28:55.870 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.039+0000 7fca1fe00700 1 -- 172.21.15.53:0/664556765 learned_addr learned my addr 172.21.15.53:0/664556765 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:28:55.872 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.040+0000 7fca1fe00700 1 -- 172.21.15.53:0/664556765 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca2019e1e0 con 0x7fca20106c60 2024-04-25T14:28:55.872 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.040+0000 7fca1fe00700 1 --2- 172.21.15.53:0/664556765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca20106c60 0x7fca2019dca0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fca10000c00 tx=0x7fca10004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:28:55.872 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.040+0000 7fca1cc00700 1 -- 172.21.15.53:0/664556765 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca10003db0 con 0x7fca20106c60 2024-04-25T14:28:55.873 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.040+0000 7fca1cc00700 1 -- 172.21.15.53:0/664556765 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca10011630 con 0x7fca20106c60 2024-04-25T14:28:55.873 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.040+0000 7fca1cc00700 1 -- 172.21.15.53:0/664556765 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca10011950 con 0x7fca20106c60 2024-04-25T14:28:55.873 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.040+0000 7fca25400700 1 -- 172.21.15.53:0/664556765 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca2019e470 con 0x7fca20106c60 2024-04-25T14:28:55.873 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.041+0000 7fca25400700 1 -- 172.21.15.53:0/664556765 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca201a1160 con 0x7fca20106c60 2024-04-25T14:28:55.873 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.041+0000 7fca1cc00700 1 -- 172.21.15.53:0/664556765 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fca1001a460 con 0x7fca20106c60 2024-04-25T14:28:55.873 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.041+0000 7fca1cc00700 1 -- 172.21.15.53:0/664556765 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fca1001b780 con 0x7fca20106c60 2024-04-25T14:28:55.873 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.042+0000 7fca25400700 1 -- 172.21.15.53:0/664556765 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca20069ba0 con 0x7fca20106c60 2024-04-25T14:28:55.873 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.045+0000 7fca1cc00700 1 -- 172.21.15.53:0/664556765 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fca1001b980 con 0x7fca20106c60 2024-04-25T14:28:55.873 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.126+0000 7fca25400700 1 -- 172.21.15.53:0/664556765 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fca201a1330 con 0x7fca20106c60 2024-04-25T14:28:55.873 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.127+0000 7fca1cc00700 1 -- 172.21.15.53:0/664556765 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fca1001bb60 con 0x7fca20106c60 2024-04-25T14:28:55.873 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.128+0000 7fca25400700 1 -- 172.21.15.53:0/664556765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca20106c60 msgr2=0x7fca2019dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:28:55.873 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.128+0000 7fca25400700 1 --2- 172.21.15.53:0/664556765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca20106c60 0x7fca2019dca0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fca10000c00 tx=0x7fca10004750 comp rx=0 tx=0).stop 2024-04-25T14:28:55.874 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.128+0000 7fca25400700 1 -- 172.21.15.53:0/664556765 shutdown_connections 2024-04-25T14:28:55.874 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.128+0000 7fca25400700 1 --2- 172.21.15.53:0/664556765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca20106c60 0x7fca2019dca0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:28:55.874 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.128+0000 7fca25400700 1 -- 172.21.15.53:0/664556765 >> 172.21.15.53:0/664556765 conn(0x7fca20102820 msgr2=0x7fca201942e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:28:55.874 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.129+0000 7fca25400700 1 -- 172.21.15.53:0/664556765 shutdown_connections 2024-04-25T14:28:55.874 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:55.129+0000 7fca25400700 1 -- 172.21.15.53:0/664556765 wait complete. 2024-04-25T14:28:55.874 INFO:teuthology.orchestra.run.smithi053.stdout:mon is available 2024-04-25T14:28:55.874 INFO:teuthology.orchestra.run.smithi053.stdout:Assimilating anything we can from ceph.conf... 2024-04-25T14:28:56.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:28:56 smithi053 ceph-mon[27398]: from='client.? 172.21.15.53:0/3181169814' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T14:28:56.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:28:56 smithi053 ceph-mon[27398]: from='client.? 172.21.15.53:0/3181169814' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-25T14:28:57.360 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 2024-04-25T14:28:57.361 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T14:28:57.361 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout fsid = fd30e6ce-030f-11ef-bc93-c7b262605968 2024-04-25T14:28:57.361 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T14:28:57.361 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.53:3300,v1:172.21.15.53:6789] 2024-04-25T14:28:57.361 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T14:28:57.361 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T14:28:57.361 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T14:28:57.361 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T14:28:57.361 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 2024-04-25T14:28:57.361 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T14:28:57.361 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T14:28:57.361 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 2024-04-25T14:28:57.361 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T14:28:57.362 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-25T14:28:57.362 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T14:28:57.367 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T14:28:57.367 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T14:28:57.367 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T14:28:57.367 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T14:28:57.367 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.551+0000 7f2036a00700 1 Processor -- start 2024-04-25T14:28:57.367 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.551+0000 7f2036a00700 1 -- start start 2024-04-25T14:28:57.367 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.552+0000 7f2036a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2030110220 0x7f2030110640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:28:57.368 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.552+0000 7f2036a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2030110c10 con 0x7f2030110220 2024-04-25T14:28:57.368 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.552+0000 7f202fe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2030110220 0x7f2030110640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:28:57.368 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.552+0000 7f202fe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2030110220 0x7f2030110640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33114/0 (socket says 172.21.15.53:33114) 2024-04-25T14:28:57.368 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.552+0000 7f202fe00700 1 -- 172.21.15.53:0/3261749348 learned_addr learned my addr 172.21.15.53:0/3261749348 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:28:57.368 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.552+0000 7f202fe00700 1 -- 172.21.15.53:0/3261749348 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f203010ece0 con 0x7f2030110220 2024-04-25T14:28:57.368 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.553+0000 7f202fe00700 1 --2- 172.21.15.53:0/3261749348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2030110220 0x7f2030110640 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f20200098a0 tx=0x7f2020009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e8122b62649ade server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:28:57.368 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.553+0000 7f202ea00700 1 -- 172.21.15.53:0/3261749348 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2020004160 con 0x7f2030110220 2024-04-25T14:28:57.368 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.553+0000 7f202ea00700 1 -- 172.21.15.53:0/3261749348 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2020004300 con 0x7f2030110220 2024-04-25T14:28:57.368 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.553+0000 7f202ea00700 1 -- 172.21.15.53:0/3261749348 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20200036a0 con 0x7f2030110220 2024-04-25T14:28:57.369 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.554+0000 7f2036a00700 1 -- 172.21.15.53:0/3261749348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2030110220 msgr2=0x7f2030110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:28:57.370 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.554+0000 7f2036a00700 1 --2- 172.21.15.53:0/3261749348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2030110220 0x7f2030110640 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f20200098a0 tx=0x7f2020009bb0 comp rx=0 tx=0).stop 2024-04-25T14:28:57.370 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.554+0000 7f2036a00700 1 -- 172.21.15.53:0/3261749348 shutdown_connections 2024-04-25T14:28:57.370 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.554+0000 7f2036a00700 1 --2- 172.21.15.53:0/3261749348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2030110220 0x7f2030110640 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:28:57.370 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.554+0000 7f2036a00700 1 -- 172.21.15.53:0/3261749348 >> 172.21.15.53:0/3261749348 conn(0x7f203007de20 msgr2=0x7f2030080240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:28:57.370 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.554+0000 7f2036a00700 1 -- 172.21.15.53:0/3261749348 shutdown_connections 2024-04-25T14:28:57.370 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.554+0000 7f2036a00700 1 -- 172.21.15.53:0/3261749348 wait complete. 2024-04-25T14:28:57.370 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.555+0000 7f2036a00700 1 Processor -- start 2024-04-25T14:28:57.370 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.556+0000 7f2036a00700 1 -- start start 2024-04-25T14:28:57.370 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.556+0000 7f2036a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2030110220 0x7f20301a2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:28:57.370 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.556+0000 7f2036a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f203010f1b0 con 0x7f2030110220 2024-04-25T14:28:57.370 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.556+0000 7f202fe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2030110220 0x7f20301a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:28:57.370 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.556+0000 7f202fe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2030110220 0x7f20301a2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33120/0 (socket says 172.21.15.53:33120) 2024-04-25T14:28:57.370 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.556+0000 7f202fe00700 1 -- 172.21.15.53:0/3181169814 learned_addr learned my addr 172.21.15.53:0/3181169814 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:28:57.371 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.557+0000 7f202fe00700 1 -- 172.21.15.53:0/3181169814 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20301a28b0 con 0x7f2030110220 2024-04-25T14:28:57.371 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.557+0000 7f202fe00700 1 --2- 172.21.15.53:0/3181169814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2030110220 0x7f20301a2370 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f2020000c00 tx=0x7f2020004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:28:57.371 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.557+0000 7f202cc00700 1 -- 172.21.15.53:0/3181169814 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2020003db0 con 0x7f2030110220 2024-04-25T14:28:57.371 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.557+0000 7f202cc00700 1 -- 172.21.15.53:0/3181169814 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2020011630 con 0x7f2030110220 2024-04-25T14:28:57.371 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.557+0000 7f202cc00700 1 -- 172.21.15.53:0/3181169814 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2020011950 con 0x7f2030110220 2024-04-25T14:28:57.371 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.557+0000 7f2036a00700 1 -- 172.21.15.53:0/3181169814 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20301a2b40 con 0x7f2030110220 2024-04-25T14:28:57.371 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.557+0000 7f2036a00700 1 -- 172.21.15.53:0/3181169814 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20301a5830 con 0x7f2030110220 2024-04-25T14:28:57.371 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.558+0000 7f202cc00700 1 -- 172.21.15.53:0/3181169814 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f202001a460 con 0x7f2030110220 2024-04-25T14:28:57.371 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.558+0000 7f202cc00700 1 -- 172.21.15.53:0/3181169814 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f202001b780 con 0x7f2030110220 2024-04-25T14:28:57.371 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.559+0000 7f2036a00700 1 -- 172.21.15.53:0/3181169814 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20300640a0 con 0x7f2030110220 2024-04-25T14:28:57.371 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.562+0000 7f202cc00700 1 -- 172.21.15.53:0/3181169814 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f2020031020 con 0x7f2030110220 2024-04-25T14:28:57.371 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.641+0000 7f2036a00700 1 -- 172.21.15.53:0/3181169814 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f203002e170 con 0x7f2030110220 2024-04-25T14:28:57.371 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.653+0000 7f202cc00700 1 -- 172.21.15.53:0/3181169814 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+492 (secure 0 0 0) 0x7f202001b9d0 con 0x7f2030110220 2024-04-25T14:28:57.372 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.654+0000 7f202cc00700 1 -- 172.21.15.53:0/3181169814 <== mon.0 v2:172.21.15.53:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f202001ad20 con 0x7f2030110220 2024-04-25T14:28:57.372 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.655+0000 7f2036a00700 1 -- 172.21.15.53:0/3181169814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2030110220 msgr2=0x7f20301a2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:28:57.372 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.655+0000 7f2036a00700 1 --2- 172.21.15.53:0/3181169814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2030110220 0x7f20301a2370 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f2020000c00 tx=0x7f2020004750 comp rx=0 tx=0).stop 2024-04-25T14:28:57.372 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.655+0000 7f2036a00700 1 -- 172.21.15.53:0/3181169814 shutdown_connections 2024-04-25T14:28:57.372 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.655+0000 7f2036a00700 1 --2- 172.21.15.53:0/3181169814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2030110220 0x7f20301a2370 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:28:57.372 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.655+0000 7f2036a00700 1 -- 172.21.15.53:0/3181169814 >> 172.21.15.53:0/3181169814 conn(0x7f203007de20 msgr2=0x7f2030198f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:28:57.372 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.655+0000 7f2036a00700 1 -- 172.21.15.53:0/3181169814 shutdown_connections 2024-04-25T14:28:57.372 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:56.655+0000 7f2036a00700 1 -- 172.21.15.53:0/3181169814 wait complete. 2024-04-25T14:28:57.372 INFO:teuthology.orchestra.run.smithi053.stdout:Generating new minimal ceph.conf... 2024-04-25T14:28:58.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:28:58 smithi053 ceph-mon[27398]: from='client.? 172.21.15.53:0/2236134800' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:28:58.781 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.004+0000 7fec05a00700 1 Processor -- start 2024-04-25T14:28:58.781 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.005+0000 7fec05a00700 1 -- start start 2024-04-25T14:28:58.781 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.005+0000 7fec05a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec0007cda0 0x7fec0007b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:28:58.781 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.005+0000 7fec05a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fec0007b7d0 con 0x7fec0007cda0 2024-04-25T14:28:58.781 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.005+0000 7febfea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec0007cda0 0x7fec0007b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:28:58.781 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.006+0000 7febfea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec0007cda0 0x7fec0007b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33136/0 (socket says 172.21.15.53:33136) 2024-04-25T14:28:58.781 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.006+0000 7febfea00700 1 -- 172.21.15.53:0/3281758829 learned_addr learned my addr 172.21.15.53:0/3281758829 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:28:58.781 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.006+0000 7febfea00700 1 -- 172.21.15.53:0/3281758829 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec0007b950 con 0x7fec0007cda0 2024-04-25T14:28:58.782 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.006+0000 7febfea00700 1 --2- 172.21.15.53:0/3281758829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec0007cda0 0x7fec0007b200 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7febf00098a0 tx=0x7febf0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbd65134f7856cc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:28:58.782 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.007+0000 7febfe000700 1 -- 172.21.15.53:0/3281758829 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febf0004160 con 0x7fec0007cda0 2024-04-25T14:28:58.782 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.007+0000 7febfe000700 1 -- 172.21.15.53:0/3281758829 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7febf0004300 con 0x7fec0007cda0 2024-04-25T14:28:58.782 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.007+0000 7febfe000700 1 -- 172.21.15.53:0/3281758829 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febf0011af0 con 0x7fec0007cda0 2024-04-25T14:28:58.782 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.007+0000 7fec05a00700 1 -- 172.21.15.53:0/3281758829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec0007cda0 msgr2=0x7fec0007b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:28:58.782 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.007+0000 7fec05a00700 1 --2- 172.21.15.53:0/3281758829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec0007cda0 0x7fec0007b200 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7febf00098a0 tx=0x7febf0009bb0 comp rx=0 tx=0).stop 2024-04-25T14:28:58.782 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.008+0000 7fec05a00700 1 -- 172.21.15.53:0/3281758829 shutdown_connections 2024-04-25T14:28:58.782 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.008+0000 7fec05a00700 1 --2- 172.21.15.53:0/3281758829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec0007cda0 0x7fec0007b200 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:28:58.782 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.008+0000 7fec05a00700 1 -- 172.21.15.53:0/3281758829 >> 172.21.15.53:0/3281758829 conn(0x7fec00103b50 msgr2=0x7fec00105f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:28:58.782 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.008+0000 7fec05a00700 1 -- 172.21.15.53:0/3281758829 shutdown_connections 2024-04-25T14:28:58.782 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.008+0000 7fec05a00700 1 -- 172.21.15.53:0/3281758829 wait complete. 2024-04-25T14:28:58.782 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.009+0000 7fec05a00700 1 Processor -- start 2024-04-25T14:28:58.783 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.009+0000 7fec05a00700 1 -- start start 2024-04-25T14:28:58.783 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.010+0000 7fec05a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec0007cda0 0x7fec0019e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:28:58.783 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.010+0000 7fec05a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fec0010eae0 con 0x7fec0007cda0 2024-04-25T14:28:58.783 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.010+0000 7febfea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec0007cda0 0x7fec0019e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:28:58.783 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.010+0000 7febfea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec0007cda0 0x7fec0019e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33140/0 (socket says 172.21.15.53:33140) 2024-04-25T14:28:58.783 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.010+0000 7febfea00700 1 -- 172.21.15.53:0/2236134800 learned_addr learned my addr 172.21.15.53:0/2236134800 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:28:58.783 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.011+0000 7febfea00700 1 -- 172.21.15.53:0/2236134800 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec0019ebe0 con 0x7fec0007cda0 2024-04-25T14:28:58.783 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.011+0000 7febfea00700 1 --2- 172.21.15.53:0/2236134800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec0007cda0 0x7fec0019e6a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7febf0000c00 tx=0x7febf0011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:28:58.783 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.011+0000 7febf7e00700 1 -- 172.21.15.53:0/2236134800 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febf0003b20 con 0x7fec0007cda0 2024-04-25T14:28:58.783 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.011+0000 7febf7e00700 1 -- 172.21.15.53:0/2236134800 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7febf001a8c0 con 0x7fec0007cda0 2024-04-25T14:28:58.783 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.011+0000 7febf7e00700 1 -- 172.21.15.53:0/2236134800 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febf0024940 con 0x7fec0007cda0 2024-04-25T14:28:58.783 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.011+0000 7fec05a00700 1 -- 172.21.15.53:0/2236134800 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec0019ee70 con 0x7fec0007cda0 2024-04-25T14:28:58.783 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.011+0000 7fec05a00700 1 -- 172.21.15.53:0/2236134800 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec001a1b60 con 0x7fec0007cda0 2024-04-25T14:28:58.783 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.013+0000 7febf7e00700 1 -- 172.21.15.53:0/2236134800 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7febf0022020 con 0x7fec0007cda0 2024-04-25T14:28:58.784 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.013+0000 7febf7e00700 1 -- 172.21.15.53:0/2236134800 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7febf002d930 con 0x7fec0007cda0 2024-04-25T14:28:58.784 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.013+0000 7fec05a00700 1 -- 172.21.15.53:0/2236134800 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec000640a0 con 0x7fec0007cda0 2024-04-25T14:28:58.784 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.016+0000 7febf7e00700 1 -- 172.21.15.53:0/2236134800 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7febf00042a0 con 0x7fec0007cda0 2024-04-25T14:28:58.784 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.095+0000 7fec05a00700 1 -- 172.21.15.53:0/2236134800 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fec001a2100 con 0x7fec0007cda0 2024-04-25T14:28:58.784 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.096+0000 7febf7e00700 1 -- 172.21.15.53:0/2236134800 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7febf0032030 con 0x7fec0007cda0 2024-04-25T14:28:58.784 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.097+0000 7fec05a00700 1 -- 172.21.15.53:0/2236134800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec0007cda0 msgr2=0x7fec0019e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:28:58.784 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.097+0000 7fec05a00700 1 --2- 172.21.15.53:0/2236134800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec0007cda0 0x7fec0019e6a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7febf0000c00 tx=0x7febf0011b70 comp rx=0 tx=0).stop 2024-04-25T14:28:58.784 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.097+0000 7fec05a00700 1 -- 172.21.15.53:0/2236134800 shutdown_connections 2024-04-25T14:28:58.785 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.097+0000 7fec05a00700 1 --2- 172.21.15.53:0/2236134800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec0007cda0 0x7fec0019e6a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:28:58.785 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.097+0000 7fec05a00700 1 -- 172.21.15.53:0/2236134800 >> 172.21.15.53:0/2236134800 conn(0x7fec00103b50 msgr2=0x7fec00104660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:28:58.785 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.097+0000 7fec05a00700 1 -- 172.21.15.53:0/2236134800 shutdown_connections 2024-04-25T14:28:58.785 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:28:58.098+0000 7fec05a00700 1 -- 172.21.15.53:0/2236134800 wait complete. 2024-04-25T14:28:58.785 INFO:teuthology.orchestra.run.smithi053.stdout:Restarting the monitor... 2024-04-25T14:28:59.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:28:58 smithi053 systemd[1]: Stopping Ceph mon.a for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:29:00.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:00 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mon-a[27394]: 2024-04-25T14:29:00.056+0000 7faf0ea00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:29:00.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:00 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mon-a[27394]: 2024-04-25T14:29:00.056+0000 7faf0ea00700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-25T14:29:00.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:00 smithi053 podman[27642]: 2024-04-25 14:29:00.109648191 +0000 UTC m=+0.954029874 container died 63b7c854f04407123226cb12609b541dc2436a739e76b96046fda166fd6e7d9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-25T14:29:09.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:08 smithi053 podman[27642]: 2024-04-25 14:29:08.791444295 +0000 UTC m=+9.635825985 container cleanup 63b7c854f04407123226cb12609b541dc2436a739e76b96046fda166fd6e7d9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:29:09.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:08 smithi053 bash[27642]: ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mon-a 2024-04-25T14:29:09.507 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:09 smithi053 podman[27655]: 2024-04-25 14:29:09.063533809 +0000 UTC m=+8.939352168 container remove 63b7c854f04407123226cb12609b541dc2436a739e76b96046fda166fd6e7d9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-25T14:29:09.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:09 smithi053 systemd[1]: ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-25T14:29:09.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:09 smithi053 systemd[1]: Stopped Ceph mon.a for fd30e6ce-030f-11ef-bc93-c7b262605968. 2024-04-25T14:29:09.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:09 smithi053 systemd[1]: ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mon.a.service: Consumed 2.012s CPU time. 2024-04-25T14:29:09.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:09 smithi053 systemd[1]: Starting Ceph mon.a for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:29:10.051 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:09 smithi053 podman[27747]: 2024-04-25 14:29:09.845856545 +0000 UTC m=+0.040669006 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T14:29:10.371 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 podman[27747]: 2024-04-25 14:29:10.051419938 +0000 UTC m=+0.246232470 container create fc4336fa1ec1552dd465a162c199d50f3682c6bbffff41df7bd13f1574ee5f8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-25T14:29:10.503 INFO:teuthology.orchestra.run.smithi053.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-25T14:29:10.662 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 podman[27747]: 2024-04-25 14:29:10.371868554 +0000 UTC m=+0.566681060 container init fc4336fa1ec1552dd465a162c199d50f3682c6bbffff41df7bd13f1574ee5f8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mon-a, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416) 2024-04-25T14:29:10.662 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 podman[27747]: 2024-04-25 14:29:10.383069686 +0000 UTC m=+0.577882189 container start fc4336fa1ec1552dd465a162c199d50f3682c6bbffff41df7bd13f1574ee5f8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD) 2024-04-25T14:29:10.663 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T14:29:10.663 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-25T14:29:10.663 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: pidfile_write: ignore empty --pid-file 2024-04-25T14:29:10.663 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: load: jerasure load: lrc 2024-04-25T14:29:10.663 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: RocksDB version: 7.9.2 2024-04-25T14:29:10.663 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Git sha 0 2024-04-25T14:29:10.663 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-25T14:29:10.663 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: DB SUMMARY 2024-04-25T14:29:10.663 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: DB Session ID: A1HTQSBVRBJS1MU3UVL5 2024-04-25T14:29:10.663 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: CURRENT file: CURRENT 2024-04-25T14:29:10.663 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T14:29:10.663 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-25T14:29:10.663 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-25T14:29:10.663 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105444 ; 2024-04-25T14:29:10.664 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.error_if_exists: 0 2024-04-25T14:29:10.664 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.create_if_missing: 0 2024-04-25T14:29:10.664 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.paranoid_checks: 1 2024-04-25T14:29:10.664 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T14:29:10.664 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T14:29:10.664 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T14:29:10.664 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.env: 0x5584c50e1cc0 2024-04-25T14:29:10.664 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T14:29:10.664 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.info_log: 0x558504387c40 2024-04-25T14:29:10.664 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T14:29:10.664 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.statistics: (nil) 2024-04-25T14:29:10.665 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.use_fsync: 0 2024-04-25T14:29:10.665 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.max_log_file_size: 0 2024-04-25T14:29:10.665 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T14:29:10.665 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T14:29:10.665 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T14:29:10.665 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T14:29:10.665 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.allow_fallocate: 1 2024-04-25T14:29:10.665 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T14:29:10.666 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T14:29:10.666 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.use_direct_reads: 0 2024-04-25T14:29:10.666 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T14:29:10.666 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T14:29:10.666 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.db_log_dir: 2024-04-25T14:29:10.666 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.wal_dir: 2024-04-25T14:29:10.666 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T14:29:10.666 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T14:29:10.666 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T14:29:10.666 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T14:29:10.666 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T14:29:10.666 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T14:29:10.666 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T14:29:10.666 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T14:29:10.667 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.write_buffer_manager: 0x55850431b0e0 2024-04-25T14:29:10.667 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T14:29:10.667 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T14:29:10.667 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T14:29:10.667 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T14:29:10.667 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T14:29:10.667 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T14:29:10.667 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T14:29:10.667 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T14:29:10.667 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.unordered_write: 0 2024-04-25T14:29:10.667 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T14:29:10.667 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T14:29:10.667 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T14:29:10.667 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T14:29:10.668 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.row_cache: None 2024-04-25T14:29:10.668 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.wal_filter: None 2024-04-25T14:29:10.668 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T14:29:10.668 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T14:29:10.668 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.two_write_queues: 0 2024-04-25T14:29:10.668 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T14:29:10.668 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.wal_compression: 0 2024-04-25T14:29:10.668 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.atomic_flush: 0 2024-04-25T14:29:10.668 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T14:29:10.668 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T14:29:10.668 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T14:29:10.668 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.log_readahead_size: 0 2024-04-25T14:29:10.669 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T14:29:10.669 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T14:29:10.669 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T14:29:10.669 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T14:29:10.669 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T14:29:10.669 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T14:29:10.669 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T14:29:10.669 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.max_background_jobs: 2 2024-04-25T14:29:10.669 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.max_background_compactions: -1 2024-04-25T14:29:10.669 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.max_subcompactions: 1 2024-04-25T14:29:10.669 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T14:29:10.669 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T14:29:10.669 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T14:29:10.669 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T14:29:10.669 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T14:29:10.670 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T14:29:10.670 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T14:29:10.670 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T14:29:10.670 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.max_open_files: -1 2024-04-25T14:29:10.670 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T14:29:10.670 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T14:29:10.670 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T14:29:10.670 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T14:29:10.670 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.max_background_flushes: -1 2024-04-25T14:29:10.670 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Compression algorithms supported: 2024-04-25T14:29:10.670 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T14:29:10.670 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: kZSTD supported: 0 2024-04-25T14:29:10.670 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: kXpressCompression supported: 0 2024-04-25T14:29:10.670 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T14:29:10.670 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: kZlibCompression supported: 1 2024-04-25T14:29:10.671 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: kSnappyCompression supported: 1 2024-04-25T14:29:10.671 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: kLZ4Compression supported: 1 2024-04-25T14:29:10.671 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: kBZip2Compression supported: 0 2024-04-25T14:29:10.671 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T14:29:10.671 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T14:29:10.671 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-25T14:29:10.671 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T14:29:10.671 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T14:29:10.671 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.merge_operator: 2024-04-25T14:29:10.671 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.compaction_filter: None 2024-04-25T14:29:10.671 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T14:29:10.671 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T14:29:10.671 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T14:29:10.671 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T14:29:10.671 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558504387e00) 2024-04-25T14:29:10.672 INFO:journalctl@ceph.mon.a.smithi053.stdout: cache_index_and_filter_blocks: 1 2024-04-25T14:29:10.672 INFO:journalctl@ceph.mon.a.smithi053.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T14:29:10.672 INFO:journalctl@ceph.mon.a.smithi053.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T14:29:10.672 INFO:journalctl@ceph.mon.a.smithi053.stdout: pin_top_level_index_and_filter: 1 2024-04-25T14:29:10.672 INFO:journalctl@ceph.mon.a.smithi053.stdout: index_type: 0 2024-04-25T14:29:10.672 INFO:journalctl@ceph.mon.a.smithi053.stdout: data_block_index_type: 0 2024-04-25T14:29:10.672 INFO:journalctl@ceph.mon.a.smithi053.stdout: index_shortening: 1 2024-04-25T14:29:10.672 INFO:journalctl@ceph.mon.a.smithi053.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T14:29:10.672 INFO:journalctl@ceph.mon.a.smithi053.stdout: checksum: 4 2024-04-25T14:29:10.672 INFO:journalctl@ceph.mon.a.smithi053.stdout: no_block_cache: 0 2024-04-25T14:29:10.672 INFO:journalctl@ceph.mon.a.smithi053.stdout: block_cache: 0x55850434d1f0 2024-04-25T14:29:10.672 INFO:journalctl@ceph.mon.a.smithi053.stdout: block_cache_name: BinnedLRUCache 2024-04-25T14:29:10.672 INFO:journalctl@ceph.mon.a.smithi053.stdout: block_cache_options: 2024-04-25T14:29:10.672 INFO:journalctl@ceph.mon.a.smithi053.stdout: capacity : 536870912 2024-04-25T14:29:10.672 INFO:journalctl@ceph.mon.a.smithi053.stdout: num_shard_bits : 4 2024-04-25T14:29:10.672 INFO:journalctl@ceph.mon.a.smithi053.stdout: strict_capacity_limit : 0 2024-04-25T14:29:10.673 INFO:journalctl@ceph.mon.a.smithi053.stdout: high_pri_pool_ratio: 0.000 2024-04-25T14:29:10.673 INFO:journalctl@ceph.mon.a.smithi053.stdout: block_cache_compressed: (nil) 2024-04-25T14:29:10.673 INFO:journalctl@ceph.mon.a.smithi053.stdout: persistent_cache: (nil) 2024-04-25T14:29:10.673 INFO:journalctl@ceph.mon.a.smithi053.stdout: block_size: 4096 2024-04-25T14:29:10.673 INFO:journalctl@ceph.mon.a.smithi053.stdout: block_size_deviation: 10 2024-04-25T14:29:10.673 INFO:journalctl@ceph.mon.a.smithi053.stdout: block_restart_interval: 16 2024-04-25T14:29:10.673 INFO:journalctl@ceph.mon.a.smithi053.stdout: index_block_restart_interval: 1 2024-04-25T14:29:10.673 INFO:journalctl@ceph.mon.a.smithi053.stdout: metadata_block_size: 4096 2024-04-25T14:29:10.673 INFO:journalctl@ceph.mon.a.smithi053.stdout: partition_filters: 0 2024-04-25T14:29:10.673 INFO:journalctl@ceph.mon.a.smithi053.stdout: use_delta_encoding: 1 2024-04-25T14:29:10.673 INFO:journalctl@ceph.mon.a.smithi053.stdout: filter_policy: bloomfilter 2024-04-25T14:29:10.673 INFO:journalctl@ceph.mon.a.smithi053.stdout: whole_key_filtering: 1 2024-04-25T14:29:10.673 INFO:journalctl@ceph.mon.a.smithi053.stdout: verify_compression: 0 2024-04-25T14:29:10.673 INFO:journalctl@ceph.mon.a.smithi053.stdout: read_amp_bytes_per_bit: 0 2024-04-25T14:29:10.673 INFO:journalctl@ceph.mon.a.smithi053.stdout: format_version: 5 2024-04-25T14:29:10.673 INFO:journalctl@ceph.mon.a.smithi053.stdout: enable_index_compression: 1 2024-04-25T14:29:10.673 INFO:journalctl@ceph.mon.a.smithi053.stdout: block_align: 0 2024-04-25T14:29:10.674 INFO:journalctl@ceph.mon.a.smithi053.stdout: max_auto_readahead_size: 262144 2024-04-25T14:29:10.674 INFO:journalctl@ceph.mon.a.smithi053.stdout: prepopulate_block_cache: 0 2024-04-25T14:29:10.674 INFO:journalctl@ceph.mon.a.smithi053.stdout: initial_auto_readahead_size: 8192 2024-04-25T14:29:10.674 INFO:journalctl@ceph.mon.a.smithi053.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T14:29:10.675 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T14:29:10.675 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T14:29:10.675 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.compression: NoCompression 2024-04-25T14:29:10.675 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T14:29:10.675 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T14:29:10.675 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T14:29:10.675 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.num_levels: 7 2024-04-25T14:29:10.675 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T14:29:10.675 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T14:29:10.675 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T14:29:10.675 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T14:29:10.675 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T14:29:10.675 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T14:29:10.676 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T14:29:10.676 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T14:29:10.676 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T14:29:10.676 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T14:29:10.676 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T14:29:10.676 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T14:29:10.676 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T14:29:10.676 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T14:29:10.676 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T14:29:10.676 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T14:29:10.676 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T14:29:10.676 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T14:29:10.676 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T14:29:10.676 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T14:29:10.676 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T14:29:10.676 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T14:29:10.677 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T14:29:10.677 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T14:29:10.677 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T14:29:10.677 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T14:29:10.677 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T14:29:10.677 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T14:29:10.677 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T14:29:10.677 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T14:29:10.677 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T14:29:10.677 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T14:29:10.677 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T14:29:10.677 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T14:29:10.677 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T14:29:10.677 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T14:29:10.678 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T14:29:10.678 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T14:29:10.678 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T14:29:10.678 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T14:29:10.678 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T14:29:10.678 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T14:29:10.678 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T14:29:10.678 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T14:29:10.678 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T14:29:10.678 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T14:29:10.678 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T14:29:10.678 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T14:29:10.678 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T14:29:10.678 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T14:29:10.679 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T14:29:10.679 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T14:29:10.679 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T14:29:10.679 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T14:29:10.679 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.inplace_update_support: 0 2024-04-25T14:29:10.679 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T14:29:10.679 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T14:29:10.679 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T14:29:10.679 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T14:29:10.679 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.bloom_locality: 0 2024-04-25T14:29:10.679 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.max_successive_merges: 0 2024-04-25T14:29:10.679 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T14:29:10.679 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T14:29:10.679 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T14:29:10.679 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T14:29:10.680 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.ttl: 2592000 2024-04-25T14:29:10.680 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T14:29:10.680 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T14:29:10.680 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T14:29:10.680 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.enable_blob_files: false 2024-04-25T14:29:10.680 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.min_blob_size: 0 2024-04-25T14:29:10.680 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T14:29:10.680 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T14:29:10.680 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T14:29:10.680 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T14:29:10.680 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T14:29:10.680 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T14:29:10.680 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T14:29:10.680 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T14:29:10.680 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-25T14:29:10.681 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-25T14:29:10.681 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: aeda4326-292a-4870-8f76-703ea3dee051 2024-04-25T14:29:10.681 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714055350439668, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-25T14:29:10.681 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-25T14:29:10.681 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714055350441782, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100556, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98441, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14264, "raw_average_key_size": 49, "raw_value_size": 90695, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714055350, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "aeda4326-292a-4870-8f76-703ea3dee051", "db_session_id": "A1HTQSBVRBJS1MU3UVL5", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-25T14:29:10.681 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714055350441933, "job": 1, "event": "recovery_finished"} 2024-04-25T14:29:10.681 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-25T14:29:10.681 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T14:29:10.681 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x558504462000 2024-04-25T14:29:10.681 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: DB pointer 0x55850444c000 2024-04-25T14:29:10.681 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T14:29:10.681 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T14:29:10.681 INFO:journalctl@ceph.mon.a.smithi053.stdout: ** DB Stats ** 2024-04-25T14:29:10.681 INFO:journalctl@ceph.mon.a.smithi053.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T14:29:10.681 INFO:journalctl@ceph.mon.a.smithi053.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T14:29:10.682 INFO:journalctl@ceph.mon.a.smithi053.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T14:29:10.682 INFO:journalctl@ceph.mon.a.smithi053.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T14:29:10.682 INFO:journalctl@ceph.mon.a.smithi053.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T14:29:10.682 INFO:journalctl@ceph.mon.a.smithi053.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T14:29:10.682 INFO:journalctl@ceph.mon.a.smithi053.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T14:29:10.682 INFO:journalctl@ceph.mon.a.smithi053.stdout: 2024-04-25T14:29:10.682 INFO:journalctl@ceph.mon.a.smithi053.stdout: ** Compaction Stats [default] ** 2024-04-25T14:29:10.682 INFO:journalctl@ceph.mon.a.smithi053.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T14:29:10.682 INFO:journalctl@ceph.mon.a.smithi053.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T14:29:10.682 INFO:journalctl@ceph.mon.a.smithi053.stdout: L0 2/0 100.06 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 67.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T14:29:10.682 INFO:journalctl@ceph.mon.a.smithi053.stdout: Sum 2/0 100.06 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 67.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T14:29:10.682 INFO:journalctl@ceph.mon.a.smithi053.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 67.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T14:29:10.682 INFO:journalctl@ceph.mon.a.smithi053.stdout: 2024-04-25T14:29:10.682 INFO:journalctl@ceph.mon.a.smithi053.stdout: ** Compaction Stats [default] ** 2024-04-25T14:29:10.683 INFO:journalctl@ceph.mon.a.smithi053.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T14:29:10.683 INFO:journalctl@ceph.mon.a.smithi053.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T14:29:10.683 INFO:journalctl@ceph.mon.a.smithi053.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 67.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T14:29:10.683 INFO:journalctl@ceph.mon.a.smithi053.stdout: 2024-04-25T14:29:10.683 INFO:journalctl@ceph.mon.a.smithi053.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T14:29:10.683 INFO:journalctl@ceph.mon.a.smithi053.stdout: 2024-04-25T14:29:10.683 INFO:journalctl@ceph.mon.a.smithi053.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T14:29:10.683 INFO:journalctl@ceph.mon.a.smithi053.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T14:29:10.683 INFO:journalctl@ceph.mon.a.smithi053.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T14:29:10.683 INFO:journalctl@ceph.mon.a.smithi053.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T14:29:10.683 INFO:journalctl@ceph.mon.a.smithi053.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T14:29:10.683 INFO:journalctl@ceph.mon.a.smithi053.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T14:29:10.683 INFO:journalctl@ceph.mon.a.smithi053.stdout: Cumulative compaction: 0.00 GB write, 14.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T14:29:10.684 INFO:journalctl@ceph.mon.a.smithi053.stdout: Interval compaction: 0.00 GB write, 14.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T14:29:10.684 INFO:journalctl@ceph.mon.a.smithi053.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T14:29:10.684 INFO:journalctl@ceph.mon.a.smithi053.stdout: Block cache BinnedLRUCache@0x55850434d1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-04-25T14:29:10.684 INFO:journalctl@ceph.mon.a.smithi053.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-25T14:29:10.684 INFO:journalctl@ceph.mon.a.smithi053.stdout: 2024-04-25T14:29:10.684 INFO:journalctl@ceph.mon.a.smithi053.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T14:29:10.684 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: starting mon.a rank 0 at public addrs [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] at bind addrs [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid fd30e6ce-030f-11ef-bc93-c7b262605968 2024-04-25T14:29:10.684 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: mon.a@-1(???) e1 preinit fsid fd30e6ce-030f-11ef-bc93-c7b262605968 2024-04-25T14:29:10.684 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: mon.a@-1(???).mds e1 new map 2024-04-25T14:29:10.684 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: mon.a@-1(???).mds e1 print_map 2024-04-25T14:29:10.684 INFO:journalctl@ceph.mon.a.smithi053.stdout: e1 2024-04-25T14:29:10.684 INFO:journalctl@ceph.mon.a.smithi053.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T14:29:10.684 INFO:journalctl@ceph.mon.a.smithi053.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T14:29:10.684 INFO:journalctl@ceph.mon.a.smithi053.stdout: legacy client fscid: -1 2024-04-25T14:29:10.685 INFO:journalctl@ceph.mon.a.smithi053.stdout: 2024-04-25T14:29:10.685 INFO:journalctl@ceph.mon.a.smithi053.stdout: No filesystems configured 2024-04-25T14:29:10.685 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T14:29:10.685 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T14:29:10.685 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T14:29:10.685 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T14:29:10.685 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-25T14:29:10.685 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 bash[27747]: fc4336fa1ec1552dd465a162c199d50f3682c6bbffff41df7bd13f1574ee5f8f 2024-04-25T14:29:10.685 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 systemd[1]: Started Ceph mon.a for fd30e6ce-030f-11ef-bc93-c7b262605968. 2024-04-25T14:29:10.685 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T14:29:10.685 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: monmap epoch 1 2024-04-25T14:29:10.685 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: fsid fd30e6ce-030f-11ef-bc93-c7b262605968 2024-04-25T14:29:10.685 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: last_changed 2024-04-25T14:28:49.679784+0000 2024-04-25T14:29:10.685 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: created 2024-04-25T14:28:49.679784+0000 2024-04-25T14:29:10.686 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: min_mon_release 19 (squid) 2024-04-25T14:29:10.686 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: election_strategy: 1 2024-04-25T14:29:10.686 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: 0: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.a 2024-04-25T14:29:10.686 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: fsmap 2024-04-25T14:29:10.686 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T14:29:10.686 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:10 smithi053 ceph-mon[27773]: mgrmap e1: no daemons active 2024-04-25T14:29:12.194 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.270+0000 7f5522c00700 1 Processor -- start 2024-04-25T14:29:12.199 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.270+0000 7f5522c00700 1 -- start start 2024-04-25T14:29:12.199 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.270+0000 7f5522c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f551c10ef40 0x7f551c10f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:12.199 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.270+0000 7f5522c00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f551c10f930 con 0x7f551c10ef40 2024-04-25T14:29:12.199 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.270+0000 7f5521800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f551c10ef40 0x7f551c10f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:12.199 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.270+0000 7f5521800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f551c10ef40 0x7f551c10f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34978/0 (socket says 172.21.15.53:34978) 2024-04-25T14:29:12.199 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.270+0000 7f5521800700 1 -- 172.21.15.53:0/3393696558 learned_addr learned my addr 172.21.15.53:0/3393696558 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:29:12.199 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.271+0000 7f5521800700 1 -- 172.21.15.53:0/3393696558 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f551c10fab0 con 0x7f551c10ef40 2024-04-25T14:29:12.200 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.271+0000 7f5521800700 1 --2- 172.21.15.53:0/3393696558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f551c10ef40 0x7f551c10f360 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f55100098a0 tx=0x7f5510009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deabc3d442dfcdd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:12.200 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.271+0000 7f551be00700 1 -- 172.21.15.53:0/3393696558 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5510004160 con 0x7f551c10ef40 2024-04-25T14:29:12.200 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.272+0000 7f551be00700 1 -- 172.21.15.53:0/3393696558 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5510004300 con 0x7f551c10ef40 2024-04-25T14:29:12.200 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.272+0000 7f551be00700 1 -- 172.21.15.53:0/3393696558 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5510011af0 con 0x7f551c10ef40 2024-04-25T14:29:12.200 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.272+0000 7f5522c00700 1 -- 172.21.15.53:0/3393696558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f551c10ef40 msgr2=0x7f551c10f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:12.200 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.272+0000 7f5522c00700 1 --2- 172.21.15.53:0/3393696558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f551c10ef40 0x7f551c10f360 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f55100098a0 tx=0x7f5510009bb0 comp rx=0 tx=0).stop 2024-04-25T14:29:12.200 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.272+0000 7f5522c00700 1 -- 172.21.15.53:0/3393696558 shutdown_connections 2024-04-25T14:29:12.200 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.272+0000 7f5522c00700 1 --2- 172.21.15.53:0/3393696558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f551c10ef40 0x7f551c10f360 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:12.200 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.272+0000 7f5522c00700 1 -- 172.21.15.53:0/3393696558 >> 172.21.15.53:0/3393696558 conn(0x7f551c07da00 msgr2=0x7f551c07fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:29:12.200 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.272+0000 7f5522c00700 1 -- 172.21.15.53:0/3393696558 shutdown_connections 2024-04-25T14:29:12.200 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.272+0000 7f5522c00700 1 -- 172.21.15.53:0/3393696558 wait complete. 2024-04-25T14:29:12.200 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.273+0000 7f5522c00700 1 Processor -- start 2024-04-25T14:29:12.200 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.274+0000 7f5522c00700 1 -- start start 2024-04-25T14:29:12.200 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.274+0000 7f5522c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f551c10ef40 0x7f551c1a28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:12.201 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.274+0000 7f5522c00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f551c110ae0 con 0x7f551c10ef40 2024-04-25T14:29:12.201 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.274+0000 7f5521800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f551c10ef40 0x7f551c1a28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:12.201 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.274+0000 7f5521800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f551c10ef40 0x7f551c1a28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34986/0 (socket says 172.21.15.53:34986) 2024-04-25T14:29:12.201 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.274+0000 7f5521800700 1 -- 172.21.15.53:0/3877679955 learned_addr learned my addr 172.21.15.53:0/3877679955 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:29:12.201 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.275+0000 7f5521800700 1 -- 172.21.15.53:0/3877679955 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f551c1a2e30 con 0x7f551c10ef40 2024-04-25T14:29:12.201 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.275+0000 7f5521800700 1 --2- 172.21.15.53:0/3877679955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f551c10ef40 0x7f551c1a28f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5510009fb0 tx=0x7f5510011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:12.201 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.275+0000 7f551a000700 1 -- 172.21.15.53:0/3877679955 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55100036a0 con 0x7f551c10ef40 2024-04-25T14:29:12.201 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.275+0000 7f551a000700 1 -- 172.21.15.53:0/3877679955 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f551001a430 con 0x7f551c10ef40 2024-04-25T14:29:12.201 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.275+0000 7f551a000700 1 -- 172.21.15.53:0/3877679955 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5510019550 con 0x7f551c10ef40 2024-04-25T14:29:12.201 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.275+0000 7f5522c00700 1 -- 172.21.15.53:0/3877679955 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f551c1a30c0 con 0x7f551c10ef40 2024-04-25T14:29:12.201 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.275+0000 7f5522c00700 1 -- 172.21.15.53:0/3877679955 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f551c1a5db0 con 0x7f551c10ef40 2024-04-25T14:29:12.201 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.276+0000 7f551a000700 1 -- 172.21.15.53:0/3877679955 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5510019be0 con 0x7f551c10ef40 2024-04-25T14:29:12.201 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.276+0000 7f551a000700 1 -- 172.21.15.53:0/3877679955 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f551002dd50 con 0x7f551c10ef40 2024-04-25T14:29:12.201 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.276+0000 7f5522c00700 1 -- 172.21.15.53:0/3877679955 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f551c069ba0 con 0x7f551c10ef40 2024-04-25T14:29:12.201 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.280+0000 7f551a000700 1 -- 172.21.15.53:0/3877679955 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f5510003b80 con 0x7f551c10ef40 2024-04-25T14:29:12.202 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.357+0000 7f5522c00700 1 -- 172.21.15.53:0/3877679955 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f551c1a5f80 con 0x7f551c10ef40 2024-04-25T14:29:12.202 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.362+0000 7f551a000700 1 -- 172.21.15.53:0/3877679955 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f5510032030 con 0x7f551c10ef40 2024-04-25T14:29:12.202 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.363+0000 7f5522c00700 1 -- 172.21.15.53:0/3877679955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f551c10ef40 msgr2=0x7f551c1a28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:12.202 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.363+0000 7f5522c00700 1 --2- 172.21.15.53:0/3877679955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f551c10ef40 0x7f551c1a28f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5510009fb0 tx=0x7f5510011940 comp rx=0 tx=0).stop 2024-04-25T14:29:12.202 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.363+0000 7f5522c00700 1 -- 172.21.15.53:0/3877679955 shutdown_connections 2024-04-25T14:29:12.202 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.363+0000 7f5522c00700 1 --2- 172.21.15.53:0/3877679955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f551c10ef40 0x7f551c1a28f0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:12.202 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.363+0000 7f5522c00700 1 -- 172.21.15.53:0/3877679955 >> 172.21.15.53:0/3877679955 conn(0x7f551c07da00 msgr2=0x7f551c07e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:29:12.202 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.363+0000 7f5522c00700 1 -- 172.21.15.53:0/3877679955 shutdown_connections 2024-04-25T14:29:12.202 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:11.363+0000 7f5522c00700 1 -- 172.21.15.53:0/3877679955 wait complete. 2024-04-25T14:29:12.202 INFO:teuthology.orchestra.run.smithi053.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-25T14:29:12.235 INFO:teuthology.orchestra.run.smithi053.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-25T14:29:12.235 INFO:teuthology.orchestra.run.smithi053.stdout:Creating mgr... 2024-04-25T14:29:12.236 INFO:teuthology.orchestra.run.smithi053.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-25T14:29:12.237 INFO:teuthology.orchestra.run.smithi053.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-25T14:29:12.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:12 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/3877679955' entity='client.admin' 2024-04-25T14:29:12.723 INFO:teuthology.orchestra.run.smithi053.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mgr.x 2024-04-25T14:29:12.723 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stderr Failed to reset failed state of unit ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-25T14:29:13.012 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-fd30e6ce-030f-11ef-bc93-c7b262605968.target.wants/ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@.service. 2024-04-25T14:29:13.418 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:13 smithi053 systemd[1]: Starting Ceph mgr.x for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:29:13.683 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:13 smithi053 podman[27990]: 2024-04-25 14:29:13.323728407 +0000 UTC m=+0.039739275 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T14:29:13.684 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:13 smithi053 podman[27990]: 2024-04-25 14:29:13.487405268 +0000 UTC m=+0.203416168 container create a15886d6773de773af9639b0035bf7487d6de774c4071c5a178d9b6df94bf2da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T14:29:14.272 INFO:teuthology.orchestra.run.smithi053.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T14:29:14.272 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stdout disabled 2024-04-25T14:29:14.291 INFO:teuthology.orchestra.run.smithi053.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T14:29:14.291 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stdout inactive 2024-04-25T14:29:14.291 INFO:teuthology.orchestra.run.smithi053.stdout:firewalld.service is not enabled 2024-04-25T14:29:14.292 INFO:teuthology.orchestra.run.smithi053.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T14:29:14.318 INFO:teuthology.orchestra.run.smithi053.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T14:29:14.318 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stdout disabled 2024-04-25T14:29:14.341 INFO:teuthology.orchestra.run.smithi053.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T14:29:14.342 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stdout inactive 2024-04-25T14:29:14.342 INFO:teuthology.orchestra.run.smithi053.stdout:firewalld.service is not enabled 2024-04-25T14:29:14.342 INFO:teuthology.orchestra.run.smithi053.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-25T14:29:14.342 INFO:teuthology.orchestra.run.smithi053.stdout:Waiting for mgr to start... 2024-04-25T14:29:14.342 INFO:teuthology.orchestra.run.smithi053.stdout:Waiting for mgr... 2024-04-25T14:29:14.372 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:14 smithi053 podman[27990]: 2024-04-25 14:29:14.116424997 +0000 UTC m=+0.832435908 container init a15886d6773de773af9639b0035bf7487d6de774c4071c5a178d9b6df94bf2da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-25T14:29:14.372 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:14 smithi053 podman[27990]: 2024-04-25 14:29:14.127311129 +0000 UTC m=+0.843322033 container start a15886d6773de773af9639b0035bf7487d6de774c4071c5a178d9b6df94bf2da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:29:14.372 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:14 smithi053 bash[27990]: a15886d6773de773af9639b0035bf7487d6de774c4071c5a178d9b6df94bf2da 2024-04-25T14:29:14.372 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:14 smithi053 systemd[1]: Started Ceph mgr.x for fd30e6ce-030f-11ef-bc93-c7b262605968. 2024-04-25T14:29:14.644 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:14 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:14.371+0000 7fd34611a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:29:14.644 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:14 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:14.643+0000 7fd34611a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:29:14.943 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:14 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:14.783+0000 7fd34611a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:29:15.208 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:15 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/871133693' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T14:29:15.532 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:15 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:15.208+0000 7fd34611a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:29:15.532 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:15 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:15.324+0000 7fd34611a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:29:15.981 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 2024-04-25T14:29:15.981 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout { 2024-04-25T14:29:15.981 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "fsid": "fd30e6ce-030f-11ef-bc93-c7b262605968", 2024-04-25T14:29:15.981 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T14:29:15.982 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T14:29:15.982 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T14:29:15.982 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T14:29:15.982 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:29:15.982 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T14:29:15.982 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T14:29:15.982 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 0 2024-04-25T14:29:15.982 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:29:15.982 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T14:29:15.983 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T14:29:15.983 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:29:15.983 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "quorum_age": 4, 2024-04-25T14:29:15.983 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T14:29:15.983 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:29:15.983 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T14:29:15.983 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T14:29:15.983 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:29:15.983 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T14:29:15.983 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:29:15.983 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T14:29:15.983 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T14:29:15.983 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T14:29:15.983 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T14:29:15.984 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T14:29:15.984 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T14:29:15.984 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:29:15.984 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T14:29:15.984 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T14:29:15.984 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T14:29:15.984 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T14:29:15.984 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T14:29:15.984 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T14:29:15.984 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T14:29:15.984 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T14:29:15.984 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T14:29:15.985 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:29:15.985 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T14:29:15.985 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:29:15.985 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T14:29:15.985 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T14:29:15.985 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:29:15.986 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T14:29:15.986 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T14:29:15.986 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T14:29:15.986 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T14:29:15.986 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T14:29:15.986 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T14:29:15.987 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T14:29:15.987 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:29:15.987 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T14:29:15.987 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:29:15.987 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T14:29:15.987 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:29:15.987 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T14:28:54.330939+0000", 2024-04-25T14:29:15.987 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T14:29:15.988 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:29:15.988 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T14:29:15.988 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout } 2024-04-25T14:29:15.988 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.002+0000 7f3688800700 1 Processor -- start 2024-04-25T14:29:15.988 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.002+0000 7f3688800700 1 -- start start 2024-04-25T14:29:15.988 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.002+0000 7f3688800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3680107440 0x7f3680107860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:15.988 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.002+0000 7f3688800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3680107e30 con 0x7f3680107440 2024-04-25T14:29:15.988 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.003+0000 7f3685e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3680107440 0x7f3680107860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:15.988 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.003+0000 7f3685e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3680107440 0x7f3680107860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55554/0 (socket says 172.21.15.53:55554) 2024-04-25T14:29:15.988 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.003+0000 7f3685e00700 1 -- 172.21.15.53:0/621233774 learned_addr learned my addr 172.21.15.53:0/621233774 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:29:15.989 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.003+0000 7f3685e00700 1 -- 172.21.15.53:0/621233774 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3680108640 con 0x7f3680107440 2024-04-25T14:29:15.989 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.004+0000 7f3685e00700 1 --2- 172.21.15.53:0/621233774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3680107440 0x7f3680107860 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f36740098a0 tx=0x7f3674009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a620f2b0e1d2935 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:15.989 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.004+0000 7f3684a00700 1 -- 172.21.15.53:0/621233774 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3674004160 con 0x7f3680107440 2024-04-25T14:29:15.989 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.004+0000 7f3684a00700 1 -- 172.21.15.53:0/621233774 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3674004300 con 0x7f3680107440 2024-04-25T14:29:15.989 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.004+0000 7f3684a00700 1 -- 172.21.15.53:0/621233774 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3674011af0 con 0x7f3680107440 2024-04-25T14:29:15.989 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.005+0000 7f3688800700 1 -- 172.21.15.53:0/621233774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3680107440 msgr2=0x7f3680107860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:15.989 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.005+0000 7f3688800700 1 --2- 172.21.15.53:0/621233774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3680107440 0x7f3680107860 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f36740098a0 tx=0x7f3674009bb0 comp rx=0 tx=0).stop 2024-04-25T14:29:15.989 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.005+0000 7f3688800700 1 -- 172.21.15.53:0/621233774 shutdown_connections 2024-04-25T14:29:15.989 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.005+0000 7f3688800700 1 --2- 172.21.15.53:0/621233774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3680107440 0x7f3680107860 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:15.989 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.005+0000 7f3688800700 1 -- 172.21.15.53:0/621233774 >> 172.21.15.53:0/621233774 conn(0x7f3680102a40 msgr2=0x7f3680104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:29:15.989 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.005+0000 7f3688800700 1 -- 172.21.15.53:0/621233774 shutdown_connections 2024-04-25T14:29:15.989 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.005+0000 7f3688800700 1 -- 172.21.15.53:0/621233774 wait complete. 2024-04-25T14:29:15.990 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.006+0000 7f3688800700 1 Processor -- start 2024-04-25T14:29:15.990 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.006+0000 7f3688800700 1 -- start start 2024-04-25T14:29:15.990 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.007+0000 7f3688800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3680107440 0x7f368019c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:15.990 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.007+0000 7f3688800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f368010b9f0 con 0x7f3680107440 2024-04-25T14:29:15.990 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.007+0000 7f3685e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3680107440 0x7f368019c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:15.990 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.007+0000 7f3685e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3680107440 0x7f368019c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55564/0 (socket says 172.21.15.53:55564) 2024-04-25T14:29:15.990 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.007+0000 7f3685e00700 1 -- 172.21.15.53:0/871133693 learned_addr learned my addr 172.21.15.53:0/871133693 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:29:15.990 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.007+0000 7f3685e00700 1 -- 172.21.15.53:0/871133693 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f368019ca10 con 0x7f3680107440 2024-04-25T14:29:15.990 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.008+0000 7f3685e00700 1 --2- 172.21.15.53:0/871133693 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3680107440 0x7f368019c4d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3674000c00 tx=0x7f3674011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:15.990 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.008+0000 7f367aa00700 1 -- 172.21.15.53:0/871133693 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3674003b20 con 0x7f3680107440 2024-04-25T14:29:15.990 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.008+0000 7f367aa00700 1 -- 172.21.15.53:0/871133693 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f367401a8c0 con 0x7f3680107440 2024-04-25T14:29:15.990 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.008+0000 7f367aa00700 1 -- 172.21.15.53:0/871133693 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3674024940 con 0x7f3680107440 2024-04-25T14:29:15.991 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.008+0000 7f3688800700 1 -- 172.21.15.53:0/871133693 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f368019cca0 con 0x7f3680107440 2024-04-25T14:29:15.991 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.008+0000 7f3688800700 1 -- 172.21.15.53:0/871133693 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f368019f990 con 0x7f3680107440 2024-04-25T14:29:15.991 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.009+0000 7f367aa00700 1 -- 172.21.15.53:0/871133693 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3674022020 con 0x7f3680107440 2024-04-25T14:29:15.991 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.009+0000 7f367aa00700 1 -- 172.21.15.53:0/871133693 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f36740193c0 con 0x7f3680107440 2024-04-25T14:29:15.991 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.009+0000 7f3688800700 1 -- 172.21.15.53:0/871133693 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36800640a0 con 0x7f3680107440 2024-04-25T14:29:15.991 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.014+0000 7f367aa00700 1 -- 172.21.15.53:0/871133693 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f367402dde0 con 0x7f3680107440 2024-04-25T14:29:15.991 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.097+0000 7f3688800700 1 -- 172.21.15.53:0/871133693 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f368002e180 con 0x7f3680107440 2024-04-25T14:29:15.992 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.098+0000 7f367aa00700 1 -- 172.21.15.53:0/871133693 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f367401aa70 con 0x7f3680107440 2024-04-25T14:29:15.992 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.099+0000 7f3688800700 1 -- 172.21.15.53:0/871133693 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3680107440 msgr2=0x7f368019c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:15.992 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.099+0000 7f3688800700 1 --2- 172.21.15.53:0/871133693 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3680107440 0x7f368019c4d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3674000c00 tx=0x7f3674011b70 comp rx=0 tx=0).stop 2024-04-25T14:29:15.992 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.099+0000 7f3688800700 1 -- 172.21.15.53:0/871133693 shutdown_connections 2024-04-25T14:29:15.992 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.099+0000 7f3688800700 1 --2- 172.21.15.53:0/871133693 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3680107440 0x7f368019c4d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:15.992 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.099+0000 7f3688800700 1 -- 172.21.15.53:0/871133693 >> 172.21.15.53:0/871133693 conn(0x7f3680102a40 msgr2=0x7f36801038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:29:15.993 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.100+0000 7f3688800700 1 -- 172.21.15.53:0/871133693 shutdown_connections 2024-04-25T14:29:15.993 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:15.100+0000 7f3688800700 1 -- 172.21.15.53:0/871133693 wait complete. 2024-04-25T14:29:15.993 INFO:teuthology.orchestra.run.smithi053.stdout:mgr not available, waiting (1/15)... 2024-04-25T14:29:17.282 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:16 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:16.964+0000 7fd34611a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:29:17.282 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:17 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:17.195+0000 7fd34611a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:29:17.782 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:17 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:17.413+0000 7fd34611a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:29:18.133 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:17 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:17.799+0000 7fd34611a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:29:18.133 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:17 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:17.924+0000 7fd34611a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:29:18.427 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:18 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:18.190+0000 7fd34611a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:29:18.677 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:18 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:18.453+0000 7fd34611a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:29:19.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:18 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/762332504' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T14:29:19.343 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:19 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:19.269+0000 7fd34611a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T14:29:19.355 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 2024-04-25T14:29:19.355 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout { 2024-04-25T14:29:19.355 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "fsid": "fd30e6ce-030f-11ef-bc93-c7b262605968", 2024-04-25T14:29:19.355 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T14:29:19.355 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T14:29:19.355 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T14:29:19.355 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T14:29:19.356 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:29:19.356 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T14:29:19.356 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T14:29:19.356 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 0 2024-04-25T14:29:19.356 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:29:19.356 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T14:29:19.356 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T14:29:19.356 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:29:19.356 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-04-25T14:29:19.356 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T14:29:19.356 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:29:19.357 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T14:29:19.357 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T14:29:19.357 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:29:19.357 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T14:29:19.357 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:29:19.357 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T14:29:19.357 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T14:29:19.366 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T14:29:19.366 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T14:29:19.366 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T14:29:19.366 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T14:29:19.366 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:29:19.367 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T14:29:19.367 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T14:29:19.367 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T14:29:19.367 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T14:29:19.367 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T14:29:19.367 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T14:29:19.367 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T14:29:19.367 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T14:29:19.367 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T14:29:19.367 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:29:19.367 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T14:29:19.367 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:29:19.367 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T14:29:19.367 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T14:29:19.368 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:29:19.368 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T14:29:19.368 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T14:29:19.368 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T14:29:19.368 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T14:29:19.368 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T14:29:19.368 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T14:29:19.368 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T14:29:19.368 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:29:19.368 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T14:29:19.368 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:29:19.369 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T14:29:19.369 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:29:19.369 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T14:28:54.330939+0000", 2024-04-25T14:29:19.369 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T14:29:19.369 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:29:19.369 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T14:29:19.369 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout } 2024-04-25T14:29:19.369 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.582+0000 7f2bb6a00700 1 Processor -- start 2024-04-25T14:29:19.370 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.582+0000 7f2bb6a00700 1 -- start start 2024-04-25T14:29:19.370 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.583+0000 7f2bb6a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bb0108ff0 0x7f2bb010b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:19.370 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.583+0000 7f2bb6a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb0063d20 con 0x7f2bb0108ff0 2024-04-25T14:29:19.370 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.583+0000 7f2bafe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bb0108ff0 0x7f2bb010b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:19.370 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.583+0000 7f2bafe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bb0108ff0 0x7f2bb010b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55568/0 (socket says 172.21.15.53:55568) 2024-04-25T14:29:19.370 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.583+0000 7f2bafe00700 1 -- 172.21.15.53:0/1996070301 learned_addr learned my addr 172.21.15.53:0/1996070301 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:29:19.371 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.584+0000 7f2bafe00700 1 -- 172.21.15.53:0/1996070301 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bb010b920 con 0x7f2bb0108ff0 2024-04-25T14:29:19.371 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.584+0000 7f2bafe00700 1 --2- 172.21.15.53:0/1996070301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bb0108ff0 0x7f2bb010b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2ba00098a0 tx=0x7f2ba0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4278bb7f7ed9bc23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:19.371 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.584+0000 7f2baf400700 1 -- 172.21.15.53:0/1996070301 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ba0004160 con 0x7f2bb0108ff0 2024-04-25T14:29:19.371 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.584+0000 7f2baf400700 1 -- 172.21.15.53:0/1996070301 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2ba0004300 con 0x7f2bb0108ff0 2024-04-25T14:29:19.371 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.584+0000 7f2baf400700 1 -- 172.21.15.53:0/1996070301 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ba0011af0 con 0x7f2bb0108ff0 2024-04-25T14:29:19.371 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.585+0000 7f2bb6a00700 1 -- 172.21.15.53:0/1996070301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bb0108ff0 msgr2=0x7f2bb010b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:19.371 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.585+0000 7f2bb6a00700 1 --2- 172.21.15.53:0/1996070301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bb0108ff0 0x7f2bb010b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2ba00098a0 tx=0x7f2ba0009bb0 comp rx=0 tx=0).stop 2024-04-25T14:29:19.371 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.585+0000 7f2bb6a00700 1 -- 172.21.15.53:0/1996070301 shutdown_connections 2024-04-25T14:29:19.371 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.585+0000 7f2bb6a00700 1 --2- 172.21.15.53:0/1996070301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bb0108ff0 0x7f2bb010b3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:19.371 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.585+0000 7f2bb6a00700 1 -- 172.21.15.53:0/1996070301 >> 172.21.15.53:0/1996070301 conn(0x7f2bb0102a40 msgr2=0x7f2bb0104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:29:19.372 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.585+0000 7f2bb6a00700 1 -- 172.21.15.53:0/1996070301 shutdown_connections 2024-04-25T14:29:19.372 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.586+0000 7f2bb6a00700 1 -- 172.21.15.53:0/1996070301 wait complete. 2024-04-25T14:29:19.372 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.587+0000 7f2bb6a00700 1 Processor -- start 2024-04-25T14:29:19.372 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.587+0000 7f2bb6a00700 1 -- start start 2024-04-25T14:29:19.372 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.587+0000 7f2bb6a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bb0108ff0 0x7f2bb019c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:19.372 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.587+0000 7f2bb6a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb010c8e0 con 0x7f2bb0108ff0 2024-04-25T14:29:19.372 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.588+0000 7f2bafe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bb0108ff0 0x7f2bb019c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:19.372 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.588+0000 7f2bafe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bb0108ff0 0x7f2bb019c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55570/0 (socket says 172.21.15.53:55570) 2024-04-25T14:29:19.372 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.588+0000 7f2bafe00700 1 -- 172.21.15.53:0/762332504 learned_addr learned my addr 172.21.15.53:0/762332504 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:29:19.372 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.588+0000 7f2bafe00700 1 -- 172.21.15.53:0/762332504 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bb019c970 con 0x7f2bb0108ff0 2024-04-25T14:29:19.372 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.588+0000 7f2bafe00700 1 --2- 172.21.15.53:0/762332504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bb0108ff0 0x7f2bb019c430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0003f40 tx=0x7f2ba0011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:19.372 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.589+0000 7f2bad600700 1 -- 172.21.15.53:0/762332504 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ba00038b0 con 0x7f2bb0108ff0 2024-04-25T14:29:19.372 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.589+0000 7f2bb6a00700 1 -- 172.21.15.53:0/762332504 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bb019cc00 con 0x7f2bb0108ff0 2024-04-25T14:29:19.372 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.589+0000 7f2bad600700 1 -- 172.21.15.53:0/762332504 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2ba0003a50 con 0x7f2bb0108ff0 2024-04-25T14:29:19.372 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.589+0000 7f2bad600700 1 -- 172.21.15.53:0/762332504 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ba001a5c0 con 0x7f2bb0108ff0 2024-04-25T14:29:19.373 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.589+0000 7f2bb6a00700 1 -- 172.21.15.53:0/762332504 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bb019f8f0 con 0x7f2bb0108ff0 2024-04-25T14:29:19.373 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.589+0000 7f2bad600700 1 -- 172.21.15.53:0/762332504 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2ba001ac50 con 0x7f2bb0108ff0 2024-04-25T14:29:19.373 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.589+0000 7f2bad600700 1 -- 172.21.15.53:0/762332504 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2ba0019700 con 0x7f2bb0108ff0 2024-04-25T14:29:19.373 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.590+0000 7f2bb6a00700 1 -- 172.21.15.53:0/762332504 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bb00640a0 con 0x7f2bb0108ff0 2024-04-25T14:29:19.373 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.593+0000 7f2bad600700 1 -- 172.21.15.53:0/762332504 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f2ba0019900 con 0x7f2bb0108ff0 2024-04-25T14:29:19.373 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.673+0000 7f2bb6a00700 1 -- 172.21.15.53:0/762332504 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f2bb019fe30 con 0x7f2bb0108ff0 2024-04-25T14:29:19.373 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.674+0000 7f2bad600700 1 -- 172.21.15.53:0/762332504 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f2ba0019ae0 con 0x7f2bb0108ff0 2024-04-25T14:29:19.373 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.675+0000 7f2bb6a00700 1 -- 172.21.15.53:0/762332504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bb0108ff0 msgr2=0x7f2bb019c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:19.373 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.675+0000 7f2bb6a00700 1 --2- 172.21.15.53:0/762332504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bb0108ff0 0x7f2bb019c430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0003f40 tx=0x7f2ba0011890 comp rx=0 tx=0).stop 2024-04-25T14:29:19.373 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.675+0000 7f2bb6a00700 1 -- 172.21.15.53:0/762332504 shutdown_connections 2024-04-25T14:29:19.373 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.675+0000 7f2bb6a00700 1 --2- 172.21.15.53:0/762332504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bb0108ff0 0x7f2bb019c430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:19.373 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.675+0000 7f2bb6a00700 1 -- 172.21.15.53:0/762332504 >> 172.21.15.53:0/762332504 conn(0x7f2bb0102a40 msgr2=0x7f2bb0103530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:29:19.373 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.675+0000 7f2bb6a00700 1 -- 172.21.15.53:0/762332504 shutdown_connections 2024-04-25T14:29:19.373 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:18.675+0000 7f2bb6a00700 1 -- 172.21.15.53:0/762332504 wait complete. 2024-04-25T14:29:19.374 INFO:teuthology.orchestra.run.smithi053.stdout:mgr not available, waiting (2/15)... 2024-04-25T14:29:20.232 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:19 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:19.836+0000 7fd34611a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:29:20.232 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:19 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:19.964+0000 7fd34611a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:29:20.532 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:20 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:20.232+0000 7fd34611a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T14:29:21.282 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:20 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:20.894+0000 7fd34611a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:29:21.282 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:21 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:21.020+0000 7fd34611a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:29:21.663 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:21 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:21.345+0000 7fd34611a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:29:21.663 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:21 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:21.528+0000 7fd34611a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:29:21.663 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:21 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:21.663+0000 7fd34611a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:29:21.929 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:21 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:21.818+0000 7fd34611a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:29:22.200 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:22 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/3872359585' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T14:29:22.532 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:22 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:22.200+0000 7fd34611a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:29:22.884 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 2024-04-25T14:29:22.884 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout { 2024-04-25T14:29:22.884 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "fsid": "fd30e6ce-030f-11ef-bc93-c7b262605968", 2024-04-25T14:29:22.884 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T14:29:22.884 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T14:29:22.884 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T14:29:22.885 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T14:29:22.885 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:29:22.885 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T14:29:22.885 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T14:29:22.885 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 0 2024-04-25T14:29:22.885 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:29:22.885 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T14:29:22.885 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T14:29:22.885 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:29:22.885 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-04-25T14:29:22.885 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T14:29:22.885 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:29:22.885 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T14:29:22.885 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T14:29:22.886 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:29:22.886 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T14:29:22.886 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:29:22.886 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T14:29:22.886 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T14:29:22.886 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T14:29:22.886 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T14:29:22.886 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T14:29:22.886 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T14:29:22.886 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:29:22.886 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T14:29:22.886 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T14:29:22.886 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T14:29:22.886 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T14:29:22.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T14:29:22.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T14:29:22.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T14:29:22.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T14:29:22.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T14:29:22.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:29:22.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T14:29:22.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:29:22.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T14:29:22.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T14:29:22.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:29:22.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T14:29:22.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T14:29:22.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T14:29:22.888 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T14:29:22.888 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T14:29:22.888 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T14:29:22.888 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T14:29:22.888 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:29:22.888 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T14:29:22.888 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:29:22.888 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T14:29:22.888 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:29:22.888 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T14:28:54.330939+0000", 2024-04-25T14:29:22.888 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T14:29:22.888 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:29:22.888 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T14:29:22.888 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout } 2024-04-25T14:29:22.888 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:21.996+0000 7f8fd4e00700 1 Processor -- start 2024-04-25T14:29:22.889 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:21.996+0000 7f8fd4e00700 1 -- start start 2024-04-25T14:29:22.889 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:21.996+0000 7f8fd4e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fd0106dc0 0x7f8fd01071e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:22.889 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:21.996+0000 7f8fd4e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd0063d40 con 0x7f8fd0106dc0 2024-04-25T14:29:22.889 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:21.997+0000 7f8fcde00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fd0106dc0 0x7f8fd01071e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:22.889 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:21.997+0000 7f8fcde00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fd0106dc0 0x7f8fd01071e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55576/0 (socket says 172.21.15.53:55576) 2024-04-25T14:29:22.889 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:21.997+0000 7f8fcde00700 1 -- 172.21.15.53:0/4185000799 learned_addr learned my addr 172.21.15.53:0/4185000799 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:29:22.889 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:21.998+0000 7f8fcde00700 1 -- 172.21.15.53:0/4185000799 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fd0107720 con 0x7f8fd0106dc0 2024-04-25T14:29:22.889 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:21.998+0000 7f8fcde00700 1 --2- 172.21.15.53:0/4185000799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fd0106dc0 0x7f8fd01071e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0009e80 tx=0x7f8fc000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d37c7389511c2bab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:22.889 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:21.998+0000 7f8fcca00700 1 -- 172.21.15.53:0/4185000799 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fc0017070 con 0x7f8fd0106dc0 2024-04-25T14:29:22.889 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:21.998+0000 7f8fcca00700 1 -- 172.21.15.53:0/4185000799 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8fc000b8c0 con 0x7f8fd0106dc0 2024-04-25T14:29:22.889 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:21.998+0000 7f8fcca00700 1 -- 172.21.15.53:0/4185000799 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fc0004490 con 0x7f8fd0106dc0 2024-04-25T14:29:22.889 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:21.999+0000 7f8fd4e00700 1 -- 172.21.15.53:0/4185000799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fd0106dc0 msgr2=0x7f8fd01071e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:22.889 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:21.999+0000 7f8fd4e00700 1 --2- 172.21.15.53:0/4185000799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fd0106dc0 0x7f8fd01071e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0009e80 tx=0x7f8fc000b330 comp rx=0 tx=0).stop 2024-04-25T14:29:22.889 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:21.999+0000 7f8fd4e00700 1 -- 172.21.15.53:0/4185000799 shutdown_connections 2024-04-25T14:29:22.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:21.999+0000 7f8fd4e00700 1 --2- 172.21.15.53:0/4185000799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fd0106dc0 0x7f8fd01071e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:22.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:21.999+0000 7f8fd4e00700 1 -- 172.21.15.53:0/4185000799 >> 172.21.15.53:0/4185000799 conn(0x7f8fd0102a40 msgr2=0x7f8fd0104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:29:22.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:21.999+0000 7f8fd4e00700 1 -- 172.21.15.53:0/4185000799 shutdown_connections 2024-04-25T14:29:22.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:21.999+0000 7f8fd4e00700 1 -- 172.21.15.53:0/4185000799 wait complete. 2024-04-25T14:29:22.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:22.000+0000 7f8fd4e00700 1 Processor -- start 2024-04-25T14:29:22.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:22.000+0000 7f8fd4e00700 1 -- start start 2024-04-25T14:29:22.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:22.001+0000 7f8fd4e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fd0106dc0 0x7f8fd019e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:22.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:22.001+0000 7f8fd4e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd0108750 con 0x7f8fd0106dc0 2024-04-25T14:29:22.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:22.001+0000 7f8fcde00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fd0106dc0 0x7f8fd019e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:22.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:22.001+0000 7f8fcde00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fd0106dc0 0x7f8fd019e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55584/0 (socket says 172.21.15.53:55584) 2024-04-25T14:29:22.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:22.001+0000 7f8fcde00700 1 -- 172.21.15.53:0/3872359585 learned_addr learned my addr 172.21.15.53:0/3872359585 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:29:22.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:22.001+0000 7f8fcde00700 1 -- 172.21.15.53:0/3872359585 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fd019eb50 con 0x7f8fd0106dc0 2024-04-25T14:29:22.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:22.002+0000 7f8fcde00700 1 --2- 172.21.15.53:0/3872359585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fd0106dc0 0x7f8fd019e610 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8fc000b7b0 tx=0x7f8fc000be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:22.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:22.002+0000 7f8fc6a00700 1 -- 172.21.15.53:0/3872359585 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fc0020020 con 0x7f8fd0106dc0 2024-04-25T14:29:22.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:22.002+0000 7f8fc6a00700 1 -- 172.21.15.53:0/3872359585 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8fc0003d10 con 0x7f8fd0106dc0 2024-04-25T14:29:22.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:22.002+0000 7f8fc6a00700 1 -- 172.21.15.53:0/3872359585 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fc0017040 con 0x7f8fd0106dc0 2024-04-25T14:29:22.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:22.002+0000 7f8fd4e00700 1 -- 172.21.15.53:0/3872359585 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fd019ede0 con 0x7f8fd0106dc0 2024-04-25T14:29:22.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:22.002+0000 7f8fd4e00700 1 -- 172.21.15.53:0/3872359585 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fd01a1ad0 con 0x7f8fd0106dc0 2024-04-25T14:29:22.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:22.003+0000 7f8fc6a00700 1 -- 172.21.15.53:0/3872359585 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8fc0023020 con 0x7f8fd0106dc0 2024-04-25T14:29:22.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:22.003+0000 7f8fc6a00700 1 -- 172.21.15.53:0/3872359585 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8fc000f050 con 0x7f8fd0106dc0 2024-04-25T14:29:22.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:22.003+0000 7f8fd4e00700 1 -- 172.21.15.53:0/3872359585 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fd00640a0 con 0x7f8fd0106dc0 2024-04-25T14:29:22.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:22.006+0000 7f8fc6a00700 1 -- 172.21.15.53:0/3872359585 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f8fc002a020 con 0x7f8fd0106dc0 2024-04-25T14:29:22.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:22.087+0000 7f8fd4e00700 1 -- 172.21.15.53:0/3872359585 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f8fd002e1f0 con 0x7f8fd0106dc0 2024-04-25T14:29:22.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:22.088+0000 7f8fc6a00700 1 -- 172.21.15.53:0/3872359585 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f8fc0003ec0 con 0x7f8fd0106dc0 2024-04-25T14:29:22.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:22.089+0000 7f8fd4e00700 1 -- 172.21.15.53:0/3872359585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fd0106dc0 msgr2=0x7f8fd019e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:22.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:22.089+0000 7f8fd4e00700 1 --2- 172.21.15.53:0/3872359585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fd0106dc0 0x7f8fd019e610 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8fc000b7b0 tx=0x7f8fc000be90 comp rx=0 tx=0).stop 2024-04-25T14:29:22.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:22.089+0000 7f8fd4e00700 1 -- 172.21.15.53:0/3872359585 shutdown_connections 2024-04-25T14:29:22.892 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:22.089+0000 7f8fd4e00700 1 --2- 172.21.15.53:0/3872359585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fd0106dc0 0x7f8fd019e610 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:22.892 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:22.090+0000 7f8fd4e00700 1 -- 172.21.15.53:0/3872359585 >> 172.21.15.53:0/3872359585 conn(0x7f8fd0102a40 msgr2=0x7f8fd0195230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:29:22.892 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:22.090+0000 7f8fd4e00700 1 -- 172.21.15.53:0/3872359585 shutdown_connections 2024-04-25T14:29:22.892 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:22.090+0000 7f8fd4e00700 1 -- 172.21.15.53:0/3872359585 wait complete. 2024-04-25T14:29:22.892 INFO:teuthology.orchestra.run.smithi053.stdout:mgr not available, waiting (3/15)... 2024-04-25T14:29:23.277 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:22 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:22.987+0000 7fd34611a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:29:23.532 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:23 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:23.276+0000 7fd34611a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:29:23.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:23 smithi053 ceph-mon[27773]: Activating manager daemon x 2024-04-25T14:29:23.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:23 smithi053 ceph-mon[27773]: mgrmap e2: x(active, starting, since 0.00853279s) 2024-04-25T14:29:23.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:23 smithi053 ceph-mon[27773]: from='mgr.14100 172.21.15.53:0/1746053579' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T14:29:23.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:23 smithi053 ceph-mon[27773]: from='mgr.14100 172.21.15.53:0/1746053579' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T14:29:23.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:23 smithi053 ceph-mon[27773]: from='mgr.14100 172.21.15.53:0/1746053579' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T14:29:23.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:23 smithi053 ceph-mon[27773]: from='mgr.14100 172.21.15.53:0/1746053579' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:29:23.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:23 smithi053 ceph-mon[27773]: from='mgr.14100 172.21.15.53:0/1746053579' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T14:29:23.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:23 smithi053 ceph-mon[27773]: Manager daemon x is now available 2024-04-25T14:29:23.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:23 smithi053 ceph-mon[27773]: from='mgr.14100 172.21.15.53:0/1746053579' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T14:29:23.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:23 smithi053 ceph-mon[27773]: from='mgr.14100 172.21.15.53:0/1746053579' entity='mgr.x' 2024-04-25T14:29:23.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:23 smithi053 ceph-mon[27773]: from='mgr.14100 172.21.15.53:0/1746053579' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T14:29:23.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:23 smithi053 ceph-mon[27773]: from='mgr.14100 172.21.15.53:0/1746053579' entity='mgr.x' 2024-04-25T14:29:24.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:24 smithi053 ceph-mon[27773]: from='mgr.14100 172.21.15.53:0/1746053579' entity='mgr.x' 2024-04-25T14:29:24.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:24 smithi053 ceph-mon[27773]: mgrmap e3: x(active, since 1.0201s) 2024-04-25T14:29:26.455 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:26 smithi053 ceph-mon[27773]: mgrmap e4: x(active, since 2s) 2024-04-25T14:29:26.455 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:26 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1318030554' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T14:29:26.469 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 2024-04-25T14:29:26.469 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout { 2024-04-25T14:29:26.469 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "fsid": "fd30e6ce-030f-11ef-bc93-c7b262605968", 2024-04-25T14:29:26.469 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T14:29:26.469 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T14:29:26.469 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T14:29:26.469 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T14:29:26.469 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:29:26.469 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T14:29:26.469 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T14:29:26.469 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 0 2024-04-25T14:29:26.470 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:29:26.470 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T14:29:26.470 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T14:29:26.470 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:29:26.470 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2024-04-25T14:29:26.470 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T14:29:26.470 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:29:26.470 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T14:29:26.470 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T14:29:26.470 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:29:26.470 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T14:29:26.470 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:29:26.470 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T14:29:26.471 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T14:29:26.479 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T14:29:26.480 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T14:29:26.480 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T14:29:26.480 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T14:29:26.480 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:29:26.480 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T14:29:26.480 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T14:29:26.480 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T14:29:26.480 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T14:29:26.480 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T14:29:26.480 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T14:29:26.480 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T14:29:26.480 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T14:29:26.480 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T14:29:26.481 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:29:26.481 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T14:29:26.481 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:29:26.481 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T14:29:26.481 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T14:29:26.481 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:29:26.482 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T14:29:26.482 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T14:29:26.482 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T14:29:26.482 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T14:29:26.482 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T14:29:26.482 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T14:29:26.482 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T14:29:26.482 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:29:26.482 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T14:29:26.482 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:29:26.482 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T14:29:26.482 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:29:26.482 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T14:28:54.330939+0000", 2024-04-25T14:29:26.482 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T14:29:26.483 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:29:26.483 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T14:29:26.483 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout } 2024-04-25T14:29:26.483 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.497+0000 7f1daa400700 1 Processor -- start 2024-04-25T14:29:26.483 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.498+0000 7f1daa400700 1 -- start start 2024-04-25T14:29:26.483 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.498+0000 7f1daa400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1da407dea0 0x7f1da407c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:26.483 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.498+0000 7f1daa400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1da407c920 con 0x7f1da407dea0 2024-04-25T14:29:26.483 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.499+0000 7f1da3400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1da407dea0 0x7f1da407c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:26.483 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.499+0000 7f1da3400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1da407dea0 0x7f1da407c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46560/0 (socket says 172.21.15.53:46560) 2024-04-25T14:29:26.483 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.499+0000 7f1da3400700 1 -- 172.21.15.53:0/1954104855 learned_addr learned my addr 172.21.15.53:0/1954104855 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:29:26.483 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.499+0000 7f1da3400700 1 -- 172.21.15.53:0/1954104855 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da407caa0 con 0x7f1da407dea0 2024-04-25T14:29:26.483 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.500+0000 7f1da3400700 1 --2- 172.21.15.53:0/1954104855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1da407dea0 0x7f1da407c350 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1d940098a0 tx=0x7f1d94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2b951cbbe95e25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:26.483 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.500+0000 7f1da2000700 1 -- 172.21.15.53:0/1954104855 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d94004160 con 0x7f1da407dea0 2024-04-25T14:29:26.483 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.500+0000 7f1da2000700 1 -- 172.21.15.53:0/1954104855 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1d94004300 con 0x7f1da407dea0 2024-04-25T14:29:26.484 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.500+0000 7f1da2000700 1 -- 172.21.15.53:0/1954104855 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d94011af0 con 0x7f1da407dea0 2024-04-25T14:29:26.484 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.500+0000 7f1daa400700 1 -- 172.21.15.53:0/1954104855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1da407dea0 msgr2=0x7f1da407c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:26.484 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.500+0000 7f1daa400700 1 --2- 172.21.15.53:0/1954104855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1da407dea0 0x7f1da407c350 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1d940098a0 tx=0x7f1d94009bb0 comp rx=0 tx=0).stop 2024-04-25T14:29:26.484 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.501+0000 7f1daa400700 1 -- 172.21.15.53:0/1954104855 shutdown_connections 2024-04-25T14:29:26.484 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.501+0000 7f1daa400700 1 --2- 172.21.15.53:0/1954104855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1da407dea0 0x7f1da407c350 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:26.484 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.501+0000 7f1daa400700 1 -- 172.21.15.53:0/1954104855 >> 172.21.15.53:0/1954104855 conn(0x7f1da4103b10 msgr2=0x7f1da4105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:29:26.484 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.501+0000 7f1daa400700 1 -- 172.21.15.53:0/1954104855 shutdown_connections 2024-04-25T14:29:26.484 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.501+0000 7f1daa400700 1 -- 172.21.15.53:0/1954104855 wait complete. 2024-04-25T14:29:26.484 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.502+0000 7f1daa400700 1 Processor -- start 2024-04-25T14:29:26.484 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.502+0000 7f1daa400700 1 -- start start 2024-04-25T14:29:26.484 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.502+0000 7f1daa400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1da407dea0 0x7f1da41a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:26.484 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.502+0000 7f1daa400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1da410a6b0 con 0x7f1da407dea0 2024-04-25T14:29:26.485 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.503+0000 7f1da3400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1da407dea0 0x7f1da41a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:26.485 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.503+0000 7f1da3400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1da407dea0 0x7f1da41a2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46564/0 (socket says 172.21.15.53:46564) 2024-04-25T14:29:26.485 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.503+0000 7f1da3400700 1 -- 172.21.15.53:0/1318030554 learned_addr learned my addr 172.21.15.53:0/1318030554 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:29:26.485 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.503+0000 7f1da3400700 1 -- 172.21.15.53:0/1318030554 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da41a2ff0 con 0x7f1da407dea0 2024-04-25T14:29:26.485 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.503+0000 7f1da3400700 1 --2- 172.21.15.53:0/1318030554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1da407dea0 0x7f1da41a2ab0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1d94011ac0 tx=0x7f1d94011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:26.485 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.504+0000 7f1d9be00700 1 -- 172.21.15.53:0/1318030554 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d94003970 con 0x7f1da407dea0 2024-04-25T14:29:26.485 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.504+0000 7f1d9be00700 1 -- 172.21.15.53:0/1318030554 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1d94003b10 con 0x7f1da407dea0 2024-04-25T14:29:26.485 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.504+0000 7f1d9be00700 1 -- 172.21.15.53:0/1318030554 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d94024610 con 0x7f1da407dea0 2024-04-25T14:29:26.485 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.504+0000 7f1daa400700 1 -- 172.21.15.53:0/1318030554 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1da41a3280 con 0x7f1da407dea0 2024-04-25T14:29:26.485 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.504+0000 7f1daa400700 1 -- 172.21.15.53:0/1318030554 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1da41a5f70 con 0x7f1da407dea0 2024-04-25T14:29:26.485 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.505+0000 7f1d9be00700 1 -- 172.21.15.53:0/1318030554 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f1d9401a9e0 con 0x7f1da407dea0 2024-04-25T14:29:26.485 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.505+0000 7f1daa400700 1 -- 172.21.15.53:0/1318030554 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1da40640a0 con 0x7f1da407dea0 2024-04-25T14:29:26.485 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.506+0000 7f1d9be00700 1 --2- 172.21.15.53:0/1318030554 >> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] conn(0x7f1d880395b0 0x7f1d8803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:26.486 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.506+0000 7f1d9be00700 1 -- 172.21.15.53:0/1318030554 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1d9404d860 con 0x7f1da407dea0 2024-04-25T14:29:26.486 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.506+0000 7f1da2a00700 1 --2- 172.21.15.53:0/1318030554 >> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] conn(0x7f1d880395b0 0x7f1d8803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:26.486 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.507+0000 7f1da2a00700 1 --2- 172.21.15.53:0/1318030554 >> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] conn(0x7f1d880395b0 0x7f1d8803ba70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1d90006fd0 tx=0x7f1d90006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:26.486 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.513+0000 7f1d9be00700 1 -- 172.21.15.53:0/1318030554 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1d9400b3e0 con 0x7f1da407dea0 2024-04-25T14:29:26.486 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.830+0000 7f1daa400700 1 -- 172.21.15.53:0/1318030554 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f1da41a6140 con 0x7f1da407dea0 2024-04-25T14:29:26.486 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.831+0000 7f1d9be00700 1 -- 172.21.15.53:0/1318030554 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f1da41a6140 con 0x7f1da407dea0 2024-04-25T14:29:26.486 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.834+0000 7f1daa400700 1 -- 172.21.15.53:0/1318030554 >> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] conn(0x7f1d880395b0 msgr2=0x7f1d8803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:26.486 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.834+0000 7f1daa400700 1 --2- 172.21.15.53:0/1318030554 >> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] conn(0x7f1d880395b0 0x7f1d8803ba70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1d90006fd0 tx=0x7f1d90006e40 comp rx=0 tx=0).stop 2024-04-25T14:29:26.486 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.834+0000 7f1daa400700 1 -- 172.21.15.53:0/1318030554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1da407dea0 msgr2=0x7f1da41a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:26.486 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.834+0000 7f1daa400700 1 --2- 172.21.15.53:0/1318030554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1da407dea0 0x7f1da41a2ab0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1d94011ac0 tx=0x7f1d94011ba0 comp rx=0 tx=0).stop 2024-04-25T14:29:26.486 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.834+0000 7f1daa400700 1 -- 172.21.15.53:0/1318030554 shutdown_connections 2024-04-25T14:29:26.487 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.834+0000 7f1daa400700 1 --2- 172.21.15.53:0/1318030554 >> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] conn(0x7f1d880395b0 0x7f1d8803ba70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:26.487 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.834+0000 7f1daa400700 1 --2- 172.21.15.53:0/1318030554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1da407dea0 0x7f1da41a2ab0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:26.487 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.834+0000 7f1daa400700 1 -- 172.21.15.53:0/1318030554 >> 172.21.15.53:0/1318030554 conn(0x7f1da4103b10 msgr2=0x7f1da4105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:29:26.487 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.834+0000 7f1daa400700 1 -- 172.21.15.53:0/1318030554 shutdown_connections 2024-04-25T14:29:26.487 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:25.835+0000 7f1daa400700 1 -- 172.21.15.53:0/1318030554 wait complete. 2024-04-25T14:29:26.487 INFO:teuthology.orchestra.run.smithi053.stdout:mgr is available 2024-04-25T14:29:27.623 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:27 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1820020864' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T14:29:27.946 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 2024-04-25T14:29:27.947 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T14:29:27.947 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout fsid = fd30e6ce-030f-11ef-bc93-c7b262605968 2024-04-25T14:29:27.947 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T14:29:27.947 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T14:29:27.947 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T14:29:27.947 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T14:29:27.947 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T14:29:27.947 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 2024-04-25T14:29:27.947 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T14:29:27.949 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T14:29:27.949 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 2024-04-25T14:29:27.949 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T14:29:27.949 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-25T14:29:27.949 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T14:29:27.949 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T14:29:27.949 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T14:29:27.949 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T14:29:27.949 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T14:29:27.949 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.071+0000 7f4993000700 1 Processor -- start 2024-04-25T14:29:27.949 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.071+0000 7f4993000700 1 -- start start 2024-04-25T14:29:27.952 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.072+0000 7f4993000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f498c110220 0x7f498c110640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:27.952 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.072+0000 7f4993000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f498c110c10 con 0x7f498c110220 2024-04-25T14:29:27.952 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.072+0000 7f498be00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f498c110220 0x7f498c110640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:27.952 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.072+0000 7f498be00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f498c110220 0x7f498c110640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46570/0 (socket says 172.21.15.53:46570) 2024-04-25T14:29:27.952 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.072+0000 7f498be00700 1 -- 172.21.15.53:0/3077676235 learned_addr learned my addr 172.21.15.53:0/3077676235 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:29:27.952 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.072+0000 7f498be00700 1 -- 172.21.15.53:0/3077676235 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f498c10ece0 con 0x7f498c110220 2024-04-25T14:29:27.952 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.073+0000 7f498be00700 1 --2- 172.21.15.53:0/3077676235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f498c110220 0x7f498c110640 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f497c0098a0 tx=0x7f497c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9aff3025639a3f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:27.952 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.073+0000 7f498aa00700 1 -- 172.21.15.53:0/3077676235 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f497c004160 con 0x7f498c110220 2024-04-25T14:29:27.952 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.073+0000 7f498aa00700 1 -- 172.21.15.53:0/3077676235 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f497c004300 con 0x7f498c110220 2024-04-25T14:29:27.952 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.073+0000 7f498aa00700 1 -- 172.21.15.53:0/3077676235 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f497c011af0 con 0x7f498c110220 2024-04-25T14:29:27.953 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.074+0000 7f4993000700 1 -- 172.21.15.53:0/3077676235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f498c110220 msgr2=0x7f498c110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:27.953 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.074+0000 7f4993000700 1 --2- 172.21.15.53:0/3077676235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f498c110220 0x7f498c110640 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f497c0098a0 tx=0x7f497c009bb0 comp rx=0 tx=0).stop 2024-04-25T14:29:27.953 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.074+0000 7f4993000700 1 -- 172.21.15.53:0/3077676235 shutdown_connections 2024-04-25T14:29:27.953 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.074+0000 7f4993000700 1 --2- 172.21.15.53:0/3077676235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f498c110220 0x7f498c110640 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:27.953 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.074+0000 7f4993000700 1 -- 172.21.15.53:0/3077676235 >> 172.21.15.53:0/3077676235 conn(0x7f498c07de20 msgr2=0x7f498c080240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:29:27.953 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.074+0000 7f4993000700 1 -- 172.21.15.53:0/3077676235 shutdown_connections 2024-04-25T14:29:27.953 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.074+0000 7f4993000700 1 -- 172.21.15.53:0/3077676235 wait complete. 2024-04-25T14:29:27.953 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.075+0000 7f4993000700 1 Processor -- start 2024-04-25T14:29:27.953 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.075+0000 7f4993000700 1 -- start start 2024-04-25T14:29:27.953 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.076+0000 7f4993000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f498c110220 0x7f498c1a2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:27.953 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.076+0000 7f4993000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f498c113140 con 0x7f498c110220 2024-04-25T14:29:27.953 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.076+0000 7f498be00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f498c110220 0x7f498c1a2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:27.956 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.076+0000 7f498be00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f498c110220 0x7f498c1a2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46580/0 (socket says 172.21.15.53:46580) 2024-04-25T14:29:27.956 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.076+0000 7f498be00700 1 -- 172.21.15.53:0/1820020864 learned_addr learned my addr 172.21.15.53:0/1820020864 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:29:27.956 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.076+0000 7f498be00700 1 -- 172.21.15.53:0/1820020864 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f498c1a3160 con 0x7f498c110220 2024-04-25T14:29:27.957 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.077+0000 7f498be00700 1 --2- 172.21.15.53:0/1820020864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f498c110220 0x7f498c1a2c20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f497c0117b0 tx=0x7f497c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:27.957 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.077+0000 7f4988c00700 1 -- 172.21.15.53:0/1820020864 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f497c003af0 con 0x7f498c110220 2024-04-25T14:29:27.957 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.077+0000 7f4988c00700 1 -- 172.21.15.53:0/1820020864 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f497c020070 con 0x7f498c110220 2024-04-25T14:29:27.957 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.077+0000 7f4988c00700 1 -- 172.21.15.53:0/1820020864 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f497c019630 con 0x7f498c110220 2024-04-25T14:29:27.957 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.077+0000 7f4993000700 1 -- 172.21.15.53:0/1820020864 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f498c1a33f0 con 0x7f498c110220 2024-04-25T14:29:27.957 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.077+0000 7f4993000700 1 -- 172.21.15.53:0/1820020864 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f498c1a60e0 con 0x7f498c110220 2024-04-25T14:29:27.957 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.078+0000 7f4988c00700 1 -- 172.21.15.53:0/1820020864 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f497c01a9c0 con 0x7f498c110220 2024-04-25T14:29:27.957 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.078+0000 7f4993000700 1 -- 172.21.15.53:0/1820020864 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f498c0640a0 con 0x7f498c110220 2024-04-25T14:29:27.957 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.079+0000 7f4988c00700 1 --2- 172.21.15.53:0/1820020864 >> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] conn(0x7f49700395b0 0x7f497003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:27.957 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.079+0000 7f4988c00700 1 -- 172.21.15.53:0/1820020864 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f497c04dae0 con 0x7f498c110220 2024-04-25T14:29:27.957 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.079+0000 7f498b400700 1 --2- 172.21.15.53:0/1820020864 >> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] conn(0x7f49700395b0 0x7f497003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:27.957 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.080+0000 7f498b400700 1 --2- 172.21.15.53:0/1820020864 >> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] conn(0x7f49700395b0 0x7f497003ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4974006fd0 tx=0x7f4974006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:27.957 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.084+0000 7f4988c00700 1 -- 172.21.15.53:0/1820020864 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f497c0042a0 con 0x7f498c110220 2024-04-25T14:29:27.958 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.302+0000 7f4993000700 1 -- 172.21.15.53:0/1820020864 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f498c1a7780 con 0x7f498c110220 2024-04-25T14:29:27.958 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.303+0000 7f4988c00700 1 -- 172.21.15.53:0/1820020864 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+436 (secure 0 0 0) 0x7f497c00e260 con 0x7f498c110220 2024-04-25T14:29:27.958 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.306+0000 7f4993000700 1 -- 172.21.15.53:0/1820020864 >> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] conn(0x7f49700395b0 msgr2=0x7f497003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:27.958 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.306+0000 7f4993000700 1 --2- 172.21.15.53:0/1820020864 >> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] conn(0x7f49700395b0 0x7f497003ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4974006fd0 tx=0x7f4974006e40 comp rx=0 tx=0).stop 2024-04-25T14:29:27.958 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.306+0000 7f4993000700 1 -- 172.21.15.53:0/1820020864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f498c110220 msgr2=0x7f498c1a2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:27.958 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.306+0000 7f4993000700 1 --2- 172.21.15.53:0/1820020864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f498c110220 0x7f498c1a2c20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f497c0117b0 tx=0x7f497c011b70 comp rx=0 tx=0).stop 2024-04-25T14:29:27.958 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.306+0000 7f4993000700 1 -- 172.21.15.53:0/1820020864 shutdown_connections 2024-04-25T14:29:27.958 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.306+0000 7f4993000700 1 --2- 172.21.15.53:0/1820020864 >> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] conn(0x7f49700395b0 0x7f497003ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:27.958 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.306+0000 7f4993000700 1 --2- 172.21.15.53:0/1820020864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f498c110220 0x7f498c1a2c20 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:27.958 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.306+0000 7f4993000700 1 -- 172.21.15.53:0/1820020864 >> 172.21.15.53:0/1820020864 conn(0x7f498c07de20 msgr2=0x7f498c07f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:29:27.958 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.307+0000 7f4993000700 1 -- 172.21.15.53:0/1820020864 shutdown_connections 2024-04-25T14:29:27.958 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:27.307+0000 7f4993000700 1 -- 172.21.15.53:0/1820020864 wait complete. 2024-04-25T14:29:27.959 INFO:teuthology.orchestra.run.smithi053.stdout:Enabling cephadm module... 2024-04-25T14:29:29.239 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:28 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/4232912697' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-25T14:29:29.239 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:29 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: ignoring --setuser ceph since I am not root 2024-04-25T14:29:29.239 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:29 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: ignoring --setgroup ceph since I am not root 2024-04-25T14:29:29.239 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:29 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:29.238+0000 7f0c568e1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:29:29.685 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.587+0000 7f644b200700 1 Processor -- start 2024-04-25T14:29:29.685 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.587+0000 7f644b200700 1 -- start start 2024-04-25T14:29:29.685 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.588+0000 7f644b200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6444106db0 0x7f64441071d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:29.685 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.588+0000 7f644b200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f644407bec0 con 0x7f6444106db0 2024-04-25T14:29:29.685 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.588+0000 7f6443e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6444106db0 0x7f64441071d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:29.685 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.588+0000 7f6443e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6444106db0 0x7f64441071d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46592/0 (socket says 172.21.15.53:46592) 2024-04-25T14:29:29.685 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.588+0000 7f6443e00700 1 -- 172.21.15.53:0/2738217050 learned_addr learned my addr 172.21.15.53:0/2738217050 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:29:29.685 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.589+0000 7f6443e00700 1 -- 172.21.15.53:0/2738217050 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6444107710 con 0x7f6444106db0 2024-04-25T14:29:29.685 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.589+0000 7f6443e00700 1 --2- 172.21.15.53:0/2738217050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6444106db0 0x7f64441071d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f64340098a0 tx=0x7f6434009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd51ae8b85f0a635 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:29.685 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.589+0000 7f6442a00700 1 -- 172.21.15.53:0/2738217050 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6434004160 con 0x7f6444106db0 2024-04-25T14:29:29.685 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.589+0000 7f6442a00700 1 -- 172.21.15.53:0/2738217050 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6434004300 con 0x7f6444106db0 2024-04-25T14:29:29.685 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.589+0000 7f6442a00700 1 -- 172.21.15.53:0/2738217050 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6434011af0 con 0x7f6444106db0 2024-04-25T14:29:29.686 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.590+0000 7f644b200700 1 -- 172.21.15.53:0/2738217050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6444106db0 msgr2=0x7f64441071d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:29.686 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.590+0000 7f644b200700 1 --2- 172.21.15.53:0/2738217050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6444106db0 0x7f64441071d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f64340098a0 tx=0x7f6434009bb0 comp rx=0 tx=0).stop 2024-04-25T14:29:29.686 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.590+0000 7f644b200700 1 -- 172.21.15.53:0/2738217050 shutdown_connections 2024-04-25T14:29:29.686 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.590+0000 7f644b200700 1 --2- 172.21.15.53:0/2738217050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6444106db0 0x7f64441071d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:29.686 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.590+0000 7f644b200700 1 -- 172.21.15.53:0/2738217050 >> 172.21.15.53:0/2738217050 conn(0x7f6444102a50 msgr2=0x7f6444104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:29:29.686 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.590+0000 7f644b200700 1 -- 172.21.15.53:0/2738217050 shutdown_connections 2024-04-25T14:29:29.686 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.590+0000 7f644b200700 1 -- 172.21.15.53:0/2738217050 wait complete. 2024-04-25T14:29:29.686 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.591+0000 7f644b200700 1 Processor -- start 2024-04-25T14:29:29.686 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.592+0000 7f644b200700 1 -- start start 2024-04-25T14:29:29.686 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.592+0000 7f644b200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6444106db0 0x7f644419e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:29.686 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.592+0000 7f644b200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6444108740 con 0x7f6444106db0 2024-04-25T14:29:29.686 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.592+0000 7f6443e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6444106db0 0x7f644419e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:29.686 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.592+0000 7f6443e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6444106db0 0x7f644419e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46596/0 (socket says 172.21.15.53:46596) 2024-04-25T14:29:29.686 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.592+0000 7f6443e00700 1 -- 172.21.15.53:0/4232912697 learned_addr learned my addr 172.21.15.53:0/4232912697 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:29:29.687 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.593+0000 7f6443e00700 1 -- 172.21.15.53:0/4232912697 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644419ebe0 con 0x7f6444106db0 2024-04-25T14:29:29.687 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.593+0000 7f6443e00700 1 --2- 172.21.15.53:0/4232912697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6444106db0 0x7f644419e6a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6434011ac0 tx=0x7f6434011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:29.687 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.593+0000 7f6440c00700 1 -- 172.21.15.53:0/4232912697 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6434003970 con 0x7f6444106db0 2024-04-25T14:29:29.687 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.593+0000 7f6440c00700 1 -- 172.21.15.53:0/4232912697 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6434003b10 con 0x7f6444106db0 2024-04-25T14:29:29.687 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.593+0000 7f644b200700 1 -- 172.21.15.53:0/4232912697 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f644419ee70 con 0x7f6444106db0 2024-04-25T14:29:29.687 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.593+0000 7f6440c00700 1 -- 172.21.15.53:0/4232912697 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6434024610 con 0x7f6444106db0 2024-04-25T14:29:29.687 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.594+0000 7f644b200700 1 -- 172.21.15.53:0/4232912697 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64441a1b60 con 0x7f6444106db0 2024-04-25T14:29:29.687 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.595+0000 7f6440c00700 1 -- 172.21.15.53:0/4232912697 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f643401a9e0 con 0x7f6444106db0 2024-04-25T14:29:29.687 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.595+0000 7f644b200700 1 -- 172.21.15.53:0/4232912697 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64440640a0 con 0x7f6444106db0 2024-04-25T14:29:29.687 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.595+0000 7f6440c00700 1 --2- 172.21.15.53:0/4232912697 >> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] conn(0x7f6428039560 0x7f642803ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:29.687 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.595+0000 7f6440c00700 1 -- 172.21.15.53:0/4232912697 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f643404d890 con 0x7f6444106db0 2024-04-25T14:29:29.687 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.596+0000 7f6443400700 1 --2- 172.21.15.53:0/4232912697 >> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] conn(0x7f6428039560 0x7f642803ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:29.687 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.596+0000 7f6443400700 1 --2- 172.21.15.53:0/4232912697 >> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] conn(0x7f6428039560 0x7f642803ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6430006fd0 tx=0x7f6430006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:29.687 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.601+0000 7f6440c00700 1 -- 172.21.15.53:0/4232912697 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f643400b3e0 con 0x7f6444106db0 2024-04-25T14:29:29.688 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.896+0000 7f644b200700 1 -- 172.21.15.53:0/4232912697 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f64440008d0 con 0x7f6444106db0 2024-04-25T14:29:29.688 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.950+0000 7f6440c00700 1 -- 172.21.15.53:0/4232912697 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f643404d2b0 con 0x7f6444106db0 2024-04-25T14:29:29.688 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.954+0000 7f6440c00700 1 -- 172.21.15.53:0/4232912697 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f643402c350 con 0x7f6444106db0 2024-04-25T14:29:29.688 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.954+0000 7f644b200700 1 -- 172.21.15.53:0/4232912697 >> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] conn(0x7f6428039560 msgr2=0x7f642803ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:29.688 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.954+0000 7f644b200700 1 --2- 172.21.15.53:0/4232912697 >> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] conn(0x7f6428039560 0x7f642803ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6430006fd0 tx=0x7f6430006e40 comp rx=0 tx=0).stop 2024-04-25T14:29:29.688 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.954+0000 7f644b200700 1 -- 172.21.15.53:0/4232912697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6444106db0 msgr2=0x7f644419e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:29.688 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.954+0000 7f644b200700 1 --2- 172.21.15.53:0/4232912697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6444106db0 0x7f644419e6a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6434011ac0 tx=0x7f6434011ba0 comp rx=0 tx=0).stop 2024-04-25T14:29:29.688 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.956+0000 7f644b200700 1 -- 172.21.15.53:0/4232912697 shutdown_connections 2024-04-25T14:29:29.688 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.956+0000 7f644b200700 1 --2- 172.21.15.53:0/4232912697 >> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] conn(0x7f6428039560 0x7f642803ba20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:29.688 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.956+0000 7f644b200700 1 --2- 172.21.15.53:0/4232912697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6444106db0 0x7f644419e6a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:29.688 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.956+0000 7f644b200700 1 -- 172.21.15.53:0/4232912697 >> 172.21.15.53:0/4232912697 conn(0x7f6444102a50 msgr2=0x7f6444195360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:29:29.688 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.956+0000 7f644b200700 1 -- 172.21.15.53:0/4232912697 shutdown_connections 2024-04-25T14:29:29.688 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:28.956+0000 7f644b200700 1 -- 172.21.15.53:0/4232912697 wait complete. 2024-04-25T14:29:29.782 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:29 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:29.491+0000 7f0c568e1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:29:29.782 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:29 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:29.627+0000 7f0c568e1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:29:30.153 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:29 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/4232912697' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-25T14:29:30.153 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:29 smithi053 ceph-mon[27773]: mgrmap e5: x(active, since 5s) 2024-04-25T14:29:30.154 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:30 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:30.042+0000 7f0c568e1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:29:30.518 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:30 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:30.157+0000 7f0c568e1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:29:31.256 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:30 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1391199674' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T14:29:31.271 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout { 2024-04-25T14:29:31.272 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-25T14:29:31.272 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T14:29:31.272 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T14:29:31.272 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T14:29:31.272 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout } 2024-04-25T14:29:31.273 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.309+0000 7f5ddfc00700 1 Processor -- start 2024-04-25T14:29:31.273 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.309+0000 7f5ddfc00700 1 -- start start 2024-04-25T14:29:31.273 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.309+0000 7f5ddfc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5dd8108ff0 0x7f5dd810b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:31.273 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.310+0000 7f5ddfc00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd8063d20 con 0x7f5dd8108ff0 2024-04-25T14:29:31.273 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.310+0000 7f5ddd200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5dd8108ff0 0x7f5dd810b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:31.273 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.310+0000 7f5ddd200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5dd8108ff0 0x7f5dd810b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46614/0 (socket says 172.21.15.53:46614) 2024-04-25T14:29:31.273 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.310+0000 7f5ddd200700 1 -- 172.21.15.53:0/2733380621 learned_addr learned my addr 172.21.15.53:0/2733380621 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:29:31.282 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.310+0000 7f5ddd200700 1 -- 172.21.15.53:0/2733380621 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dd810b920 con 0x7f5dd8108ff0 2024-04-25T14:29:31.282 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.311+0000 7f5ddd200700 1 --2- 172.21.15.53:0/2733380621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5dd8108ff0 0x7f5dd810b3e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8009e80 tx=0x7f5dc800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75346f05b5f9ea34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:31.282 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.311+0000 7f5dd7e00700 1 -- 172.21.15.53:0/2733380621 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5dc8017070 con 0x7f5dd8108ff0 2024-04-25T14:29:31.282 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.311+0000 7f5dd7e00700 1 -- 172.21.15.53:0/2733380621 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5dc800b8c0 con 0x7f5dd8108ff0 2024-04-25T14:29:31.282 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.311+0000 7f5dd7e00700 1 -- 172.21.15.53:0/2733380621 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5dc8004490 con 0x7f5dd8108ff0 2024-04-25T14:29:31.282 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.312+0000 7f5ddfc00700 1 -- 172.21.15.53:0/2733380621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5dd8108ff0 msgr2=0x7f5dd810b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:31.282 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.312+0000 7f5ddfc00700 1 --2- 172.21.15.53:0/2733380621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5dd8108ff0 0x7f5dd810b3e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8009e80 tx=0x7f5dc800b330 comp rx=0 tx=0).stop 2024-04-25T14:29:31.282 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.312+0000 7f5ddfc00700 1 -- 172.21.15.53:0/2733380621 shutdown_connections 2024-04-25T14:29:31.282 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.312+0000 7f5ddfc00700 1 --2- 172.21.15.53:0/2733380621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5dd8108ff0 0x7f5dd810b3e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:31.282 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.312+0000 7f5ddfc00700 1 -- 172.21.15.53:0/2733380621 >> 172.21.15.53:0/2733380621 conn(0x7f5dd8102a40 msgr2=0x7f5dd8104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:29:31.283 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.312+0000 7f5ddfc00700 1 -- 172.21.15.53:0/2733380621 shutdown_connections 2024-04-25T14:29:31.283 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.312+0000 7f5ddfc00700 1 -- 172.21.15.53:0/2733380621 wait complete. 2024-04-25T14:29:31.283 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.313+0000 7f5ddfc00700 1 Processor -- start 2024-04-25T14:29:31.283 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.313+0000 7f5ddfc00700 1 -- start start 2024-04-25T14:29:31.283 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.313+0000 7f5ddfc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5dd8108ff0 0x7f5dd819c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:31.283 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.313+0000 7f5ddfc00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd810c8e0 con 0x7f5dd8108ff0 2024-04-25T14:29:31.283 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.314+0000 7f5ddd200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5dd8108ff0 0x7f5dd819c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:31.283 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.314+0000 7f5ddd200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5dd8108ff0 0x7f5dd819c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46626/0 (socket says 172.21.15.53:46626) 2024-04-25T14:29:31.283 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.314+0000 7f5ddd200700 1 -- 172.21.15.53:0/1391199674 learned_addr learned my addr 172.21.15.53:0/1391199674 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:29:31.283 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.314+0000 7f5ddd200700 1 -- 172.21.15.53:0/1391199674 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dd819c930 con 0x7f5dd8108ff0 2024-04-25T14:29:31.283 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.314+0000 7f5ddd200700 1 --2- 172.21.15.53:0/1391199674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5dd8108ff0 0x7f5dd819c3f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5dc800b7b0 tx=0x7f5dc800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:31.283 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.315+0000 7f5dd6000700 1 -- 172.21.15.53:0/1391199674 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5dc8020020 con 0x7f5dd8108ff0 2024-04-25T14:29:31.283 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.315+0000 7f5dd6000700 1 -- 172.21.15.53:0/1391199674 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5dc8012430 con 0x7f5dd8108ff0 2024-04-25T14:29:31.283 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.315+0000 7f5dd6000700 1 -- 172.21.15.53:0/1391199674 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5dc8017040 con 0x7f5dd8108ff0 2024-04-25T14:29:31.283 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.315+0000 7f5ddfc00700 1 -- 172.21.15.53:0/1391199674 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dd819cbc0 con 0x7f5dd8108ff0 2024-04-25T14:29:31.284 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.315+0000 7f5ddfc00700 1 -- 172.21.15.53:0/1391199674 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dd819f8b0 con 0x7f5dd8108ff0 2024-04-25T14:29:31.284 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.316+0000 7f5dd6000700 1 -- 172.21.15.53:0/1391199674 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f5dc8023020 con 0x7f5dd8108ff0 2024-04-25T14:29:31.284 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.316+0000 7f5ddfc00700 1 -- 172.21.15.53:0/1391199674 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dd80640a0 con 0x7f5dd8108ff0 2024-04-25T14:29:31.284 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.316+0000 7f5dd6000700 1 --2- 172.21.15.53:0/1391199674 >> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] conn(0x7f5dbc0395b0 0x7f5dbc03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:31.284 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.316+0000 7f5dd6000700 1 -- 172.21.15.53:0/1391199674 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5dc800f050 con 0x7f5dd8108ff0 2024-04-25T14:29:31.284 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.316+0000 7f5dcfe00700 1 -- 172.21.15.53:0/1391199674 >> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] conn(0x7f5dbc0395b0 msgr2=0x7f5dbc03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/1722497390 2024-04-25T14:29:31.284 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.317+0000 7f5dcfe00700 1 --2- 172.21.15.53:0/1391199674 >> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] conn(0x7f5dbc0395b0 0x7f5dbc03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:29:31.284 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.322+0000 7f5dd6000700 1 -- 172.21.15.53:0/1391199674 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5dc802a020 con 0x7f5dd8108ff0 2024-04-25T14:29:31.284 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.518+0000 7f5dcfe00700 1 -- 172.21.15.53:0/1391199674 >> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] conn(0x7f5dbc0395b0 msgr2=0x7f5dbc03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/1722497390 2024-04-25T14:29:31.284 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.518+0000 7f5dcfe00700 1 --2- 172.21.15.53:0/1391199674 >> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] conn(0x7f5dbc0395b0 0x7f5dbc03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T14:29:31.284 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.611+0000 7f5ddfc00700 1 -- 172.21.15.53:0/1391199674 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f5dd819fa80 con 0x7f5dd8108ff0 2024-04-25T14:29:31.284 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.612+0000 7f5dd6000700 1 -- 172.21.15.53:0/1391199674 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f5dd819fa80 con 0x7f5dd8108ff0 2024-04-25T14:29:31.284 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.615+0000 7f5ddfc00700 1 -- 172.21.15.53:0/1391199674 >> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] conn(0x7f5dbc0395b0 msgr2=0x7f5dbc03ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:29:31.284 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.615+0000 7f5ddfc00700 1 --2- 172.21.15.53:0/1391199674 >> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] conn(0x7f5dbc0395b0 0x7f5dbc03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:31.285 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.615+0000 7f5ddfc00700 1 -- 172.21.15.53:0/1391199674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5dd8108ff0 msgr2=0x7f5dd819c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:31.285 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.615+0000 7f5ddfc00700 1 --2- 172.21.15.53:0/1391199674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5dd8108ff0 0x7f5dd819c3f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5dc800b7b0 tx=0x7f5dc800be20 comp rx=0 tx=0).stop 2024-04-25T14:29:31.285 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.616+0000 7f5ddfc00700 1 -- 172.21.15.53:0/1391199674 shutdown_connections 2024-04-25T14:29:31.285 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.616+0000 7f5ddfc00700 1 --2- 172.21.15.53:0/1391199674 >> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] conn(0x7f5dbc0395b0 0x7f5dbc03ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:31.285 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.616+0000 7f5ddfc00700 1 --2- 172.21.15.53:0/1391199674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5dd8108ff0 0x7f5dd819c3f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:31.285 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.616+0000 7f5ddfc00700 1 -- 172.21.15.53:0/1391199674 >> 172.21.15.53:0/1391199674 conn(0x7f5dd8102a40 msgr2=0x7f5dd81034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:29:31.285 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.616+0000 7f5ddfc00700 1 -- 172.21.15.53:0/1391199674 shutdown_connections 2024-04-25T14:29:31.285 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:30.616+0000 7f5ddfc00700 1 -- 172.21.15.53:0/1391199674 wait complete. 2024-04-25T14:29:31.285 INFO:teuthology.orchestra.run.smithi053.stdout:Waiting for the mgr to restart... 2024-04-25T14:29:31.285 INFO:teuthology.orchestra.run.smithi053.stdout:Waiting for mgr epoch 5... 2024-04-25T14:29:31.842 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:31 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:31.767+0000 7f0c568e1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:29:32.220 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:32 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:32.002+0000 7f0c568e1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:29:32.221 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:32 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:32.220+0000 7f0c568e1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:29:33.000 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:32 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:32.616+0000 7f0c568e1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:29:33.000 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:32 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:32.743+0000 7f0c568e1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:29:33.253 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:33 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:32.999+0000 7f0c568e1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:29:33.532 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:33 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:33.261+0000 7f0c568e1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:29:34.532 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:34 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:34.087+0000 7f0c568e1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T14:29:35.032 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:34 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:34.657+0000 7f0c568e1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:29:35.032 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:34 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:34.788+0000 7f0c568e1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:29:35.378 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:35 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:35.062+0000 7f0c568e1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T14:29:36.032 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:35 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:35.728+0000 7f0c568e1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:29:36.032 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:35 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:35.855+0000 7f0c568e1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:29:36.498 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:36 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:36.186+0000 7f0c568e1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:29:36.499 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:36 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:36.369+0000 7f0c568e1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:29:36.499 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:36 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:36.498+0000 7f0c568e1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:29:36.782 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:36 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:36.653+0000 7f0c568e1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:29:37.282 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:37 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:37.028+0000 7f0c568e1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:29:38.102 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:37 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:37.810+0000 7f0c568e1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:29:38.532 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:38 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:29:38.102+0000 7f0c568e1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:29:38.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:38 smithi053 ceph-mon[27773]: Active manager daemon x restarted 2024-04-25T14:29:38.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:38 smithi053 ceph-mon[27773]: Activating manager daemon x 2024-04-25T14:29:38.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:38 smithi053 ceph-mon[27773]: osdmap e2: 0 total, 0 up, 0 in 2024-04-25T14:29:38.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:38 smithi053 ceph-mon[27773]: mgrmap e6: x(active, starting, since 0.0107299s) 2024-04-25T14:29:38.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:38 smithi053 ceph-mon[27773]: from='mgr.14120 172.21.15.53:0/2278703669' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:29:38.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:38 smithi053 ceph-mon[27773]: from='mgr.14120 172.21.15.53:0/2278703669' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T14:29:38.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:38 smithi053 ceph-mon[27773]: from='mgr.14120 172.21.15.53:0/2278703669' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T14:29:38.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:38 smithi053 ceph-mon[27773]: from='mgr.14120 172.21.15.53:0/2278703669' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T14:29:38.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:38 smithi053 ceph-mon[27773]: from='mgr.14120 172.21.15.53:0/2278703669' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T14:29:38.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:38 smithi053 ceph-mon[27773]: Manager daemon x is now available 2024-04-25T14:29:39.439 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:39 smithi053 ceph-mon[27773]: Found migration_current of "None". Setting to last migration. 2024-04-25T14:29:39.439 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:39 smithi053 ceph-mon[27773]: from='mgr.14120 172.21.15.53:0/2278703669' entity='mgr.x' 2024-04-25T14:29:39.439 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:39 smithi053 ceph-mon[27773]: from='mgr.14120 172.21.15.53:0/2278703669' entity='mgr.x' 2024-04-25T14:29:39.440 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:39 smithi053 ceph-mon[27773]: from='mgr.14120 172.21.15.53:0/2278703669' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:29:39.440 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:39 smithi053 ceph-mon[27773]: from='mgr.14120 172.21.15.53:0/2278703669' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:29:39.440 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:39 smithi053 ceph-mon[27773]: from='mgr.14120 172.21.15.53:0/2278703669' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T14:29:39.440 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:39 smithi053 ceph-mon[27773]: from='mgr.14120 172.21.15.53:0/2278703669' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T14:29:39.440 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:39 smithi053 ceph-mon[27773]: mgrmap e7: x(active, since 1.01649s) 2024-04-25T14:29:39.440 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:39 smithi053 ceph-mon[27773]: from='mgr.14120 172.21.15.53:0/2278703669' entity='mgr.x' 2024-04-25T14:29:39.440 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:39 smithi053 ceph-mon[27773]: from='mgr.14120 172.21.15.53:0/2278703669' entity='mgr.x' 2024-04-25T14:29:39.955 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout { 2024-04-25T14:29:39.965 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-25T14:29:39.965 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T14:29:39.965 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout } 2024-04-25T14:29:39.965 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:31.842+0000 7f7174a00700 1 Processor -- start 2024-04-25T14:29:39.965 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:31.842+0000 7f7174a00700 1 -- start start 2024-04-25T14:29:39.965 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:31.843+0000 7f7174a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7170108e10 0x7f717010b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:39.965 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:31.843+0000 7f7174a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7170063d20 con 0x7f7170108e10 2024-04-25T14:29:39.965 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:31.843+0000 7f716f400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7170108e10 0x7f717010b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:39.965 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:31.843+0000 7f716f400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7170108e10 0x7f717010b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46632/0 (socket says 172.21.15.53:46632) 2024-04-25T14:29:39.965 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:31.843+0000 7f716f400700 1 -- 172.21.15.53:0/3390153132 learned_addr learned my addr 172.21.15.53:0/3390153132 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:29:39.965 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:31.843+0000 7f716f400700 1 -- 172.21.15.53:0/3390153132 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f717010b740 con 0x7f7170108e10 2024-04-25T14:29:39.966 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:31.844+0000 7f716f400700 1 --2- 172.21.15.53:0/3390153132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7170108e10 0x7f717010b200 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7160009e80 tx=0x7f716000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=705702ceff987850 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:39.966 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:31.844+0000 7f716ea00700 1 -- 172.21.15.53:0/3390153132 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7160017070 con 0x7f7170108e10 2024-04-25T14:29:39.966 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:31.844+0000 7f716ea00700 1 -- 172.21.15.53:0/3390153132 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f716000b8c0 con 0x7f7170108e10 2024-04-25T14:29:39.966 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:31.844+0000 7f716ea00700 1 -- 172.21.15.53:0/3390153132 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7160004490 con 0x7f7170108e10 2024-04-25T14:29:39.966 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:31.845+0000 7f7174a00700 1 -- 172.21.15.53:0/3390153132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7170108e10 msgr2=0x7f717010b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:39.966 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:31.845+0000 7f7174a00700 1 --2- 172.21.15.53:0/3390153132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7170108e10 0x7f717010b200 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7160009e80 tx=0x7f716000b330 comp rx=0 tx=0).stop 2024-04-25T14:29:39.966 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:31.845+0000 7f7174a00700 1 -- 172.21.15.53:0/3390153132 shutdown_connections 2024-04-25T14:29:39.966 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:31.845+0000 7f7174a00700 1 --2- 172.21.15.53:0/3390153132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7170108e10 0x7f717010b200 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:39.966 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:31.845+0000 7f7174a00700 1 -- 172.21.15.53:0/3390153132 >> 172.21.15.53:0/3390153132 conn(0x7f7170102820 msgr2=0x7f7170104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:29:39.966 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:31.845+0000 7f7174a00700 1 -- 172.21.15.53:0/3390153132 shutdown_connections 2024-04-25T14:29:39.966 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:31.845+0000 7f7174a00700 1 -- 172.21.15.53:0/3390153132 wait complete. 2024-04-25T14:29:39.966 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:31.846+0000 7f7174a00700 1 Processor -- start 2024-04-25T14:29:39.966 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:31.846+0000 7f7174a00700 1 -- start start 2024-04-25T14:29:39.966 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:31.847+0000 7f7174a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7170108e10 0x7f7170108690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:39.967 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:31.847+0000 7f7174a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f717010c700 con 0x7f7170108e10 2024-04-25T14:29:39.967 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:31.847+0000 7f716f400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7170108e10 0x7f7170108690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:39.967 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:31.847+0000 7f716f400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7170108e10 0x7f7170108690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46648/0 (socket says 172.21.15.53:46648) 2024-04-25T14:29:39.967 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:31.847+0000 7f716f400700 1 -- 172.21.15.53:0/2462403999 learned_addr learned my addr 172.21.15.53:0/2462403999 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:29:39.967 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:31.848+0000 7f716f400700 1 -- 172.21.15.53:0/2462403999 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7170108bd0 con 0x7f7170108e10 2024-04-25T14:29:39.967 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:31.848+0000 7f716f400700 1 --2- 172.21.15.53:0/2462403999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7170108e10 0x7f7170108690 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f716000b7b0 tx=0x7f716000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:39.967 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:31.848+0000 7f716cc00700 1 -- 172.21.15.53:0/2462403999 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7160020020 con 0x7f7170108e10 2024-04-25T14:29:39.967 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:31.848+0000 7f716cc00700 1 -- 172.21.15.53:0/2462403999 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7160012430 con 0x7f7170108e10 2024-04-25T14:29:39.967 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:31.848+0000 7f716cc00700 1 -- 172.21.15.53:0/2462403999 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7160017040 con 0x7f7170108e10 2024-04-25T14:29:39.967 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:31.848+0000 7f7174a00700 1 -- 172.21.15.53:0/2462403999 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7170106dc0 con 0x7f7170108e10 2024-04-25T14:29:39.967 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:31.848+0000 7f7174a00700 1 -- 172.21.15.53:0/2462403999 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7170107300 con 0x7f7170108e10 2024-04-25T14:29:39.967 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:31.849+0000 7f716cc00700 1 -- 172.21.15.53:0/2462403999 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f7160023020 con 0x7f7170108e10 2024-04-25T14:29:39.967 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:31.850+0000 7f716cc00700 1 --2- 172.21.15.53:0/2462403999 >> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] conn(0x7f71540395b0 0x7f715403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:39.967 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:31.850+0000 7f716cc00700 1 -- 172.21.15.53:0/2462403999 --> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f715403c180 con 0x7f71540395b0 2024-04-25T14:29:39.968 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:31.850+0000 7f716cc00700 1 -- 172.21.15.53:0/2462403999 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f716000f050 con 0x7f7170108e10 2024-04-25T14:29:39.968 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:31.850+0000 7f7166a00700 1 -- 172.21.15.53:0/2462403999 >> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] conn(0x7f71540395b0 msgr2=0x7f715403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/1722497390 2024-04-25T14:29:39.968 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:31.850+0000 7f7166a00700 1 --2- 172.21.15.53:0/2462403999 >> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] conn(0x7f71540395b0 0x7f715403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:29:39.968 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:32.051+0000 7f7166a00700 1 -- 172.21.15.53:0/2462403999 >> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] conn(0x7f71540395b0 msgr2=0x7f715403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/1722497390 2024-04-25T14:29:39.968 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:32.051+0000 7f7166a00700 1 --2- 172.21.15.53:0/2462403999 >> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] conn(0x7f71540395b0 0x7f715403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T14:29:39.968 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:32.452+0000 7f7166a00700 1 -- 172.21.15.53:0/2462403999 >> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] conn(0x7f71540395b0 msgr2=0x7f715403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/1722497390 2024-04-25T14:29:39.968 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:32.452+0000 7f7166a00700 1 --2- 172.21.15.53:0/2462403999 >> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] conn(0x7f71540395b0 0x7f715403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T14:29:39.968 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:33.253+0000 7f7166a00700 1 -- 172.21.15.53:0/2462403999 >> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] conn(0x7f71540395b0 msgr2=0x7f715403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/1722497390 2024-04-25T14:29:39.968 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:33.253+0000 7f7166a00700 1 --2- 172.21.15.53:0/2462403999 >> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] conn(0x7f71540395b0 0x7f715403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T14:29:39.968 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:34.855+0000 7f7166a00700 1 -- 172.21.15.53:0/2462403999 >> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] conn(0x7f71540395b0 msgr2=0x7f715403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/1722497390 2024-04-25T14:29:39.968 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:34.855+0000 7f7166a00700 1 --2- 172.21.15.53:0/2462403999 >> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] conn(0x7f71540395b0 0x7f715403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T14:29:39.968 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:38.058+0000 7f7166a00700 1 -- 172.21.15.53:0/2462403999 >> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] conn(0x7f71540395b0 msgr2=0x7f715403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/1722497390 2024-04-25T14:29:39.968 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:38.058+0000 7f7166a00700 1 --2- 172.21.15.53:0/2462403999 >> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] conn(0x7f71540395b0 0x7f715403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-04-25T14:29:39.968 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:38.113+0000 7f716cc00700 1 -- 172.21.15.53:0/2462403999 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f7160003bd0 con 0x7f7170108e10 2024-04-25T14:29:39.969 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:38.114+0000 7f716cc00700 1 -- 172.21.15.53:0/2462403999 >> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] conn(0x7f71540395b0 msgr2=0x7f715403ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:29:39.969 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:38.114+0000 7f716cc00700 1 --2- 172.21.15.53:0/2462403999 >> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] conn(0x7f71540395b0 0x7f715403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:39.969 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:39.115+0000 7f716cc00700 1 -- 172.21.15.53:0/2462403999 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f716004f0a0 con 0x7f7170108e10 2024-04-25T14:29:39.969 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:39.115+0000 7f716cc00700 1 --2- 172.21.15.53:0/2462403999 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f715403cea0 0x7f715403f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:39.969 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:39.115+0000 7f716cc00700 1 -- 172.21.15.53:0/2462403999 --> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f716000aea0 con 0x7f715403cea0 2024-04-25T14:29:39.969 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:39.118+0000 7f7166a00700 1 --2- 172.21.15.53:0/2462403999 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f715403cea0 0x7f715403f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:39.969 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:39.121+0000 7f7166a00700 1 --2- 172.21.15.53:0/2462403999 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f715403cea0 0x7f715403f290 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7158003a10 tx=0x7f71580092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:39.969 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:39.123+0000 7f716cc00700 1 -- 172.21.15.53:0/2462403999 <== mgr.14120 v2:172.21.15.53:6800/3887954889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f716000aea0 con 0x7f715403cea0 2024-04-25T14:29:39.969 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:39.132+0000 7f7174a00700 1 -- 172.21.15.53:0/2462403999 --> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f7170069ba0 con 0x7f715403cea0 2024-04-25T14:29:39.969 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:39.132+0000 7f716cc00700 1 -- 172.21.15.53:0/2462403999 <== mgr.14120 v2:172.21.15.53:6800/3887954889 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f7170069ba0 con 0x7f715403cea0 2024-04-25T14:29:39.969 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:39.133+0000 7f7174a00700 1 -- 172.21.15.53:0/2462403999 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f715403cea0 msgr2=0x7f715403f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:39.969 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:39.133+0000 7f7174a00700 1 --2- 172.21.15.53:0/2462403999 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f715403cea0 0x7f715403f290 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7158003a10 tx=0x7f71580092b0 comp rx=0 tx=0).stop 2024-04-25T14:29:39.969 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:39.133+0000 7f7174a00700 1 -- 172.21.15.53:0/2462403999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7170108e10 msgr2=0x7f7170108690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:39.970 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:39.133+0000 7f7174a00700 1 --2- 172.21.15.53:0/2462403999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7170108e10 0x7f7170108690 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f716000b7b0 tx=0x7f716000be20 comp rx=0 tx=0).stop 2024-04-25T14:29:39.970 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:39.133+0000 7f7174a00700 1 -- 172.21.15.53:0/2462403999 shutdown_connections 2024-04-25T14:29:39.970 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:39.133+0000 7f7174a00700 1 --2- 172.21.15.53:0/2462403999 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f715403cea0 0x7f715403f290 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:39.970 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:39.133+0000 7f7174a00700 1 --2- 172.21.15.53:0/2462403999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7170108e10 0x7f7170108690 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:39.970 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:39.133+0000 7f7174a00700 1 --2- 172.21.15.53:0/2462403999 >> [v2:172.21.15.53:6800/1722497390,v1:172.21.15.53:6801/1722497390] conn(0x7f71540395b0 0x7f715403ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:39.970 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:39.133+0000 7f7174a00700 1 -- 172.21.15.53:0/2462403999 >> 172.21.15.53:0/2462403999 conn(0x7f7170102820 msgr2=0x7f71701033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:29:39.970 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:39.133+0000 7f7174a00700 1 -- 172.21.15.53:0/2462403999 shutdown_connections 2024-04-25T14:29:39.970 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:39.133+0000 7f7174a00700 1 -- 172.21.15.53:0/2462403999 wait complete. 2024-04-25T14:29:39.970 INFO:teuthology.orchestra.run.smithi053.stdout:mgr epoch 5 is available 2024-04-25T14:29:39.970 INFO:teuthology.orchestra.run.smithi053.stdout:Setting orchestrator backend to cephadm... 2024-04-25T14:29:40.274 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:40 smithi053 ceph-mon[27773]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T14:29:40.274 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:40 smithi053 ceph-mon[27773]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T14:29:41.502 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:41 smithi053 ceph-mon[27773]: mgrmap e8: x(active, since 2s) 2024-04-25T14:29:41.502 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:41 smithi053 ceph-mon[27773]: [25/Apr/2024:14:29:40] ENGINE Bus STARTING 2024-04-25T14:29:41.502 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:41 smithi053 ceph-mon[27773]: [25/Apr/2024:14:29:40] ENGINE Serving on https://172.21.15.53:7150 2024-04-25T14:29:41.502 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:41 smithi053 ceph-mon[27773]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:29:41.502 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:41 smithi053 ceph-mon[27773]: from='mgr.14120 172.21.15.53:0/2278703669' entity='mgr.x' 2024-04-25T14:29:41.502 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:41 smithi053 ceph-mon[27773]: [25/Apr/2024:14:29:40] ENGINE Serving on http://172.21.15.53:8765 2024-04-25T14:29:41.503 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:41 smithi053 ceph-mon[27773]: [25/Apr/2024:14:29:40] ENGINE Bus STARTED 2024-04-25T14:29:41.503 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:41 smithi053 ceph-mon[27773]: from='mgr.14120 172.21.15.53:0/2278703669' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:29:41.516 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.557+0000 7fc704200700 1 Processor -- start 2024-04-25T14:29:41.517 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.558+0000 7fc704200700 1 -- start start 2024-04-25T14:29:41.517 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.558+0000 7fc704200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6fc109070 0x7fc6fc10b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:41.517 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.558+0000 7fc704200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc6fc063ca0 con 0x7fc6fc109070 2024-04-25T14:29:41.517 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.558+0000 7fc701800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6fc109070 0x7fc6fc10b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:41.517 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.558+0000 7fc701800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6fc109070 0x7fc6fc10b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48074/0 (socket says 172.21.15.53:48074) 2024-04-25T14:29:41.517 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.558+0000 7fc701800700 1 -- 172.21.15.53:0/1697647272 learned_addr learned my addr 172.21.15.53:0/1697647272 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:29:41.517 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.559+0000 7fc701800700 1 -- 172.21.15.53:0/1697647272 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6fc10b9a0 con 0x7fc6fc109070 2024-04-25T14:29:41.517 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.559+0000 7fc701800700 1 --2- 172.21.15.53:0/1697647272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6fc109070 0x7fc6fc10b460 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc6f00098a0 tx=0x7fc6f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8cbb04ca61c2e99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:41.517 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.559+0000 7fc6fbe00700 1 -- 172.21.15.53:0/1697647272 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6f0004160 con 0x7fc6fc109070 2024-04-25T14:29:41.518 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.560+0000 7fc6fbe00700 1 -- 172.21.15.53:0/1697647272 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc6f0004300 con 0x7fc6fc109070 2024-04-25T14:29:41.518 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.560+0000 7fc6fbe00700 1 -- 172.21.15.53:0/1697647272 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6f0011af0 con 0x7fc6fc109070 2024-04-25T14:29:41.518 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.560+0000 7fc704200700 1 -- 172.21.15.53:0/1697647272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6fc109070 msgr2=0x7fc6fc10b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:41.518 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.560+0000 7fc704200700 1 --2- 172.21.15.53:0/1697647272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6fc109070 0x7fc6fc10b460 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc6f00098a0 tx=0x7fc6f0009bb0 comp rx=0 tx=0).stop 2024-04-25T14:29:41.518 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.560+0000 7fc704200700 1 -- 172.21.15.53:0/1697647272 shutdown_connections 2024-04-25T14:29:41.518 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.560+0000 7fc704200700 1 --2- 172.21.15.53:0/1697647272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6fc109070 0x7fc6fc10b460 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:41.518 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.560+0000 7fc704200700 1 -- 172.21.15.53:0/1697647272 >> 172.21.15.53:0/1697647272 conn(0x7fc6fc102a40 msgr2=0x7fc6fc104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:29:41.518 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.560+0000 7fc704200700 1 -- 172.21.15.53:0/1697647272 shutdown_connections 2024-04-25T14:29:41.518 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.560+0000 7fc704200700 1 -- 172.21.15.53:0/1697647272 wait complete. 2024-04-25T14:29:41.518 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.562+0000 7fc704200700 1 Processor -- start 2024-04-25T14:29:41.518 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.562+0000 7fc704200700 1 -- start start 2024-04-25T14:29:41.519 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.562+0000 7fc704200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6fc109070 0x7fc6fc19c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:41.519 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.562+0000 7fc704200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc6fc10c8b0 con 0x7fc6fc109070 2024-04-25T14:29:41.519 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.562+0000 7fc701800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6fc109070 0x7fc6fc19c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:41.519 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.562+0000 7fc701800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6fc109070 0x7fc6fc19c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48088/0 (socket says 172.21.15.53:48088) 2024-04-25T14:29:41.519 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.562+0000 7fc701800700 1 -- 172.21.15.53:0/761507807 learned_addr learned my addr 172.21.15.53:0/761507807 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:29:41.519 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.563+0000 7fc701800700 1 -- 172.21.15.53:0/761507807 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6fc19c990 con 0x7fc6fc109070 2024-04-25T14:29:41.519 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.563+0000 7fc701800700 1 --2- 172.21.15.53:0/761507807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6fc109070 0x7fc6fc19c450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0011ac0 tx=0x7fc6f0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:41.519 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.563+0000 7fc6fa000700 1 -- 172.21.15.53:0/761507807 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6f0003b20 con 0x7fc6fc109070 2024-04-25T14:29:41.519 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.563+0000 7fc6fa000700 1 -- 172.21.15.53:0/761507807 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc6f001a8c0 con 0x7fc6fc109070 2024-04-25T14:29:41.519 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.563+0000 7fc704200700 1 -- 172.21.15.53:0/761507807 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6fc19cc20 con 0x7fc6fc109070 2024-04-25T14:29:41.519 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.563+0000 7fc6fa000700 1 -- 172.21.15.53:0/761507807 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6f00248d0 con 0x7fc6fc109070 2024-04-25T14:29:41.520 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.564+0000 7fc704200700 1 -- 172.21.15.53:0/761507807 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6fc19f910 con 0x7fc6fc109070 2024-04-25T14:29:41.520 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.565+0000 7fc704200700 1 -- 172.21.15.53:0/761507807 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6fc0640a0 con 0x7fc6fc109070 2024-04-25T14:29:41.520 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.565+0000 7fc6fa000700 1 -- 172.21.15.53:0/761507807 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fc6f0022020 con 0x7fc6fc109070 2024-04-25T14:29:41.520 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.565+0000 7fc6fa000700 1 --2- 172.21.15.53:0/761507807 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7fc6e4039600 0x7fc6e403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:41.520 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.565+0000 7fc6fa000700 1 -- 172.21.15.53:0/761507807 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc6f004d900 con 0x7fc6fc109070 2024-04-25T14:29:41.520 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.566+0000 7fc700e00700 1 --2- 172.21.15.53:0/761507807 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7fc6e4039600 0x7fc6e403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:41.520 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.567+0000 7fc700e00700 1 --2- 172.21.15.53:0/761507807 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7fc6e4039600 0x7fc6e403bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8006fd0 tx=0x7fc6e8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:41.520 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.572+0000 7fc6fa000700 1 -- 172.21.15.53:0/761507807 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc6f000f2e0 con 0x7fc6fc109070 2024-04-25T14:29:41.520 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.817+0000 7fc704200700 1 -- 172.21.15.53:0/761507807 --> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fc6fc103c20 con 0x7fc6e4039600 2024-04-25T14:29:41.520 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.828+0000 7fc6fa000700 1 -- 172.21.15.53:0/761507807 <== mgr.14120 v2:172.21.15.53:6800/3887954889 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc6fc103c20 con 0x7fc6e4039600 2024-04-25T14:29:41.521 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.831+0000 7fc704200700 1 -- 172.21.15.53:0/761507807 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7fc6e4039600 msgr2=0x7fc6e403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:41.521 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.831+0000 7fc704200700 1 --2- 172.21.15.53:0/761507807 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7fc6e4039600 0x7fc6e403bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8006fd0 tx=0x7fc6e8006e40 comp rx=0 tx=0).stop 2024-04-25T14:29:41.521 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.831+0000 7fc704200700 1 -- 172.21.15.53:0/761507807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6fc109070 msgr2=0x7fc6fc19c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:41.521 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.831+0000 7fc704200700 1 --2- 172.21.15.53:0/761507807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6fc109070 0x7fc6fc19c450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0011ac0 tx=0x7fc6f0011ba0 comp rx=0 tx=0).stop 2024-04-25T14:29:41.521 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.831+0000 7fc704200700 1 -- 172.21.15.53:0/761507807 shutdown_connections 2024-04-25T14:29:41.521 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.831+0000 7fc704200700 1 --2- 172.21.15.53:0/761507807 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7fc6e4039600 0x7fc6e403bac0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:41.521 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.831+0000 7fc704200700 1 --2- 172.21.15.53:0/761507807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6fc109070 0x7fc6fc19c450 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:41.521 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.831+0000 7fc704200700 1 -- 172.21.15.53:0/761507807 >> 172.21.15.53:0/761507807 conn(0x7fc6fc102a40 msgr2=0x7fc6fc103510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:29:41.521 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.832+0000 7fc704200700 1 -- 172.21.15.53:0/761507807 shutdown_connections 2024-04-25T14:29:41.521 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:40.832+0000 7fc704200700 1 -- 172.21.15.53:0/761507807 wait complete. 2024-04-25T14:29:43.161 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-25T14:29:43.161 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.130+0000 7f41a9a00700 1 Processor -- start 2024-04-25T14:29:43.161 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.131+0000 7f41a9a00700 1 -- start start 2024-04-25T14:29:43.161 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.131+0000 7f41a9a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41a4106dc0 0x7f41a41071e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:43.161 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.131+0000 7f41a9a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f41a4063d40 con 0x7f41a4106dc0 2024-04-25T14:29:43.161 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.132+0000 7f41a2a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41a4106dc0 0x7f41a41071e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:43.161 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.132+0000 7f41a2a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41a4106dc0 0x7f41a41071e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48100/0 (socket says 172.21.15.53:48100) 2024-04-25T14:29:43.161 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.132+0000 7f41a2a00700 1 -- 172.21.15.53:0/300683546 learned_addr learned my addr 172.21.15.53:0/300683546 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:29:43.161 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.132+0000 7f41a2a00700 1 -- 172.21.15.53:0/300683546 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41a4107720 con 0x7f41a4106dc0 2024-04-25T14:29:43.167 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.133+0000 7f41a2a00700 1 --2- 172.21.15.53:0/300683546 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41a4106dc0 0x7f41a41071e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f41940098a0 tx=0x7f4194009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc404e8bd2cb26c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:43.167 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.133+0000 7f41a1600700 1 -- 172.21.15.53:0/300683546 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4194004160 con 0x7f41a4106dc0 2024-04-25T14:29:43.167 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.133+0000 7f41a1600700 1 -- 172.21.15.53:0/300683546 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4194004300 con 0x7f41a4106dc0 2024-04-25T14:29:43.167 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.133+0000 7f41a1600700 1 -- 172.21.15.53:0/300683546 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4194011af0 con 0x7f41a4106dc0 2024-04-25T14:29:43.167 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.133+0000 7f41a9a00700 1 -- 172.21.15.53:0/300683546 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41a4106dc0 msgr2=0x7f41a41071e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:43.167 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.133+0000 7f41a9a00700 1 --2- 172.21.15.53:0/300683546 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41a4106dc0 0x7f41a41071e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f41940098a0 tx=0x7f4194009bb0 comp rx=0 tx=0).stop 2024-04-25T14:29:43.167 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.134+0000 7f41a9a00700 1 -- 172.21.15.53:0/300683546 shutdown_connections 2024-04-25T14:29:43.167 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.134+0000 7f41a9a00700 1 --2- 172.21.15.53:0/300683546 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41a4106dc0 0x7f41a41071e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:43.167 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.134+0000 7f41a9a00700 1 -- 172.21.15.53:0/300683546 >> 172.21.15.53:0/300683546 conn(0x7f41a4102a40 msgr2=0x7f41a4104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:29:43.167 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.134+0000 7f41a9a00700 1 -- 172.21.15.53:0/300683546 shutdown_connections 2024-04-25T14:29:43.167 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.134+0000 7f41a9a00700 1 -- 172.21.15.53:0/300683546 wait complete. 2024-04-25T14:29:43.167 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.135+0000 7f41a9a00700 1 Processor -- start 2024-04-25T14:29:43.167 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.136+0000 7f41a9a00700 1 -- start start 2024-04-25T14:29:43.168 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.136+0000 7f41a9a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41a4106dc0 0x7f41a419e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:43.168 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.136+0000 7f41a9a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f41a4108750 con 0x7f41a4106dc0 2024-04-25T14:29:43.168 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.136+0000 7f41a2a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41a4106dc0 0x7f41a419e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:43.168 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.136+0000 7f41a2a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41a4106dc0 0x7f41a419e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48110/0 (socket says 172.21.15.53:48110) 2024-04-25T14:29:43.168 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.136+0000 7f41a2a00700 1 -- 172.21.15.53:0/1833150456 learned_addr learned my addr 172.21.15.53:0/1833150456 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:29:43.168 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.137+0000 7f41a2a00700 1 -- 172.21.15.53:0/1833150456 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41a419eb80 con 0x7f41a4106dc0 2024-04-25T14:29:43.168 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.137+0000 7f41a2a00700 1 --2- 172.21.15.53:0/1833150456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41a4106dc0 0x7f41a419e640 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4194011ac0 tx=0x7f4194011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:43.168 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.137+0000 7f419b400700 1 -- 172.21.15.53:0/1833150456 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4194003b20 con 0x7f41a4106dc0 2024-04-25T14:29:43.168 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.138+0000 7f419b400700 1 -- 172.21.15.53:0/1833150456 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f419401a8c0 con 0x7f41a4106dc0 2024-04-25T14:29:43.168 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.138+0000 7f419b400700 1 -- 172.21.15.53:0/1833150456 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41940248d0 con 0x7f41a4106dc0 2024-04-25T14:29:43.168 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.138+0000 7f41a9a00700 1 -- 172.21.15.53:0/1833150456 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41a419ee10 con 0x7f41a4106dc0 2024-04-25T14:29:43.168 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.138+0000 7f41a9a00700 1 -- 172.21.15.53:0/1833150456 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41a41a1b00 con 0x7f41a4106dc0 2024-04-25T14:29:43.168 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.139+0000 7f419b400700 1 -- 172.21.15.53:0/1833150456 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f4194022020 con 0x7f41a4106dc0 2024-04-25T14:29:43.168 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.139+0000 7f4198c00700 1 -- 172.21.15.53:0/1833150456 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41a4069ba0 con 0x7f41a4106dc0 2024-04-25T14:29:43.169 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.140+0000 7f419b400700 1 --2- 172.21.15.53:0/1833150456 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f4188039600 0x7f418803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:43.169 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.140+0000 7f419b400700 1 -- 172.21.15.53:0/1833150456 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f419404dd20 con 0x7f41a4106dc0 2024-04-25T14:29:43.169 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.140+0000 7f41a2000700 1 --2- 172.21.15.53:0/1833150456 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f4188039600 0x7f418803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:43.169 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.141+0000 7f41a2000700 1 --2- 172.21.15.53:0/1833150456 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f4188039600 0x7f418803bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4190006fd0 tx=0x7f4190006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:43.169 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.146+0000 7f419b400700 1 -- 172.21.15.53:0/1833150456 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f419400f2e0 con 0x7f41a4106dc0 2024-04-25T14:29:43.169 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.405+0000 7f4198c00700 1 -- 172.21.15.53:0/1833150456 --> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f41a419f2f0 con 0x7f4188039600 2024-04-25T14:29:43.169 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.406+0000 7f419b400700 1 -- 172.21.15.53:0/1833150456 <== mgr.14120 v2:172.21.15.53:6800/3887954889 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f41a419f2f0 con 0x7f4188039600 2024-04-25T14:29:43.169 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.409+0000 7f4198c00700 1 -- 172.21.15.53:0/1833150456 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f4188039600 msgr2=0x7f418803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:43.169 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.409+0000 7f4198c00700 1 --2- 172.21.15.53:0/1833150456 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f4188039600 0x7f418803bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4190006fd0 tx=0x7f4190006e40 comp rx=0 tx=0).stop 2024-04-25T14:29:43.169 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.409+0000 7f4198c00700 1 -- 172.21.15.53:0/1833150456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41a4106dc0 msgr2=0x7f41a419e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:43.169 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.409+0000 7f4198c00700 1 --2- 172.21.15.53:0/1833150456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41a4106dc0 0x7f41a419e640 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4194011ac0 tx=0x7f4194011ba0 comp rx=0 tx=0).stop 2024-04-25T14:29:43.169 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.409+0000 7f4198c00700 1 -- 172.21.15.53:0/1833150456 shutdown_connections 2024-04-25T14:29:43.169 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.409+0000 7f4198c00700 1 --2- 172.21.15.53:0/1833150456 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f4188039600 0x7f418803bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:43.169 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.409+0000 7f4198c00700 1 --2- 172.21.15.53:0/1833150456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41a4106dc0 0x7f41a419e640 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:43.170 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.409+0000 7f4198c00700 1 -- 172.21.15.53:0/1833150456 >> 172.21.15.53:0/1833150456 conn(0x7f41a4102a40 msgr2=0x7f41a4195220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:29:43.170 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.409+0000 7f4198c00700 1 -- 172.21.15.53:0/1833150456 shutdown_connections 2024-04-25T14:29:43.170 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:42.410+0000 7f4198c00700 1 -- 172.21.15.53:0/1833150456 wait complete. 2024-04-25T14:29:43.170 INFO:teuthology.orchestra.run.smithi053.stdout:Generating ssh key... 2024-04-25T14:29:43.447 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:43 smithi053 ceph-mon[27773]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:29:44.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:44 smithi053 ceph-mon[27773]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:29:44.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:44 smithi053 ceph-mon[27773]: Generating ssh key... 2024-04-25T14:29:44.533 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:44 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: Generating public/private rsa key pair. 2024-04-25T14:29:44.533 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:44 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: Your identification has been saved in /tmp/tmp_an2hy2p/key. 2024-04-25T14:29:44.533 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:44 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: Your public key has been saved in /tmp/tmp_an2hy2p/key.pub. 2024-04-25T14:29:44.533 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:44 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: The key fingerprint is: 2024-04-25T14:29:44.534 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:44 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: SHA256:IiEJrgikGH21nfcJr3rcfhc5KgRr/p0/kDvWLxwY9Do ceph-fd30e6ce-030f-11ef-bc93-c7b262605968 2024-04-25T14:29:44.534 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:44 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: The key's randomart image is: 2024-04-25T14:29:44.534 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:44 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: +---[RSA 3072]----+ 2024-04-25T14:29:44.534 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:44 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: |oo .. | 2024-04-25T14:29:44.534 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:44 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: |=o... o . . | 2024-04-25T14:29:44.534 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:44 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: |+.o.. . o o. . | 2024-04-25T14:29:44.534 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:44 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: |+ . . o +... | 2024-04-25T14:29:44.534 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:44 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: |o . . S o ++. .| 2024-04-25T14:29:44.534 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:44 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: | . . o oEo.+ | 2024-04-25T14:29:44.534 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:44 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: | o.o. o=.o| 2024-04-25T14:29:44.534 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:44 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: | oo.o==o.| 2024-04-25T14:29:44.534 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:44 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: | ...o++o++| 2024-04-25T14:29:44.534 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:29:44 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: +----[SHA256]-----+ 2024-04-25T14:29:44.976 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:43.785+0000 7fe449600700 1 Processor -- start 2024-04-25T14:29:44.976 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:43.785+0000 7fe449600700 1 -- start start 2024-04-25T14:29:44.976 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:43.786+0000 7fe449600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe44410a780 0x7fe44410aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:44.976 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:43.786+0000 7fe449600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe44410b170 con 0x7fe44410a780 2024-04-25T14:29:44.977 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:43.786+0000 7fe442a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe44410a780 0x7fe44410aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:44.977 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:43.787+0000 7fe442a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe44410a780 0x7fe44410aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47618/0 (socket says 172.21.15.53:47618) 2024-04-25T14:29:44.977 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:43.787+0000 7fe442a00700 1 -- 172.21.15.53:0/1715691673 learned_addr learned my addr 172.21.15.53:0/1715691673 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:29:44.977 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:43.787+0000 7fe442a00700 1 -- 172.21.15.53:0/1715691673 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe44410b930 con 0x7fe44410a780 2024-04-25T14:29:44.977 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:43.787+0000 7fe442a00700 1 --2- 172.21.15.53:0/1715691673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe44410a780 0x7fe44410aba0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe4340098a0 tx=0x7fe434009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de39ad9440829edd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:44.979 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:43.788+0000 7fe441600700 1 -- 172.21.15.53:0/1715691673 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe434004160 con 0x7fe44410a780 2024-04-25T14:29:44.979 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:43.788+0000 7fe441600700 1 -- 172.21.15.53:0/1715691673 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe434004300 con 0x7fe44410a780 2024-04-25T14:29:44.979 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:43.788+0000 7fe441600700 1 -- 172.21.15.53:0/1715691673 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe434011af0 con 0x7fe44410a780 2024-04-25T14:29:44.979 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:43.788+0000 7fe449600700 1 -- 172.21.15.53:0/1715691673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe44410a780 msgr2=0x7fe44410aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:44.980 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:43.788+0000 7fe449600700 1 --2- 172.21.15.53:0/1715691673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe44410a780 0x7fe44410aba0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe4340098a0 tx=0x7fe434009bb0 comp rx=0 tx=0).stop 2024-04-25T14:29:44.980 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:43.788+0000 7fe449600700 1 -- 172.21.15.53:0/1715691673 shutdown_connections 2024-04-25T14:29:44.980 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:43.788+0000 7fe449600700 1 --2- 172.21.15.53:0/1715691673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe44410a780 0x7fe44410aba0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:44.980 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:43.788+0000 7fe449600700 1 -- 172.21.15.53:0/1715691673 >> 172.21.15.53:0/1715691673 conn(0x7fe444105d40 msgr2=0x7fe444108160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:29:44.980 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:43.789+0000 7fe449600700 1 -- 172.21.15.53:0/1715691673 shutdown_connections 2024-04-25T14:29:44.980 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:43.789+0000 7fe449600700 1 -- 172.21.15.53:0/1715691673 wait complete. 2024-04-25T14:29:44.980 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:43.790+0000 7fe449600700 1 Processor -- start 2024-04-25T14:29:44.980 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:43.790+0000 7fe449600700 1 -- start start 2024-04-25T14:29:44.980 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:43.790+0000 7fe449600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe44410a780 0x7fe44419e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:44.980 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:43.790+0000 7fe449600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe44410ece0 con 0x7fe44410a780 2024-04-25T14:29:44.980 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:43.791+0000 7fe442a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe44410a780 0x7fe44419e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:44.980 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:43.791+0000 7fe442a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe44410a780 0x7fe44419e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47628/0 (socket says 172.21.15.53:47628) 2024-04-25T14:29:44.980 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:43.791+0000 7fe442a00700 1 -- 172.21.15.53:0/1307592640 learned_addr learned my addr 172.21.15.53:0/1307592640 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:29:44.980 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:43.791+0000 7fe442a00700 1 -- 172.21.15.53:0/1307592640 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe44419ebc0 con 0x7fe44410a780 2024-04-25T14:29:44.981 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:43.791+0000 7fe442a00700 1 --2- 172.21.15.53:0/1307592640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe44410a780 0x7fe44419e680 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe434000c00 tx=0x7fe434011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:44.981 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:43.792+0000 7fe43b400700 1 -- 172.21.15.53:0/1307592640 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe434003b20 con 0x7fe44410a780 2024-04-25T14:29:44.981 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:43.792+0000 7fe43b400700 1 -- 172.21.15.53:0/1307592640 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe43401a8c0 con 0x7fe44410a780 2024-04-25T14:29:44.983 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:43.792+0000 7fe449600700 1 -- 172.21.15.53:0/1307592640 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe44419ee50 con 0x7fe44410a780 2024-04-25T14:29:44.983 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:43.792+0000 7fe43b400700 1 -- 172.21.15.53:0/1307592640 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe434024940 con 0x7fe44410a780 2024-04-25T14:29:44.983 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:43.792+0000 7fe449600700 1 -- 172.21.15.53:0/1307592640 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4441a1b40 con 0x7fe44410a780 2024-04-25T14:29:44.983 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:43.793+0000 7fe449600700 1 -- 172.21.15.53:0/1307592640 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4440640a0 con 0x7fe44410a780 2024-04-25T14:29:44.983 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:43.793+0000 7fe43b400700 1 -- 172.21.15.53:0/1307592640 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fe434022020 con 0x7fe44410a780 2024-04-25T14:29:44.983 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:43.794+0000 7fe43b400700 1 --2- 172.21.15.53:0/1307592640 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7fe4280395b0 0x7fe42803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:44.984 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:43.794+0000 7fe43b400700 1 -- 172.21.15.53:0/1307592640 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe43404d8a0 con 0x7fe44410a780 2024-04-25T14:29:44.984 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:43.794+0000 7fe442000700 1 --2- 172.21.15.53:0/1307592640 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7fe4280395b0 0x7fe42803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:44.984 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:43.795+0000 7fe442000700 1 --2- 172.21.15.53:0/1307592640 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7fe4280395b0 0x7fe42803ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe430006fd0 tx=0x7fe430006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:44.984 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:43.799+0000 7fe43b400700 1 -- 172.21.15.53:0/1307592640 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe43400f2e0 con 0x7fe44410a780 2024-04-25T14:29:44.984 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:44.037+0000 7fe449600700 1 -- 172.21.15.53:0/1307592640 --> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fe444106ef0 con 0x7fe4280395b0 2024-04-25T14:29:44.984 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:44.202+0000 7fe43b400700 1 -- 172.21.15.53:0/1307592640 <== mgr.14120 v2:172.21.15.53:6800/3887954889 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe444106ef0 con 0x7fe4280395b0 2024-04-25T14:29:44.984 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:44.206+0000 7fe449600700 1 -- 172.21.15.53:0/1307592640 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7fe4280395b0 msgr2=0x7fe42803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:44.984 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:44.206+0000 7fe449600700 1 --2- 172.21.15.53:0/1307592640 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7fe4280395b0 0x7fe42803ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe430006fd0 tx=0x7fe430006e40 comp rx=0 tx=0).stop 2024-04-25T14:29:44.984 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:44.206+0000 7fe449600700 1 -- 172.21.15.53:0/1307592640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe44410a780 msgr2=0x7fe44419e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:44.984 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:44.206+0000 7fe449600700 1 --2- 172.21.15.53:0/1307592640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe44410a780 0x7fe44419e680 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe434000c00 tx=0x7fe434011b70 comp rx=0 tx=0).stop 2024-04-25T14:29:44.985 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:44.207+0000 7fe449600700 1 -- 172.21.15.53:0/1307592640 shutdown_connections 2024-04-25T14:29:44.985 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:44.207+0000 7fe449600700 1 --2- 172.21.15.53:0/1307592640 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7fe4280395b0 0x7fe42803ba70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:44.985 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:44.207+0000 7fe449600700 1 --2- 172.21.15.53:0/1307592640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe44410a780 0x7fe44419e680 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:44.985 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:44.207+0000 7fe449600700 1 -- 172.21.15.53:0/1307592640 >> 172.21.15.53:0/1307592640 conn(0x7fe444105d40 msgr2=0x7fe4441067e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:29:44.985 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:44.207+0000 7fe449600700 1 -- 172.21.15.53:0/1307592640 shutdown_connections 2024-04-25T14:29:44.985 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:44.207+0000 7fe449600700 1 -- 172.21.15.53:0/1307592640 wait complete. 2024-04-25T14:29:45.266 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:45 smithi053 ceph-mon[27773]: from='mgr.14120 172.21.15.53:0/2278703669' entity='mgr.x' 2024-04-25T14:29:45.267 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:45 smithi053 ceph-mon[27773]: from='mgr.14120 172.21.15.53:0/2278703669' entity='mgr.x' 2024-04-25T14:29:46.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:46 smithi053 ceph-mon[27773]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:29:46.667 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrCLUe/I7I53ZLzB43ipFc6lpRBE7ADQ18JN0n9Fsf1x8tiaJkmaNaFVBeK2GDjmE1b65JnV4cyo8PBNntqrUH0f8fi5xx0is/r5OuMN5o34dnu3wBI4VqTonNTnAFKp6eT9pIu3vSFr8uy4Yi9Sk0QuGgi6JFNgudd+yGNNt7KUOGYH+PmMiC8IjCr6wyX27zAZXc24ldDsuC4f4Kl80zL0c5LtyCeLTAtT8e0jcJb9MUJCOif7gjUYIMPNizVoa/fdWMpjCZqFBgxEFQWJtY7lDk5xoWU+0E0h4eZDD5xOBCb9iR9x2QLzWZE7C/AkhmZ8XntHapXlxTlrVx8UuBLz5WE8ptcPULceUd2z3XEB1xtv9S48Jm+rS2gVGnVTqjIuMrvrqvoqK430OpSTKmhveepVmJkPMbmZkO4fu3SViLP5/Jl9/u36MPLGng2shnTp4TqgvcibofLE09iTImto7YiBG1ub6ASmlTnV+nKSKSRlxwGh8L8vyApLL+tHE= ceph-fd30e6ce-030f-11ef-bc93-c7b262605968 2024-04-25T14:29:46.673 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.678+0000 7f2db0200700 1 Processor -- start 2024-04-25T14:29:46.673 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.679+0000 7f2db0200700 1 -- start start 2024-04-25T14:29:46.673 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.679+0000 7f2db0200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2da807cda0 0x7f2da807b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:46.673 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.679+0000 7f2db0200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2da807b7d0 con 0x7f2da807cda0 2024-04-25T14:29:46.673 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.680+0000 7f2dad800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2da807cda0 0x7f2da807b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:46.673 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.680+0000 7f2dad800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2da807cda0 0x7f2da807b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47640/0 (socket says 172.21.15.53:47640) 2024-04-25T14:29:46.673 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.680+0000 7f2dad800700 1 -- 172.21.15.53:0/3040739355 learned_addr learned my addr 172.21.15.53:0/3040739355 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:29:46.673 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.680+0000 7f2dad800700 1 -- 172.21.15.53:0/3040739355 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da807b950 con 0x7f2da807cda0 2024-04-25T14:29:46.674 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.681+0000 7f2dad800700 1 --2- 172.21.15.53:0/3040739355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2da807cda0 0x7f2da807b200 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c0098a0 tx=0x7f2d9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2f9aa875e711462 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:46.674 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.681+0000 7f2da7e00700 1 -- 172.21.15.53:0/3040739355 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d9c004160 con 0x7f2da807cda0 2024-04-25T14:29:46.674 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.681+0000 7f2da7e00700 1 -- 172.21.15.53:0/3040739355 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2d9c004300 con 0x7f2da807cda0 2024-04-25T14:29:46.674 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.681+0000 7f2da7e00700 1 -- 172.21.15.53:0/3040739355 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d9c011af0 con 0x7f2da807cda0 2024-04-25T14:29:46.674 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.682+0000 7f2db0200700 1 -- 172.21.15.53:0/3040739355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2da807cda0 msgr2=0x7f2da807b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:46.674 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.682+0000 7f2db0200700 1 --2- 172.21.15.53:0/3040739355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2da807cda0 0x7f2da807b200 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c0098a0 tx=0x7f2d9c009bb0 comp rx=0 tx=0).stop 2024-04-25T14:29:46.674 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.682+0000 7f2db0200700 1 -- 172.21.15.53:0/3040739355 shutdown_connections 2024-04-25T14:29:46.674 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.682+0000 7f2db0200700 1 --2- 172.21.15.53:0/3040739355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2da807cda0 0x7f2da807b200 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:46.674 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.682+0000 7f2db0200700 1 -- 172.21.15.53:0/3040739355 >> 172.21.15.53:0/3040739355 conn(0x7f2da8103b50 msgr2=0x7f2da8105f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:29:46.674 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.682+0000 7f2db0200700 1 -- 172.21.15.53:0/3040739355 shutdown_connections 2024-04-25T14:29:46.674 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.682+0000 7f2db0200700 1 -- 172.21.15.53:0/3040739355 wait complete. 2024-04-25T14:29:46.674 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.683+0000 7f2db0200700 1 Processor -- start 2024-04-25T14:29:46.674 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.683+0000 7f2db0200700 1 -- start start 2024-04-25T14:29:46.675 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.683+0000 7f2db0200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2da807cda0 0x7f2da819e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:46.675 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.683+0000 7f2db0200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2da810eae0 con 0x7f2da807cda0 2024-04-25T14:29:46.675 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.684+0000 7f2dad800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2da807cda0 0x7f2da819e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:46.675 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.684+0000 7f2dad800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2da807cda0 0x7f2da819e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47656/0 (socket says 172.21.15.53:47656) 2024-04-25T14:29:46.675 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.684+0000 7f2dad800700 1 -- 172.21.15.53:0/1003772532 learned_addr learned my addr 172.21.15.53:0/1003772532 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:29:46.675 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.684+0000 7f2dad800700 1 -- 172.21.15.53:0/1003772532 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da819ebe0 con 0x7f2da807cda0 2024-04-25T14:29:46.675 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.684+0000 7f2dad800700 1 --2- 172.21.15.53:0/1003772532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2da807cda0 0x7f2da819e6a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c000c00 tx=0x7f2d9c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:46.675 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.685+0000 7f2da6000700 1 -- 172.21.15.53:0/1003772532 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d9c003b20 con 0x7f2da807cda0 2024-04-25T14:29:46.675 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.685+0000 7f2da6000700 1 -- 172.21.15.53:0/1003772532 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2d9c01a8c0 con 0x7f2da807cda0 2024-04-25T14:29:46.675 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.685+0000 7f2db0200700 1 -- 172.21.15.53:0/1003772532 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2da819ee70 con 0x7f2da807cda0 2024-04-25T14:29:46.675 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.685+0000 7f2da6000700 1 -- 172.21.15.53:0/1003772532 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d9c024940 con 0x7f2da807cda0 2024-04-25T14:29:46.675 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.685+0000 7f2db0200700 1 -- 172.21.15.53:0/1003772532 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2da81a1b60 con 0x7f2da807cda0 2024-04-25T14:29:46.675 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.686+0000 7f2db0200700 1 -- 172.21.15.53:0/1003772532 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2da80640a0 con 0x7f2da807cda0 2024-04-25T14:29:46.676 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.686+0000 7f2da6000700 1 -- 172.21.15.53:0/1003772532 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f2d9c022020 con 0x7f2da807cda0 2024-04-25T14:29:46.676 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.686+0000 7f2da6000700 1 --2- 172.21.15.53:0/1003772532 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f2d90039600 0x7f2d9003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:46.676 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.687+0000 7f2da6000700 1 -- 172.21.15.53:0/1003772532 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2d9c04d8a0 con 0x7f2da807cda0 2024-04-25T14:29:46.676 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.687+0000 7f2dace00700 1 --2- 172.21.15.53:0/1003772532 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f2d90039600 0x7f2d9003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:46.676 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.687+0000 7f2dace00700 1 --2- 172.21.15.53:0/1003772532 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f2d90039600 0x7f2d9003bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2d94006fd0 tx=0x7f2d94006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:46.676 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.692+0000 7f2da6000700 1 -- 172.21.15.53:0/1003772532 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2d9c00be50 con 0x7f2da807cda0 2024-04-25T14:29:46.676 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.925+0000 7f2db0200700 1 -- 172.21.15.53:0/1003772532 --> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f2da8104d70 con 0x7f2d90039600 2024-04-25T14:29:46.676 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.927+0000 7f2da6000700 1 -- 172.21.15.53:0/1003772532 <== mgr.14120 v2:172.21.15.53:6800/3887954889 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f2da8104d70 con 0x7f2d90039600 2024-04-25T14:29:46.676 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.929+0000 7f2db0200700 1 -- 172.21.15.53:0/1003772532 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f2d90039600 msgr2=0x7f2d9003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:46.676 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.929+0000 7f2db0200700 1 --2- 172.21.15.53:0/1003772532 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f2d90039600 0x7f2d9003bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2d94006fd0 tx=0x7f2d94006e40 comp rx=0 tx=0).stop 2024-04-25T14:29:46.676 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.929+0000 7f2db0200700 1 -- 172.21.15.53:0/1003772532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2da807cda0 msgr2=0x7f2da819e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:46.676 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.929+0000 7f2db0200700 1 --2- 172.21.15.53:0/1003772532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2da807cda0 0x7f2da819e6a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c000c00 tx=0x7f2d9c011b70 comp rx=0 tx=0).stop 2024-04-25T14:29:46.677 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.930+0000 7f2db0200700 1 -- 172.21.15.53:0/1003772532 shutdown_connections 2024-04-25T14:29:46.677 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.930+0000 7f2db0200700 1 --2- 172.21.15.53:0/1003772532 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f2d90039600 0x7f2d9003bac0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:46.677 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.930+0000 7f2db0200700 1 --2- 172.21.15.53:0/1003772532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2da807cda0 0x7f2da819e6a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:46.677 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.930+0000 7f2db0200700 1 -- 172.21.15.53:0/1003772532 >> 172.21.15.53:0/1003772532 conn(0x7f2da8103b50 msgr2=0x7f2da8104660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:29:46.677 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.930+0000 7f2db0200700 1 -- 172.21.15.53:0/1003772532 shutdown_connections 2024-04-25T14:29:46.677 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:45.930+0000 7f2db0200700 1 -- 172.21.15.53:0/1003772532 wait complete. 2024-04-25T14:29:46.677 INFO:teuthology.orchestra.run.smithi053.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-25T14:29:46.678 INFO:teuthology.orchestra.run.smithi053.stdout:Adding key to root@localhost authorized_keys... 2024-04-25T14:29:46.682 INFO:teuthology.orchestra.run.smithi053.stdout:Adding host smithi053... 2024-04-25T14:29:48.376 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:48 smithi053 ceph-mon[27773]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi053", "addr": "172.21.15.53", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:29:49.271 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:49 smithi053 ceph-mon[27773]: Deploying cephadm binary to smithi053 2024-04-25T14:29:54.614 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout Added host 'smithi053' with addr '172.21.15.53' 2024-04-25T14:29:54.614 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:47.352+0000 7f123e200700 1 Processor -- start 2024-04-25T14:29:54.615 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:47.352+0000 7f123e200700 1 -- start start 2024-04-25T14:29:54.615 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:47.353+0000 7f123e200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1238110220 0x7f1238110640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:54.615 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:47.353+0000 7f123e200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1238110c10 con 0x7f1238110220 2024-04-25T14:29:54.615 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:47.353+0000 7f1237400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1238110220 0x7f1238110640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:54.615 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:47.353+0000 7f1237400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1238110220 0x7f1238110640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47664/0 (socket says 172.21.15.53:47664) 2024-04-25T14:29:54.615 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:47.353+0000 7f1237400700 1 -- 172.21.15.53:0/575153273 learned_addr learned my addr 172.21.15.53:0/575153273 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:29:54.615 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:47.354+0000 7f1237400700 1 -- 172.21.15.53:0/575153273 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f123810ece0 con 0x7f1238110220 2024-04-25T14:29:54.615 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:47.354+0000 7f1237400700 1 --2- 172.21.15.53:0/575153273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1238110220 0x7f1238110640 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f12280098a0 tx=0x7f1228009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8effd3043ef06961 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:54.615 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:47.354+0000 7f1236000700 1 -- 172.21.15.53:0/575153273 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1228004160 con 0x7f1238110220 2024-04-25T14:29:54.615 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:47.354+0000 7f1236000700 1 -- 172.21.15.53:0/575153273 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1228004300 con 0x7f1238110220 2024-04-25T14:29:54.615 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:47.354+0000 7f1236000700 1 -- 172.21.15.53:0/575153273 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1228011af0 con 0x7f1238110220 2024-04-25T14:29:54.615 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:47.355+0000 7f123e200700 1 -- 172.21.15.53:0/575153273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1238110220 msgr2=0x7f1238110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:54.615 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:47.355+0000 7f123e200700 1 --2- 172.21.15.53:0/575153273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1238110220 0x7f1238110640 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f12280098a0 tx=0x7f1228009bb0 comp rx=0 tx=0).stop 2024-04-25T14:29:54.616 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:47.355+0000 7f123e200700 1 -- 172.21.15.53:0/575153273 shutdown_connections 2024-04-25T14:29:54.616 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:47.355+0000 7f123e200700 1 --2- 172.21.15.53:0/575153273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1238110220 0x7f1238110640 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:54.616 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:47.355+0000 7f123e200700 1 -- 172.21.15.53:0/575153273 >> 172.21.15.53:0/575153273 conn(0x7f123807de20 msgr2=0x7f1238080240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:29:54.616 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:47.355+0000 7f123e200700 1 -- 172.21.15.53:0/575153273 shutdown_connections 2024-04-25T14:29:54.616 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:47.355+0000 7f123e200700 1 -- 172.21.15.53:0/575153273 wait complete. 2024-04-25T14:29:54.616 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:47.356+0000 7f123e200700 1 Processor -- start 2024-04-25T14:29:54.616 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:47.357+0000 7f123e200700 1 -- start start 2024-04-25T14:29:54.616 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:47.357+0000 7f123e200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1238110220 0x7f12381a2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:54.616 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:47.357+0000 7f123e200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1238113140 con 0x7f1238110220 2024-04-25T14:29:54.616 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:47.357+0000 7f1237400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1238110220 0x7f12381a2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:54.616 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:47.357+0000 7f1237400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1238110220 0x7f12381a2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47676/0 (socket says 172.21.15.53:47676) 2024-04-25T14:29:54.616 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:47.357+0000 7f1237400700 1 -- 172.21.15.53:0/3485496901 learned_addr learned my addr 172.21.15.53:0/3485496901 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:29:54.616 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:47.358+0000 7f1237400700 1 -- 172.21.15.53:0/3485496901 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12381a3160 con 0x7f1238110220 2024-04-25T14:29:54.616 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:47.358+0000 7f1237400700 1 --2- 172.21.15.53:0/3485496901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1238110220 0x7f12381a2c20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f12280117b0 tx=0x7f1228011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:54.616 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:47.358+0000 7f122fe00700 1 -- 172.21.15.53:0/3485496901 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1228003af0 con 0x7f1238110220 2024-04-25T14:29:54.617 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:47.358+0000 7f122fe00700 1 -- 172.21.15.53:0/3485496901 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1228020070 con 0x7f1238110220 2024-04-25T14:29:54.617 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:47.358+0000 7f122fe00700 1 -- 172.21.15.53:0/3485496901 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1228019630 con 0x7f1238110220 2024-04-25T14:29:54.617 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:47.358+0000 7f123e200700 1 -- 172.21.15.53:0/3485496901 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12381a33f0 con 0x7f1238110220 2024-04-25T14:29:54.617 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:47.358+0000 7f123e200700 1 -- 172.21.15.53:0/3485496901 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12381a60e0 con 0x7f1238110220 2024-04-25T14:29:54.617 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:47.359+0000 7f123e200700 1 -- 172.21.15.53:0/3485496901 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12380640a0 con 0x7f1238110220 2024-04-25T14:29:54.617 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:47.360+0000 7f122fe00700 1 -- 172.21.15.53:0/3485496901 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f122801a9c0 con 0x7f1238110220 2024-04-25T14:29:54.617 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:47.360+0000 7f122fe00700 1 --2- 172.21.15.53:0/3485496901 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f121c0395b0 0x7f121c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:54.617 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:47.360+0000 7f122fe00700 1 -- 172.21.15.53:0/3485496901 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f122804dc00 con 0x7f1238110220 2024-04-25T14:29:54.617 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:47.360+0000 7f1236a00700 1 --2- 172.21.15.53:0/3485496901 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f121c0395b0 0x7f121c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:54.617 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:47.361+0000 7f1236a00700 1 --2- 172.21.15.53:0/3485496901 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f121c0395b0 0x7f121c03ba70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1224006fd0 tx=0x7f1224006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:54.617 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:47.366+0000 7f122fe00700 1 -- 172.21.15.53:0/3485496901 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f122800e260 con 0x7f1238110220 2024-04-25T14:29:54.617 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:47.599+0000 7f123e200700 1 -- 172.21.15.53:0/3485496901 --> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi053", "addr": "172.21.15.53", "target": ["mon-mgr", ""]}) -- 0x7f12380800b0 con 0x7f121c0395b0 2024-04-25T14:29:54.617 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:53.861+0000 7f122fe00700 1 -- 172.21.15.53:0/3485496901 <== mgr.14120 v2:172.21.15.53:6800/3887954889 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f12380800b0 con 0x7f121c0395b0 2024-04-25T14:29:54.617 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:53.865+0000 7f123e200700 1 -- 172.21.15.53:0/3485496901 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f121c0395b0 msgr2=0x7f121c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:54.618 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:53.865+0000 7f123e200700 1 --2- 172.21.15.53:0/3485496901 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f121c0395b0 0x7f121c03ba70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1224006fd0 tx=0x7f1224006e40 comp rx=0 tx=0).stop 2024-04-25T14:29:54.618 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:53.865+0000 7f123e200700 1 -- 172.21.15.53:0/3485496901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1238110220 msgr2=0x7f12381a2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:54.618 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:53.865+0000 7f123e200700 1 --2- 172.21.15.53:0/3485496901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1238110220 0x7f12381a2c20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f12280117b0 tx=0x7f1228011b70 comp rx=0 tx=0).stop 2024-04-25T14:29:54.618 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:53.866+0000 7f123e200700 1 -- 172.21.15.53:0/3485496901 shutdown_connections 2024-04-25T14:29:54.618 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:53.866+0000 7f123e200700 1 --2- 172.21.15.53:0/3485496901 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f121c0395b0 0x7f121c03ba70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:54.618 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:53.866+0000 7f123e200700 1 --2- 172.21.15.53:0/3485496901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1238110220 0x7f12381a2c20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:54.618 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:53.866+0000 7f123e200700 1 -- 172.21.15.53:0/3485496901 >> 172.21.15.53:0/3485496901 conn(0x7f123807de20 msgr2=0x7f123807f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:29:54.618 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:53.866+0000 7f123e200700 1 -- 172.21.15.53:0/3485496901 shutdown_connections 2024-04-25T14:29:54.618 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:53.866+0000 7f123e200700 1 -- 172.21.15.53:0/3485496901 wait complete. 2024-04-25T14:29:54.618 INFO:teuthology.orchestra.run.smithi053.stdout:Deploying unmanaged mon service... 2024-04-25T14:29:55.156 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:54 smithi053 ceph-mon[27773]: from='mgr.14120 172.21.15.53:0/2278703669' entity='mgr.x' 2024-04-25T14:29:55.156 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:54 smithi053 ceph-mon[27773]: Added host smithi053 2024-04-25T14:29:55.156 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:54 smithi053 ceph-mon[27773]: from='mgr.14120 172.21.15.53:0/2278703669' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:29:56.834 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-25T14:29:56.834 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.328+0000 7f6079000700 1 Processor -- start 2024-04-25T14:29:56.834 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.329+0000 7f6079000700 1 -- start start 2024-04-25T14:29:56.834 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.329+0000 7f6079000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6074108b20 0x7f6074108f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:56.834 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.329+0000 7f6079000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6074107060 con 0x7f6074108b20 2024-04-25T14:29:56.834 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.330+0000 7f6071e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6074108b20 0x7f6074108f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:56.834 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.330+0000 7f6071e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6074108b20 0x7f6074108f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38770/0 (socket says 172.21.15.53:38770) 2024-04-25T14:29:56.834 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.330+0000 7f6071e00700 1 -- 172.21.15.53:0/4273458735 learned_addr learned my addr 172.21.15.53:0/4273458735 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:29:56.834 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.331+0000 7f6071e00700 1 -- 172.21.15.53:0/4273458735 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60741071e0 con 0x7f6074108b20 2024-04-25T14:29:56.834 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.331+0000 7f6071e00700 1 --2- 172.21.15.53:0/4273458735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6074108b20 0x7f6074108f40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f60640098a0 tx=0x7f6064009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac42377e2d8c424e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:56.834 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.331+0000 7f6070a00700 1 -- 172.21.15.53:0/4273458735 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6064004160 con 0x7f6074108b20 2024-04-25T14:29:56.835 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.331+0000 7f6070a00700 1 -- 172.21.15.53:0/4273458735 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6064004300 con 0x7f6074108b20 2024-04-25T14:29:56.835 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.331+0000 7f6070a00700 1 -- 172.21.15.53:0/4273458735 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6064011af0 con 0x7f6074108b20 2024-04-25T14:29:56.835 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.332+0000 7f6079000700 1 -- 172.21.15.53:0/4273458735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6074108b20 msgr2=0x7f6074108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:56.835 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.332+0000 7f6079000700 1 --2- 172.21.15.53:0/4273458735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6074108b20 0x7f6074108f40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f60640098a0 tx=0x7f6064009bb0 comp rx=0 tx=0).stop 2024-04-25T14:29:56.835 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.332+0000 7f6079000700 1 -- 172.21.15.53:0/4273458735 shutdown_connections 2024-04-25T14:29:56.835 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.332+0000 7f6079000700 1 --2- 172.21.15.53:0/4273458735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6074108b20 0x7f6074108f40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:56.835 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.332+0000 7f6079000700 1 -- 172.21.15.53:0/4273458735 >> 172.21.15.53:0/4273458735 conn(0x7f6074102a60 msgr2=0x7f6074104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:29:56.835 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.332+0000 7f6079000700 1 -- 172.21.15.53:0/4273458735 shutdown_connections 2024-04-25T14:29:56.835 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.332+0000 7f6079000700 1 -- 172.21.15.53:0/4273458735 wait complete. 2024-04-25T14:29:56.835 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.333+0000 7f6079000700 1 Processor -- start 2024-04-25T14:29:56.835 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.333+0000 7f6079000700 1 -- start start 2024-04-25T14:29:56.835 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.334+0000 7f6079000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6074108b20 0x7f6074196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:56.835 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.334+0000 7f6079000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f607410b9f0 con 0x7f6074108b20 2024-04-25T14:29:56.835 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.334+0000 7f6071e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6074108b20 0x7f6074196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:56.836 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.334+0000 7f6071e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6074108b20 0x7f6074196680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38780/0 (socket says 172.21.15.53:38780) 2024-04-25T14:29:56.836 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.334+0000 7f6071e00700 1 -- 172.21.15.53:0/3956849516 learned_addr learned my addr 172.21.15.53:0/3956849516 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:29:56.836 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.334+0000 7f6071e00700 1 -- 172.21.15.53:0/3956849516 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6074196bc0 con 0x7f6074108b20 2024-04-25T14:29:56.836 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.335+0000 7f6071e00700 1 --2- 172.21.15.53:0/3956849516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6074108b20 0x7f6074196680 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6064009fb0 tx=0x7f6064011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:56.836 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.335+0000 7f606aa00700 1 -- 172.21.15.53:0/3956849516 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60640036a0 con 0x7f6074108b20 2024-04-25T14:29:56.836 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.335+0000 7f606aa00700 1 -- 172.21.15.53:0/3956849516 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f606401a430 con 0x7f6074108b20 2024-04-25T14:29:56.836 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.335+0000 7f606aa00700 1 -- 172.21.15.53:0/3956849516 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6064019550 con 0x7f6074108b20 2024-04-25T14:29:56.836 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.335+0000 7f6079000700 1 -- 172.21.15.53:0/3956849516 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6074196e50 con 0x7f6074108b20 2024-04-25T14:29:56.836 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.335+0000 7f6079000700 1 -- 172.21.15.53:0/3956849516 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6074193410 con 0x7f6074108b20 2024-04-25T14:29:56.836 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.336+0000 7f606aa00700 1 -- 172.21.15.53:0/3956849516 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f606401a8e0 con 0x7f6074108b20 2024-04-25T14:29:56.836 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.336+0000 7f6079000700 1 -- 172.21.15.53:0/3956849516 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f607410bb70 con 0x7f6074108b20 2024-04-25T14:29:56.836 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.337+0000 7f606aa00700 1 --2- 172.21.15.53:0/3956849516 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f60580395b0 0x7f605803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:56.836 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.337+0000 7f606aa00700 1 -- 172.21.15.53:0/3956849516 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f606404d8d0 con 0x7f6074108b20 2024-04-25T14:29:56.836 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.337+0000 7f6071400700 1 --2- 172.21.15.53:0/3956849516 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f60580395b0 0x7f605803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:56.836 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.338+0000 7f6071400700 1 --2- 172.21.15.53:0/3956849516 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f60580395b0 0x7f605803ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6060006fd0 tx=0x7f6060006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:56.837 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.344+0000 7f606aa00700 1 -- 172.21.15.53:0/3956849516 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f606401a690 con 0x7f6074108b20 2024-04-25T14:29:56.837 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.608+0000 7f6079000700 1 -- 172.21.15.53:0/3956849516 --> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f6074198580 con 0x7f60580395b0 2024-04-25T14:29:56.837 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.618+0000 7f606aa00700 1 -- 172.21.15.53:0/3956849516 <== mgr.14120 v2:172.21.15.53:6800/3887954889 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6074198580 con 0x7f60580395b0 2024-04-25T14:29:56.837 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.622+0000 7f6079000700 1 -- 172.21.15.53:0/3956849516 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f60580395b0 msgr2=0x7f605803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:56.837 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.622+0000 7f6079000700 1 --2- 172.21.15.53:0/3956849516 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f60580395b0 0x7f605803ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6060006fd0 tx=0x7f6060006e40 comp rx=0 tx=0).stop 2024-04-25T14:29:56.837 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.622+0000 7f6079000700 1 -- 172.21.15.53:0/3956849516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6074108b20 msgr2=0x7f6074196680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:56.837 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.622+0000 7f6079000700 1 --2- 172.21.15.53:0/3956849516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6074108b20 0x7f6074196680 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6064009fb0 tx=0x7f6064011940 comp rx=0 tx=0).stop 2024-04-25T14:29:56.837 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.622+0000 7f6079000700 1 -- 172.21.15.53:0/3956849516 shutdown_connections 2024-04-25T14:29:56.837 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.622+0000 7f6079000700 1 --2- 172.21.15.53:0/3956849516 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f60580395b0 0x7f605803ba70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:56.837 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.622+0000 7f6079000700 1 --2- 172.21.15.53:0/3956849516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6074108b20 0x7f6074196680 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:56.837 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.622+0000 7f6079000700 1 -- 172.21.15.53:0/3956849516 >> 172.21.15.53:0/3956849516 conn(0x7f6074102a60 msgr2=0x7f60741039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:29:56.837 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.622+0000 7f6079000700 1 -- 172.21.15.53:0/3956849516 shutdown_connections 2024-04-25T14:29:56.837 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:55.622+0000 7f6079000700 1 -- 172.21.15.53:0/3956849516 wait complete. 2024-04-25T14:29:56.837 INFO:teuthology.orchestra.run.smithi053.stdout:Deploying unmanaged mgr service... 2024-04-25T14:29:56.870 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:56 smithi053 ceph-mon[27773]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:29:56.870 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:56 smithi053 ceph-mon[27773]: Saving service mon spec with placement count:5 2024-04-25T14:29:56.870 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:56 smithi053 ceph-mon[27773]: from='mgr.14120 172.21.15.53:0/2278703669' entity='mgr.x' 2024-04-25T14:29:59.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:58 smithi053 ceph-mon[27773]: from='mgr.14120 172.21.15.53:0/2278703669' entity='mgr.x' 2024-04-25T14:29:59.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:58 smithi053 ceph-mon[27773]: from='mgr.14120 172.21.15.53:0/2278703669' entity='mgr.x' 2024-04-25T14:29:59.192 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-25T14:29:59.192 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.087+0000 7f8e03600700 1 Processor -- start 2024-04-25T14:29:59.192 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.087+0000 7f8e03600700 1 -- start start 2024-04-25T14:29:59.193 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.088+0000 7f8e03600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dfc07dea0 0x7f8dfc07c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:59.193 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.088+0000 7f8e03600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc07c920 con 0x7f8dfc07dea0 2024-04-25T14:29:59.193 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.088+0000 7f8e00c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dfc07dea0 0x7f8dfc07c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:59.193 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.088+0000 7f8e00c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dfc07dea0 0x7f8dfc07c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38794/0 (socket says 172.21.15.53:38794) 2024-04-25T14:29:59.193 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.088+0000 7f8e00c00700 1 -- 172.21.15.53:0/1171153731 learned_addr learned my addr 172.21.15.53:0/1171153731 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:29:59.193 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.089+0000 7f8e00c00700 1 -- 172.21.15.53:0/1171153731 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dfc07caa0 con 0x7f8dfc07dea0 2024-04-25T14:29:59.193 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.089+0000 7f8e00c00700 1 --2- 172.21.15.53:0/1171153731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dfc07dea0 0x7f8dfc07c350 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8df00098a0 tx=0x7f8df0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab7cc8786cd4262 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:59.193 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.090+0000 7f8dfb400700 1 -- 172.21.15.53:0/1171153731 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8df0004160 con 0x7f8dfc07dea0 2024-04-25T14:29:59.193 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.090+0000 7f8dfb400700 1 -- 172.21.15.53:0/1171153731 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8df0004300 con 0x7f8dfc07dea0 2024-04-25T14:29:59.193 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.090+0000 7f8dfb400700 1 -- 172.21.15.53:0/1171153731 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8df0011a20 con 0x7f8dfc07dea0 2024-04-25T14:29:59.193 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.090+0000 7f8e03600700 1 -- 172.21.15.53:0/1171153731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dfc07dea0 msgr2=0x7f8dfc07c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:59.193 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.090+0000 7f8e03600700 1 --2- 172.21.15.53:0/1171153731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dfc07dea0 0x7f8dfc07c350 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8df00098a0 tx=0x7f8df0009bb0 comp rx=0 tx=0).stop 2024-04-25T14:29:59.193 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.090+0000 7f8e03600700 1 -- 172.21.15.53:0/1171153731 shutdown_connections 2024-04-25T14:29:59.193 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.091+0000 7f8e03600700 1 --2- 172.21.15.53:0/1171153731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dfc07dea0 0x7f8dfc07c350 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:59.194 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.091+0000 7f8e03600700 1 -- 172.21.15.53:0/1171153731 >> 172.21.15.53:0/1171153731 conn(0x7f8dfc103b40 msgr2=0x7f8dfc105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:29:59.194 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.091+0000 7f8e03600700 1 -- 172.21.15.53:0/1171153731 shutdown_connections 2024-04-25T14:29:59.194 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.091+0000 7f8e03600700 1 -- 172.21.15.53:0/1171153731 wait complete. 2024-04-25T14:29:59.194 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.092+0000 7f8e03600700 1 Processor -- start 2024-04-25T14:29:59.194 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.092+0000 7f8e03600700 1 -- start start 2024-04-25T14:29:59.194 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.092+0000 7f8e03600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dfc07dea0 0x7f8dfc1a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:59.194 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.092+0000 7f8e03600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc10a710 con 0x7f8dfc07dea0 2024-04-25T14:29:59.194 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.093+0000 7f8e00c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dfc07dea0 0x7f8dfc1a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:59.194 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.093+0000 7f8e00c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dfc07dea0 0x7f8dfc1a2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38804/0 (socket says 172.21.15.53:38804) 2024-04-25T14:29:59.194 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.093+0000 7f8e00c00700 1 -- 172.21.15.53:0/970066420 learned_addr learned my addr 172.21.15.53:0/970066420 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:29:59.194 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.093+0000 7f8e00c00700 1 -- 172.21.15.53:0/970066420 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dfc1a2f70 con 0x7f8dfc07dea0 2024-04-25T14:29:59.194 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.094+0000 7f8e00c00700 1 --2- 172.21.15.53:0/970066420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dfc07dea0 0x7f8dfc1a2a30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8df0004660 tx=0x7f8df0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:59.194 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.094+0000 7f8df9600700 1 -- 172.21.15.53:0/970066420 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8df0003aa0 con 0x7f8dfc07dea0 2024-04-25T14:29:59.195 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.094+0000 7f8df9600700 1 -- 172.21.15.53:0/970066420 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8df001b8f0 con 0x7f8dfc07dea0 2024-04-25T14:29:59.195 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.094+0000 7f8df9600700 1 -- 172.21.15.53:0/970066420 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8df0023900 con 0x7f8dfc07dea0 2024-04-25T14:29:59.195 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.094+0000 7f8e03600700 1 -- 172.21.15.53:0/970066420 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dfc1a3200 con 0x7f8dfc07dea0 2024-04-25T14:29:59.195 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.094+0000 7f8e03600700 1 -- 172.21.15.53:0/970066420 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dfc1a5ef0 con 0x7f8dfc07dea0 2024-04-25T14:29:59.195 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.095+0000 7f8df9600700 1 -- 172.21.15.53:0/970066420 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f8df001baa0 con 0x7f8dfc07dea0 2024-04-25T14:29:59.195 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.096+0000 7f8df9600700 1 --2- 172.21.15.53:0/970066420 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f8de4039600 0x7f8de403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:29:59.195 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.096+0000 7f8e03600700 1 -- 172.21.15.53:0/970066420 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dfc0640a0 con 0x7f8dfc07dea0 2024-04-25T14:29:59.195 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.096+0000 7f8df9600700 1 -- 172.21.15.53:0/970066420 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8df004df80 con 0x7f8dfc07dea0 2024-04-25T14:29:59.195 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.096+0000 7f8dfbe00700 1 --2- 172.21.15.53:0/970066420 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f8de4039600 0x7f8de403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:29:59.195 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.097+0000 7f8dfbe00700 1 --2- 172.21.15.53:0/970066420 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f8de4039600 0x7f8de403bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8de8006fd0 tx=0x7f8de8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:29:59.195 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.102+0000 7f8df9600700 1 -- 172.21.15.53:0/970066420 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8df001f030 con 0x7f8dfc07dea0 2024-04-25T14:29:59.195 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.353+0000 7f8e03600700 1 -- 172.21.15.53:0/970066420 --> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f8dfc02e2c0 con 0x7f8de4039600 2024-04-25T14:29:59.195 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.361+0000 7f8df9600700 1 -- 172.21.15.53:0/970066420 <== mgr.14120 v2:172.21.15.53:6800/3887954889 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8dfc02e2c0 con 0x7f8de4039600 2024-04-25T14:29:59.196 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.363+0000 7f8e03600700 1 -- 172.21.15.53:0/970066420 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f8de4039600 msgr2=0x7f8de403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:59.196 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.363+0000 7f8e03600700 1 --2- 172.21.15.53:0/970066420 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f8de4039600 0x7f8de403bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8de8006fd0 tx=0x7f8de8006e40 comp rx=0 tx=0).stop 2024-04-25T14:29:59.196 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.363+0000 7f8e03600700 1 -- 172.21.15.53:0/970066420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dfc07dea0 msgr2=0x7f8dfc1a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:29:59.196 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.363+0000 7f8e03600700 1 --2- 172.21.15.53:0/970066420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dfc07dea0 0x7f8dfc1a2a30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8df0004660 tx=0x7f8df0011aa0 comp rx=0 tx=0).stop 2024-04-25T14:29:59.196 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.363+0000 7f8e03600700 1 -- 172.21.15.53:0/970066420 shutdown_connections 2024-04-25T14:29:59.196 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.363+0000 7f8e03600700 1 --2- 172.21.15.53:0/970066420 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f8de4039600 0x7f8de403bac0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:59.196 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.363+0000 7f8e03600700 1 --2- 172.21.15.53:0/970066420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dfc07dea0 0x7f8dfc1a2a30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:29:59.196 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.363+0000 7f8e03600700 1 -- 172.21.15.53:0/970066420 >> 172.21.15.53:0/970066420 conn(0x7f8dfc103b40 msgr2=0x7f8dfc105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:29:59.196 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.364+0000 7f8e03600700 1 -- 172.21.15.53:0/970066420 shutdown_connections 2024-04-25T14:29:59.196 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:58.364+0000 7f8e03600700 1 -- 172.21.15.53:0/970066420 wait complete. 2024-04-25T14:30:00.085 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:59 smithi053 ceph-mon[27773]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:30:00.085 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:59 smithi053 ceph-mon[27773]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:30:00.085 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:59 smithi053 ceph-mon[27773]: Saving service mgr spec with placement count:2 2024-04-25T14:30:00.085 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:29:59 smithi053 ceph-mon[27773]: from='mgr.14120 172.21.15.53:0/2278703669' entity='mgr.x' 2024-04-25T14:30:01.070 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:59.816+0000 7f38a1600700 1 Processor -- start 2024-04-25T14:30:01.071 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:59.816+0000 7f38a1600700 1 -- start start 2024-04-25T14:30:01.071 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:59.817+0000 7f38a1600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f389c07cda0 0x7f389c07b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:01.071 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:59.817+0000 7f38a1600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f389c07b7d0 con 0x7f389c07cda0 2024-04-25T14:30:01.071 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:59.817+0000 7f389aa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f389c07cda0 0x7f389c07b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:30:01.071 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:59.817+0000 7f389aa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f389c07cda0 0x7f389c07b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38812/0 (socket says 172.21.15.53:38812) 2024-04-25T14:30:01.071 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:59.817+0000 7f389aa00700 1 -- 172.21.15.53:0/813750256 learned_addr learned my addr 172.21.15.53:0/813750256 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:30:01.071 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:59.818+0000 7f389aa00700 1 -- 172.21.15.53:0/813750256 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f389c07b950 con 0x7f389c07cda0 2024-04-25T14:30:01.073 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:59.818+0000 7f389aa00700 1 --2- 172.21.15.53:0/813750256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f389c07cda0 0x7f389c07b200 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f388c0098a0 tx=0x7f388c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96196e2266ea262 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:30:01.073 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:59.818+0000 7f3899600700 1 -- 172.21.15.53:0/813750256 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f388c004160 con 0x7f389c07cda0 2024-04-25T14:30:01.073 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:59.818+0000 7f3899600700 1 -- 172.21.15.53:0/813750256 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f388c004300 con 0x7f389c07cda0 2024-04-25T14:30:01.074 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:59.818+0000 7f3899600700 1 -- 172.21.15.53:0/813750256 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f388c011a20 con 0x7f389c07cda0 2024-04-25T14:30:01.074 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:59.819+0000 7f38a1600700 1 -- 172.21.15.53:0/813750256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f389c07cda0 msgr2=0x7f389c07b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:01.074 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:59.819+0000 7f38a1600700 1 --2- 172.21.15.53:0/813750256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f389c07cda0 0x7f389c07b200 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f388c0098a0 tx=0x7f388c009bb0 comp rx=0 tx=0).stop 2024-04-25T14:30:01.074 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:59.819+0000 7f38a1600700 1 -- 172.21.15.53:0/813750256 shutdown_connections 2024-04-25T14:30:01.074 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:59.819+0000 7f38a1600700 1 --2- 172.21.15.53:0/813750256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f389c07cda0 0x7f389c07b200 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:01.074 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:59.819+0000 7f38a1600700 1 -- 172.21.15.53:0/813750256 >> 172.21.15.53:0/813750256 conn(0x7f389c103b50 msgr2=0x7f389c105f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:30:01.074 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:59.820+0000 7f38a1600700 1 -- 172.21.15.53:0/813750256 shutdown_connections 2024-04-25T14:30:01.074 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:59.820+0000 7f38a1600700 1 -- 172.21.15.53:0/813750256 wait complete. 2024-04-25T14:30:01.074 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:59.821+0000 7f38a1600700 1 Processor -- start 2024-04-25T14:30:01.074 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:59.821+0000 7f38a1600700 1 -- start start 2024-04-25T14:30:01.074 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:59.821+0000 7f38a1600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f389c07cda0 0x7f389c19e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:01.074 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:59.821+0000 7f38a1600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f389c10eb30 con 0x7f389c07cda0 2024-04-25T14:30:01.074 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:59.822+0000 7f389aa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f389c07cda0 0x7f389c19e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:30:01.075 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:59.822+0000 7f389aa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f389c07cda0 0x7f389c19e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38822/0 (socket says 172.21.15.53:38822) 2024-04-25T14:30:01.075 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:59.822+0000 7f389aa00700 1 -- 172.21.15.53:0/224100242 learned_addr learned my addr 172.21.15.53:0/224100242 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:30:01.075 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:59.822+0000 7f389aa00700 1 -- 172.21.15.53:0/224100242 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f389c19eb50 con 0x7f389c07cda0 2024-04-25T14:30:01.078 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:59.822+0000 7f389aa00700 1 --2- 172.21.15.53:0/224100242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f389c07cda0 0x7f389c19e610 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f388c009520 tx=0x7f388c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:30:01.078 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:59.823+0000 7f3893400700 1 -- 172.21.15.53:0/224100242 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f388c0039e0 con 0x7f389c07cda0 2024-04-25T14:30:01.078 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:59.823+0000 7f3893400700 1 -- 172.21.15.53:0/224100242 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f388c01b8f0 con 0x7f389c07cda0 2024-04-25T14:30:01.078 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:59.823+0000 7f3893400700 1 -- 172.21.15.53:0/224100242 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f388c019770 con 0x7f389c07cda0 2024-04-25T14:30:01.079 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:59.823+0000 7f38a1600700 1 -- 172.21.15.53:0/224100242 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f389c19ede0 con 0x7f389c07cda0 2024-04-25T14:30:01.079 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:59.823+0000 7f38a1600700 1 -- 172.21.15.53:0/224100242 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f389c1a1ad0 con 0x7f389c07cda0 2024-04-25T14:30:01.079 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:59.824+0000 7f3893400700 1 -- 172.21.15.53:0/224100242 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f388c01b440 con 0x7f389c07cda0 2024-04-25T14:30:01.079 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:59.824+0000 7f38a1600700 1 -- 172.21.15.53:0/224100242 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f389c0640a0 con 0x7f389c07cda0 2024-04-25T14:30:01.079 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:59.825+0000 7f3893400700 1 --2- 172.21.15.53:0/224100242 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f3880039600 0x7f388003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:01.079 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:59.825+0000 7f3893400700 1 -- 172.21.15.53:0/224100242 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f388c04de80 con 0x7f389c07cda0 2024-04-25T14:30:01.079 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:59.825+0000 7f389a000700 1 --2- 172.21.15.53:0/224100242 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f3880039600 0x7f388003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:30:01.079 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:59.826+0000 7f389a000700 1 --2- 172.21.15.53:0/224100242 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f3880039600 0x7f388003bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3888006fd0 tx=0x7f3888006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:30:01.079 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:29:59.834+0000 7f3893400700 1 -- 172.21.15.53:0/224100242 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f388c01f030 con 0x7f389c07cda0 2024-04-25T14:30:01.079 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:00.084+0000 7f38a1600700 1 -- 172.21.15.53:0/224100242 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f389c1a20c0 con 0x7f389c07cda0 2024-04-25T14:30:01.080 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:00.087+0000 7f3893400700 1 -- 172.21.15.53:0/224100242 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f388c00b3e0 con 0x7f389c07cda0 2024-04-25T14:30:01.080 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:00.090+0000 7f38a1600700 1 -- 172.21.15.53:0/224100242 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f3880039600 msgr2=0x7f388003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:01.080 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:00.090+0000 7f38a1600700 1 --2- 172.21.15.53:0/224100242 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f3880039600 0x7f388003bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3888006fd0 tx=0x7f3888006e40 comp rx=0 tx=0).stop 2024-04-25T14:30:01.080 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:00.090+0000 7f38a1600700 1 -- 172.21.15.53:0/224100242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f389c07cda0 msgr2=0x7f389c19e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:01.080 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:00.090+0000 7f38a1600700 1 --2- 172.21.15.53:0/224100242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f389c07cda0 0x7f389c19e610 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f388c009520 tx=0x7f388c011aa0 comp rx=0 tx=0).stop 2024-04-25T14:30:01.080 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:00.091+0000 7f38a1600700 1 -- 172.21.15.53:0/224100242 shutdown_connections 2024-04-25T14:30:01.080 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:00.091+0000 7f38a1600700 1 --2- 172.21.15.53:0/224100242 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f3880039600 0x7f388003bac0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:01.080 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:00.091+0000 7f38a1600700 1 --2- 172.21.15.53:0/224100242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f389c07cda0 0x7f389c19e610 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:01.080 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:00.091+0000 7f38a1600700 1 -- 172.21.15.53:0/224100242 >> 172.21.15.53:0/224100242 conn(0x7f389c103b50 msgr2=0x7f389c104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:30:01.080 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:00.091+0000 7f38a1600700 1 -- 172.21.15.53:0/224100242 shutdown_connections 2024-04-25T14:30:01.080 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:00.091+0000 7f38a1600700 1 -- 172.21.15.53:0/224100242 wait complete. 2024-04-25T14:30:01.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:01 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/224100242' entity='client.admin' 2024-04-25T14:30:01.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:01 smithi053 ceph-mon[27773]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:30:03.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:02 smithi053 ceph-mon[27773]: from='mgr.14120 172.21.15.53:0/2278703669' entity='mgr.x' 2024-04-25T14:30:03.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:02 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/152701544' entity='client.admin' 2024-04-25T14:30:03.093 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.096+0000 7faed5400700 1 Processor -- start 2024-04-25T14:30:03.098 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.096+0000 7faed5400700 1 -- start start 2024-04-25T14:30:03.098 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.097+0000 7faed5400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faed0108530 0x7faed0108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:03.098 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.097+0000 7faed5400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7faed0108f20 con 0x7faed0108530 2024-04-25T14:30:03.099 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.097+0000 7faece800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faed0108530 0x7faed0108950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:30:03.099 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.097+0000 7faece800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faed0108530 0x7faed0108950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38830/0 (socket says 172.21.15.53:38830) 2024-04-25T14:30:03.099 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.097+0000 7faece800700 1 -- 172.21.15.53:0/2746558872 learned_addr learned my addr 172.21.15.53:0/2746558872 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:30:03.099 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.098+0000 7faece800700 1 -- 172.21.15.53:0/2746558872 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed0109730 con 0x7faed0108530 2024-04-25T14:30:03.099 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.098+0000 7faece800700 1 --2- 172.21.15.53:0/2746558872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faed0108530 0x7faed0108950 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7faec00098a0 tx=0x7faec0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58fc4f7555997a57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:30:03.099 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.098+0000 7faecd400700 1 -- 172.21.15.53:0/2746558872 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faec0004160 con 0x7faed0108530 2024-04-25T14:30:03.099 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.099+0000 7faecd400700 1 -- 172.21.15.53:0/2746558872 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faec0004300 con 0x7faed0108530 2024-04-25T14:30:03.099 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.099+0000 7faecd400700 1 -- 172.21.15.53:0/2746558872 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faec0011a20 con 0x7faed0108530 2024-04-25T14:30:03.099 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.099+0000 7faed5400700 1 -- 172.21.15.53:0/2746558872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faed0108530 msgr2=0x7faed0108950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:03.099 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.099+0000 7faed5400700 1 --2- 172.21.15.53:0/2746558872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faed0108530 0x7faed0108950 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7faec00098a0 tx=0x7faec0009bb0 comp rx=0 tx=0).stop 2024-04-25T14:30:03.099 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.099+0000 7faed5400700 1 -- 172.21.15.53:0/2746558872 shutdown_connections 2024-04-25T14:30:03.099 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.099+0000 7faed5400700 1 --2- 172.21.15.53:0/2746558872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faed0108530 0x7faed0108950 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:03.099 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.099+0000 7faed5400700 1 -- 172.21.15.53:0/2746558872 >> 172.21.15.53:0/2746558872 conn(0x7faed0103b50 msgr2=0x7faed0105f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:30:03.099 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.100+0000 7faed5400700 1 -- 172.21.15.53:0/2746558872 shutdown_connections 2024-04-25T14:30:03.100 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.100+0000 7faed5400700 1 -- 172.21.15.53:0/2746558872 wait complete. 2024-04-25T14:30:03.100 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.101+0000 7faed5400700 1 Processor -- start 2024-04-25T14:30:03.100 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.101+0000 7faed5400700 1 -- start start 2024-04-25T14:30:03.100 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.101+0000 7faed5400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faed0108530 0x7faed019a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:03.100 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.101+0000 7faed5400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7faed007afe0 con 0x7faed0108530 2024-04-25T14:30:03.100 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.102+0000 7faece800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faed0108530 0x7faed019a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:30:03.100 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.102+0000 7faece800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faed0108530 0x7faed019a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38844/0 (socket says 172.21.15.53:38844) 2024-04-25T14:30:03.100 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.102+0000 7faece800700 1 -- 172.21.15.53:0/152701544 learned_addr learned my addr 172.21.15.53:0/152701544 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:30:03.100 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.102+0000 7faece800700 1 -- 172.21.15.53:0/152701544 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed019a760 con 0x7faed0108530 2024-04-25T14:30:03.100 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.102+0000 7faece800700 1 --2- 172.21.15.53:0/152701544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faed0108530 0x7faed019a220 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faec00116e0 tx=0x7faec0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:30:03.100 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.103+0000 7faec7400700 1 -- 172.21.15.53:0/152701544 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faec00039e0 con 0x7faed0108530 2024-04-25T14:30:03.100 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.103+0000 7faec7400700 1 -- 172.21.15.53:0/152701544 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faec001b8f0 con 0x7faed0108530 2024-04-25T14:30:03.100 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.103+0000 7faed5400700 1 -- 172.21.15.53:0/152701544 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faed019a9f0 con 0x7faed0108530 2024-04-25T14:30:03.100 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.103+0000 7faec7400700 1 -- 172.21.15.53:0/152701544 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faec0023760 con 0x7faed0108530 2024-04-25T14:30:03.101 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.103+0000 7faed5400700 1 -- 172.21.15.53:0/152701544 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faed019d6e0 con 0x7faed0108530 2024-04-25T14:30:03.101 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.104+0000 7faec7400700 1 -- 172.21.15.53:0/152701544 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7faec001b440 con 0x7faed0108530 2024-04-25T14:30:03.101 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.104+0000 7faed5400700 1 -- 172.21.15.53:0/152701544 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faed00640a0 con 0x7faed0108530 2024-04-25T14:30:03.101 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.105+0000 7faec7400700 1 --2- 172.21.15.53:0/152701544 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7faeb403d970 0x7faeb403fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:03.101 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.105+0000 7faec7400700 1 -- 172.21.15.53:0/152701544 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7faec004df00 con 0x7faed0108530 2024-04-25T14:30:03.101 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.105+0000 7faecde00700 1 --2- 172.21.15.53:0/152701544 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7faeb403d970 0x7faeb403fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:30:03.101 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.106+0000 7faecde00700 1 --2- 172.21.15.53:0/152701544 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7faeb403d970 0x7faeb403fe30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7faebc006fd0 tx=0x7faebc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:30:03.101 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.111+0000 7faec7400700 1 -- 172.21.15.53:0/152701544 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faec001f030 con 0x7faed0108530 2024-04-25T14:30:03.101 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.343+0000 7faed5400700 1 -- 172.21.15.53:0/152701544 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7faed002e240 con 0x7faed0108530 2024-04-25T14:30:03.101 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.348+0000 7faec7400700 1 -- 172.21.15.53:0/152701544 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7faec002cde0 con 0x7faed0108530 2024-04-25T14:30:03.101 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.352+0000 7faed5400700 1 -- 172.21.15.53:0/152701544 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7faeb403d970 msgr2=0x7faeb403fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:03.101 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.352+0000 7faed5400700 1 --2- 172.21.15.53:0/152701544 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7faeb403d970 0x7faeb403fe30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7faebc006fd0 tx=0x7faebc006e40 comp rx=0 tx=0).stop 2024-04-25T14:30:03.101 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.352+0000 7faed5400700 1 -- 172.21.15.53:0/152701544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faed0108530 msgr2=0x7faed019a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:03.101 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.352+0000 7faed5400700 1 --2- 172.21.15.53:0/152701544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faed0108530 0x7faed019a220 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faec00116e0 tx=0x7faec0011aa0 comp rx=0 tx=0).stop 2024-04-25T14:30:03.102 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.352+0000 7faed5400700 1 -- 172.21.15.53:0/152701544 shutdown_connections 2024-04-25T14:30:03.103 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.352+0000 7faed5400700 1 --2- 172.21.15.53:0/152701544 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7faeb403d970 0x7faeb403fe30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:03.103 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.352+0000 7faed5400700 1 --2- 172.21.15.53:0/152701544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faed0108530 0x7faed019a220 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:03.103 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.352+0000 7faed5400700 1 -- 172.21.15.53:0/152701544 >> 172.21.15.53:0/152701544 conn(0x7faed0103b50 msgr2=0x7faed0105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:30:03.103 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.352+0000 7faed5400700 1 -- 172.21.15.53:0/152701544 shutdown_connections 2024-04-25T14:30:03.103 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:02.353+0000 7faed5400700 1 -- 172.21.15.53:0/152701544 wait complete. 2024-04-25T14:30:03.103 INFO:teuthology.orchestra.run.smithi053.stdout:Enabling the dashboard module... 2024-04-25T14:30:04.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:03 smithi053 ceph-mon[27773]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:30:05.191 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:04 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1461301815' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-25T14:30:05.191 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:04 smithi053 ceph-mon[27773]: from='mgr.14120 172.21.15.53:0/2278703669' entity='mgr.x' 2024-04-25T14:30:05.192 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:30:05 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: ignoring --setuser ceph since I am not root 2024-04-25T14:30:05.192 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:30:05 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: ignoring --setgroup ceph since I am not root 2024-04-25T14:30:05.192 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:30:05 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:30:05.191+0000 7fd87418e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:30:05.532 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:30:05 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:30:05.440+0000 7fd87418e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:30:05.697 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:03.780+0000 7fd02d800700 1 Processor -- start 2024-04-25T14:30:05.697 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:03.780+0000 7fd02d800700 1 -- start start 2024-04-25T14:30:05.697 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:03.781+0000 7fd02d800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd02807cd80 0x7fd02807b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:05.697 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:03.781+0000 7fd02d800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd02807b7b0 con 0x7fd02807cd80 2024-04-25T14:30:05.697 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:03.781+0000 7fd026a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd02807cd80 0x7fd02807b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:30:05.697 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:03.781+0000 7fd026a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd02807cd80 0x7fd02807b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41026/0 (socket says 172.21.15.53:41026) 2024-04-25T14:30:05.697 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:03.781+0000 7fd026a00700 1 -- 172.21.15.53:0/1817591818 learned_addr learned my addr 172.21.15.53:0/1817591818 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:30:05.697 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:03.782+0000 7fd026a00700 1 -- 172.21.15.53:0/1817591818 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd02807b930 con 0x7fd02807cd80 2024-04-25T14:30:05.697 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:03.782+0000 7fd026a00700 1 --2- 172.21.15.53:0/1817591818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd02807cd80 0x7fd02807b1e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd0180098a0 tx=0x7fd018009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f3b44b5db4c66e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:30:05.697 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:03.783+0000 7fd025600700 1 -- 172.21.15.53:0/1817591818 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd018004160 con 0x7fd02807cd80 2024-04-25T14:30:05.697 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:03.783+0000 7fd025600700 1 -- 172.21.15.53:0/1817591818 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd018004300 con 0x7fd02807cd80 2024-04-25T14:30:05.698 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:03.783+0000 7fd025600700 1 -- 172.21.15.53:0/1817591818 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd018011a20 con 0x7fd02807cd80 2024-04-25T14:30:05.702 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:03.783+0000 7fd02d800700 1 -- 172.21.15.53:0/1817591818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd02807cd80 msgr2=0x7fd02807b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:05.702 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:03.783+0000 7fd02d800700 1 --2- 172.21.15.53:0/1817591818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd02807cd80 0x7fd02807b1e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd0180098a0 tx=0x7fd018009bb0 comp rx=0 tx=0).stop 2024-04-25T14:30:05.702 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:03.783+0000 7fd02d800700 1 -- 172.21.15.53:0/1817591818 shutdown_connections 2024-04-25T14:30:05.702 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:03.783+0000 7fd02d800700 1 --2- 172.21.15.53:0/1817591818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd02807cd80 0x7fd02807b1e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:05.702 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:03.783+0000 7fd02d800700 1 -- 172.21.15.53:0/1817591818 >> 172.21.15.53:0/1817591818 conn(0x7fd028103b50 msgr2=0x7fd028105f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:30:05.702 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:03.784+0000 7fd02d800700 1 -- 172.21.15.53:0/1817591818 shutdown_connections 2024-04-25T14:30:05.702 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:03.784+0000 7fd02d800700 1 -- 172.21.15.53:0/1817591818 wait complete. 2024-04-25T14:30:05.702 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:03.785+0000 7fd02d800700 1 Processor -- start 2024-04-25T14:30:05.702 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:03.785+0000 7fd02d800700 1 -- start start 2024-04-25T14:30:05.703 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:03.785+0000 7fd02d800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd02807cd80 0x7fd02819e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:05.703 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:03.785+0000 7fd02d800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd02810eb50 con 0x7fd02807cd80 2024-04-25T14:30:05.703 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:03.785+0000 7fd026a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd02807cd80 0x7fd02819e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:30:05.703 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:03.786+0000 7fd026a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd02807cd80 0x7fd02819e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41040/0 (socket says 172.21.15.53:41040) 2024-04-25T14:30:05.703 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:03.786+0000 7fd026a00700 1 -- 172.21.15.53:0/1461301815 learned_addr learned my addr 172.21.15.53:0/1461301815 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:30:05.703 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:03.786+0000 7fd026a00700 1 -- 172.21.15.53:0/1461301815 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd02819eb00 con 0x7fd02807cd80 2024-04-25T14:30:05.703 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:03.786+0000 7fd026a00700 1 --2- 172.21.15.53:0/1461301815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd02807cd80 0x7fd02819e5c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd018000c00 tx=0x7fd018011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:30:05.703 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:03.787+0000 7fd01f400700 1 -- 172.21.15.53:0/1461301815 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd018003aa0 con 0x7fd02807cd80 2024-04-25T14:30:05.703 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:03.787+0000 7fd01f400700 1 -- 172.21.15.53:0/1461301815 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd01801b8f0 con 0x7fd02807cd80 2024-04-25T14:30:05.703 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:03.787+0000 7fd01f400700 1 -- 172.21.15.53:0/1461301815 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd018023870 con 0x7fd02807cd80 2024-04-25T14:30:05.703 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:03.787+0000 7fd02d800700 1 -- 172.21.15.53:0/1461301815 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd02819ed90 con 0x7fd02807cd80 2024-04-25T14:30:05.703 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:03.787+0000 7fd02d800700 1 -- 172.21.15.53:0/1461301815 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0281a1a80 con 0x7fd02807cd80 2024-04-25T14:30:05.703 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:03.788+0000 7fd02d800700 1 -- 172.21.15.53:0/1461301815 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0280640a0 con 0x7fd02807cd80 2024-04-25T14:30:05.703 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:03.789+0000 7fd01f400700 1 -- 172.21.15.53:0/1461301815 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fd01801b440 con 0x7fd02807cd80 2024-04-25T14:30:05.703 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:03.789+0000 7fd01f400700 1 --2- 172.21.15.53:0/1461301815 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7fd00c039600 0x7fd00c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:05.704 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:03.789+0000 7fd01f400700 1 -- 172.21.15.53:0/1461301815 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd01804dea0 con 0x7fd02807cd80 2024-04-25T14:30:05.704 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:03.790+0000 7fd026000700 1 --2- 172.21.15.53:0/1461301815 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7fd00c039600 0x7fd00c03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:30:05.704 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:03.790+0000 7fd026000700 1 --2- 172.21.15.53:0/1461301815 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7fd00c039600 0x7fd00c03bac0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd014006fd0 tx=0x7fd014006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:30:05.704 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:03.795+0000 7fd01f400700 1 -- 172.21.15.53:0/1461301815 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd01801f030 con 0x7fd02807cd80 2024-04-25T14:30:05.704 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:04.092+0000 7fd02d800700 1 -- 172.21.15.53:0/1461301815 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fd0281a1c50 con 0x7fd02807cd80 2024-04-25T14:30:05.704 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:04.883+0000 7fd01f400700 1 -- 172.21.15.53:0/1461301815 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fd01800e260 con 0x7fd02807cd80 2024-04-25T14:30:05.704 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:04.885+0000 7fd01f400700 1 -- 172.21.15.53:0/1461301815 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fd018053020 con 0x7fd02807cd80 2024-04-25T14:30:05.704 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:04.887+0000 7fd02d800700 1 -- 172.21.15.53:0/1461301815 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7fd00c039600 msgr2=0x7fd00c03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:05.704 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:04.887+0000 7fd02d800700 1 --2- 172.21.15.53:0/1461301815 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7fd00c039600 0x7fd00c03bac0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd014006fd0 tx=0x7fd014006e40 comp rx=0 tx=0).stop 2024-04-25T14:30:05.704 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:04.887+0000 7fd02d800700 1 -- 172.21.15.53:0/1461301815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd02807cd80 msgr2=0x7fd02819e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:05.704 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:04.887+0000 7fd02d800700 1 --2- 172.21.15.53:0/1461301815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd02807cd80 0x7fd02819e5c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd018000c00 tx=0x7fd018011aa0 comp rx=0 tx=0).stop 2024-04-25T14:30:05.704 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:04.888+0000 7fd02d800700 1 -- 172.21.15.53:0/1461301815 shutdown_connections 2024-04-25T14:30:05.704 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:04.888+0000 7fd02d800700 1 --2- 172.21.15.53:0/1461301815 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7fd00c039600 0x7fd00c03bac0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:05.705 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:04.888+0000 7fd02d800700 1 --2- 172.21.15.53:0/1461301815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd02807cd80 0x7fd02819e5c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:05.705 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:04.888+0000 7fd02d800700 1 -- 172.21.15.53:0/1461301815 >> 172.21.15.53:0/1461301815 conn(0x7fd028103b50 msgr2=0x7fd0281045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:30:05.705 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:04.888+0000 7fd02d800700 1 -- 172.21.15.53:0/1461301815 shutdown_connections 2024-04-25T14:30:05.705 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:04.888+0000 7fd02d800700 1 -- 172.21.15.53:0/1461301815 wait complete. 2024-04-25T14:30:05.853 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:30:05 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:30:05.574+0000 7fd87418e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:30:06.119 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:05 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1461301815' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-25T14:30:06.119 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:05 smithi053 ceph-mon[27773]: mgrmap e9: x(active, since 26s) 2024-04-25T14:30:06.119 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:30:05 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:30:05.997+0000 7fd87418e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:30:06.390 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:30:06 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:30:06.118+0000 7fd87418e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:30:07.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:06 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/3859262152' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T14:30:07.581 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout { 2024-04-25T14:30:07.582 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-25T14:30:07.582 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T14:30:07.582 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T14:30:07.582 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T14:30:07.582 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout } 2024-04-25T14:30:07.582 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.565+0000 7f802e400700 1 Processor -- start 2024-04-25T14:30:07.582 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.565+0000 7f802e400700 1 -- start start 2024-04-25T14:30:07.582 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.565+0000 7f802e400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8028106e20 0x7f8028107240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:07.582 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.565+0000 7f802e400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8028063ce0 con 0x7f8028106e20 2024-04-25T14:30:07.582 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.566+0000 7f8027400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8028106e20 0x7f8028107240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:30:07.582 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.566+0000 7f8027400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8028106e20 0x7f8028107240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41074/0 (socket says 172.21.15.53:41074) 2024-04-25T14:30:07.582 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.566+0000 7f8027400700 1 -- 172.21.15.53:0/361158824 learned_addr learned my addr 172.21.15.53:0/361158824 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:30:07.583 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.566+0000 7f8027400700 1 -- 172.21.15.53:0/361158824 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8028107780 con 0x7f8028106e20 2024-04-25T14:30:07.583 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.567+0000 7f8027400700 1 --2- 172.21.15.53:0/361158824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8028106e20 0x7f8028107240 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f80180098a0 tx=0x7f8018009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52223f9dc4759885 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:30:07.583 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.567+0000 7f8026000700 1 -- 172.21.15.53:0/361158824 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8018004160 con 0x7f8028106e20 2024-04-25T14:30:07.583 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.567+0000 7f8026000700 1 -- 172.21.15.53:0/361158824 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8018004300 con 0x7f8028106e20 2024-04-25T14:30:07.583 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.567+0000 7f8026000700 1 -- 172.21.15.53:0/361158824 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80180119d0 con 0x7f8028106e20 2024-04-25T14:30:07.583 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.568+0000 7f802e400700 1 -- 172.21.15.53:0/361158824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8028106e20 msgr2=0x7f8028107240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:07.583 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.568+0000 7f802e400700 1 --2- 172.21.15.53:0/361158824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8028106e20 0x7f8028107240 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f80180098a0 tx=0x7f8018009bb0 comp rx=0 tx=0).stop 2024-04-25T14:30:07.583 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.568+0000 7f802e400700 1 -- 172.21.15.53:0/361158824 shutdown_connections 2024-04-25T14:30:07.583 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.568+0000 7f802e400700 1 --2- 172.21.15.53:0/361158824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8028106e20 0x7f8028107240 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:07.583 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.568+0000 7f802e400700 1 -- 172.21.15.53:0/361158824 >> 172.21.15.53:0/361158824 conn(0x7f8028102a40 msgr2=0x7f8028104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:30:07.583 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.568+0000 7f802e400700 1 -- 172.21.15.53:0/361158824 shutdown_connections 2024-04-25T14:30:07.583 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.568+0000 7f802e400700 1 -- 172.21.15.53:0/361158824 wait complete. 2024-04-25T14:30:07.583 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.569+0000 7f802e400700 1 Processor -- start 2024-04-25T14:30:07.583 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.570+0000 7f802e400700 1 -- start start 2024-04-25T14:30:07.583 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.570+0000 7f802e400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8028106e20 0x7f802819e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:07.583 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.570+0000 7f802e400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f80281087f0 con 0x7f8028106e20 2024-04-25T14:30:07.584 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.570+0000 7f8027400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8028106e20 0x7f802819e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:30:07.584 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.570+0000 7f8027400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8028106e20 0x7f802819e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41086/0 (socket says 172.21.15.53:41086) 2024-04-25T14:30:07.584 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.570+0000 7f8027400700 1 -- 172.21.15.53:0/3859262152 learned_addr learned my addr 172.21.15.53:0/3859262152 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:30:07.584 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.571+0000 7f8027400700 1 -- 172.21.15.53:0/3859262152 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f802819eaf0 con 0x7f8028106e20 2024-04-25T14:30:07.584 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.571+0000 7f8027400700 1 --2- 172.21.15.53:0/3859262152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8028106e20 0x7f802819e5b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8018000c00 tx=0x7f8018011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:30:07.584 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.571+0000 7f801fe00700 1 -- 172.21.15.53:0/3859262152 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8018003af0 con 0x7f8028106e20 2024-04-25T14:30:07.584 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.571+0000 7f801fe00700 1 -- 172.21.15.53:0/3859262152 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f801801e070 con 0x7f8028106e20 2024-04-25T14:30:07.584 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.571+0000 7f801fe00700 1 -- 172.21.15.53:0/3859262152 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f801801a6b0 con 0x7f8028106e20 2024-04-25T14:30:07.584 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.571+0000 7f802e400700 1 -- 172.21.15.53:0/3859262152 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f802819ed80 con 0x7f8028106e20 2024-04-25T14:30:07.584 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.572+0000 7f802e400700 1 -- 172.21.15.53:0/3859262152 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f802810b120 con 0x7f8028106e20 2024-04-25T14:30:07.584 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.573+0000 7f802e400700 1 -- 172.21.15.53:0/3859262152 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80280640a0 con 0x7f8028106e20 2024-04-25T14:30:07.584 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.573+0000 7f801fe00700 1 -- 172.21.15.53:0/3859262152 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f8018023a40 con 0x7f8028106e20 2024-04-25T14:30:07.584 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.574+0000 7f801fe00700 1 --2- 172.21.15.53:0/3859262152 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f800c039650 0x7f800c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:07.584 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.574+0000 7f801fe00700 1 -- 172.21.15.53:0/3859262152 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f801804de00 con 0x7f8028106e20 2024-04-25T14:30:07.584 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.574+0000 7f8026a00700 1 -- 172.21.15.53:0/3859262152 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f800c039650 msgr2=0x7f800c03bb10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/3887954889 2024-04-25T14:30:07.585 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.574+0000 7f8026a00700 1 --2- 172.21.15.53:0/3859262152 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f800c039650 0x7f800c03bb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:30:07.585 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.580+0000 7f801fe00700 1 -- 172.21.15.53:0/3859262152 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8018018350 con 0x7f8028106e20 2024-04-25T14:30:07.585 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.775+0000 7f8026a00700 1 -- 172.21.15.53:0/3859262152 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f800c039650 msgr2=0x7f800c03bb10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/3887954889 2024-04-25T14:30:07.585 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.775+0000 7f8026a00700 1 --2- 172.21.15.53:0/3859262152 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f800c039650 0x7f800c03bb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T14:30:07.585 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.910+0000 7f802e400700 1 -- 172.21.15.53:0/3859262152 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f802802e050 con 0x7f8028106e20 2024-04-25T14:30:07.585 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.911+0000 7f801fe00700 1 -- 172.21.15.53:0/3859262152 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f801804d790 con 0x7f8028106e20 2024-04-25T14:30:07.585 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.913+0000 7f802e400700 1 -- 172.21.15.53:0/3859262152 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f800c039650 msgr2=0x7f800c03bb10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:30:07.585 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.913+0000 7f802e400700 1 --2- 172.21.15.53:0/3859262152 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f800c039650 0x7f800c03bb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:07.585 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.913+0000 7f802e400700 1 -- 172.21.15.53:0/3859262152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8028106e20 msgr2=0x7f802819e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:07.585 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.913+0000 7f802e400700 1 --2- 172.21.15.53:0/3859262152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8028106e20 0x7f802819e5b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8018000c00 tx=0x7f8018011aa0 comp rx=0 tx=0).stop 2024-04-25T14:30:07.585 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.913+0000 7f802e400700 1 -- 172.21.15.53:0/3859262152 shutdown_connections 2024-04-25T14:30:07.585 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.913+0000 7f802e400700 1 --2- 172.21.15.53:0/3859262152 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f800c039650 0x7f800c03bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:07.585 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.913+0000 7f802e400700 1 --2- 172.21.15.53:0/3859262152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8028106e20 0x7f802819e5b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:07.585 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.913+0000 7f802e400700 1 -- 172.21.15.53:0/3859262152 >> 172.21.15.53:0/3859262152 conn(0x7f8028102a40 msgr2=0x7f8028195240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:30:07.585 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.914+0000 7f802e400700 1 -- 172.21.15.53:0/3859262152 shutdown_connections 2024-04-25T14:30:07.585 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:06.914+0000 7f802e400700 1 -- 172.21.15.53:0/3859262152 wait complete. 2024-04-25T14:30:07.586 INFO:teuthology.orchestra.run.smithi053.stdout:Waiting for the mgr to restart... 2024-04-25T14:30:07.586 INFO:teuthology.orchestra.run.smithi053.stdout:Waiting for mgr epoch 9... 2024-04-25T14:30:07.873 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:30:07 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:30:07.768+0000 7fd87418e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:30:08.124 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:30:08 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:30:08.007+0000 7fd87418e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:30:08.531 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:30:08 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:30:08.229+0000 7fd87418e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:30:08.783 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:30:08 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:30:08.632+0000 7fd87418e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:30:08.783 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:30:08 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:30:08.763+0000 7fd87418e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:30:09.282 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:30:09 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:30:09.023+0000 7fd87418e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:30:09.782 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:30:09 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:30:09.284+0000 7fd87418e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:30:10.532 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:30:10 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:30:10.093+0000 7fd87418e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T14:30:10.975 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:30:10 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:30:10.674+0000 7fd87418e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:30:10.975 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:30:10 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:30:10.807+0000 7fd87418e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:30:11.282 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:30:11 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:30:11.083+0000 7fd87418e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T14:30:12.032 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:30:11 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:30:11.756+0000 7fd87418e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:30:12.033 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:30:11 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:30:11.886+0000 7fd87418e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:30:12.531 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:30:12 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:30:12.219+0000 7fd87418e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:30:12.531 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:30:12 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:30:12.401+0000 7fd87418e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:30:12.531 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:30:12 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:30:12.530+0000 7fd87418e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:30:12.782 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:30:12 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:30:12.686+0000 7fd87418e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:30:13.532 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:30:13 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:30:13.063+0000 7fd87418e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:30:14.162 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:30:13 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:30:13.850+0000 7fd87418e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:30:14.532 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:30:14 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:30:14.159+0000 7fd87418e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:30:14.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:14 smithi053 ceph-mon[27773]: Active manager daemon x restarted 2024-04-25T14:30:14.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:14 smithi053 ceph-mon[27773]: Activating manager daemon x 2024-04-25T14:30:14.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:14 smithi053 ceph-mon[27773]: osdmap e3: 0 total, 0 up, 0 in 2024-04-25T14:30:14.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:14 smithi053 ceph-mon[27773]: mgrmap e10: x(active, starting, since 0.00518425s) 2024-04-25T14:30:14.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:14 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:30:14.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:14 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T14:30:14.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:14 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T14:30:14.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:14 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T14:30:14.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:14 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T14:30:14.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:14 smithi053 ceph-mon[27773]: Manager daemon x is now available 2024-04-25T14:30:15.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:15 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:30:15.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:15 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T14:30:15.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:15 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T14:30:15.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:15 smithi053 ceph-mon[27773]: [25/Apr/2024:14:30:14] ENGINE Bus STARTING 2024-04-25T14:30:15.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:15 smithi053 ceph-mon[27773]: mgrmap e11: x(active, since 1.00898s) 2024-04-25T14:30:15.897 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout { 2024-04-25T14:30:15.897 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-25T14:30:15.897 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T14:30:15.897 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout } 2024-04-25T14:30:15.897 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:08.803+0000 7f64e3400700 1 Processor -- start 2024-04-25T14:30:15.898 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:08.804+0000 7f64e3400700 1 -- start start 2024-04-25T14:30:15.898 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:08.804+0000 7f64e3400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64dc109070 0x7f64dc10b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:15.898 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:08.804+0000 7f64e3400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f64dc063ca0 con 0x7f64dc109070 2024-04-25T14:30:15.898 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:08.804+0000 7f64e0a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64dc109070 0x7f64dc10b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:30:15.898 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:08.804+0000 7f64e0a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64dc109070 0x7f64dc10b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41090/0 (socket says 172.21.15.53:41090) 2024-04-25T14:30:15.898 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:08.804+0000 7f64e0a00700 1 -- 172.21.15.53:0/2924056060 learned_addr learned my addr 172.21.15.53:0/2924056060 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:30:15.898 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:08.805+0000 7f64e0a00700 1 -- 172.21.15.53:0/2924056060 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64dc10b9a0 con 0x7f64dc109070 2024-04-25T14:30:15.898 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:08.805+0000 7f64e0a00700 1 --2- 172.21.15.53:0/2924056060 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64dc109070 0x7f64dc10b460 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f64cc0098a0 tx=0x7f64cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=173e0a34f7286bd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:30:15.898 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:08.805+0000 7f64dbe00700 1 -- 172.21.15.53:0/2924056060 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f64cc004160 con 0x7f64dc109070 2024-04-25T14:30:15.898 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:08.805+0000 7f64dbe00700 1 -- 172.21.15.53:0/2924056060 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64cc004300 con 0x7f64dc109070 2024-04-25T14:30:15.898 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:08.805+0000 7f64dbe00700 1 -- 172.21.15.53:0/2924056060 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f64cc011a20 con 0x7f64dc109070 2024-04-25T14:30:15.898 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:08.806+0000 7f64e3400700 1 -- 172.21.15.53:0/2924056060 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64dc109070 msgr2=0x7f64dc10b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:15.898 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:08.806+0000 7f64e3400700 1 --2- 172.21.15.53:0/2924056060 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64dc109070 0x7f64dc10b460 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f64cc0098a0 tx=0x7f64cc009bb0 comp rx=0 tx=0).stop 2024-04-25T14:30:15.899 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:08.806+0000 7f64e3400700 1 -- 172.21.15.53:0/2924056060 shutdown_connections 2024-04-25T14:30:15.899 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:08.806+0000 7f64e3400700 1 --2- 172.21.15.53:0/2924056060 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64dc109070 0x7f64dc10b460 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:15.899 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:08.806+0000 7f64e3400700 1 -- 172.21.15.53:0/2924056060 >> 172.21.15.53:0/2924056060 conn(0x7f64dc102a40 msgr2=0x7f64dc104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:30:15.899 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:08.806+0000 7f64e3400700 1 -- 172.21.15.53:0/2924056060 shutdown_connections 2024-04-25T14:30:15.899 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:08.806+0000 7f64e3400700 1 -- 172.21.15.53:0/2924056060 wait complete. 2024-04-25T14:30:15.899 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:08.807+0000 7f64e3400700 1 Processor -- start 2024-04-25T14:30:15.899 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:08.808+0000 7f64e3400700 1 -- start start 2024-04-25T14:30:15.899 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:08.808+0000 7f64e3400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64dc109070 0x7f64dc1088f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:15.899 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:08.808+0000 7f64e3400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f64dc10c8f0 con 0x7f64dc109070 2024-04-25T14:30:15.899 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:08.808+0000 7f64e0a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64dc109070 0x7f64dc1088f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:30:15.899 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:08.808+0000 7f64e0a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64dc109070 0x7f64dc1088f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41096/0 (socket says 172.21.15.53:41096) 2024-04-25T14:30:15.899 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:08.808+0000 7f64e0a00700 1 -- 172.21.15.53:0/3767040449 learned_addr learned my addr 172.21.15.53:0/3767040449 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:30:15.899 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:08.809+0000 7f64e0a00700 1 -- 172.21.15.53:0/3767040449 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64dc108e30 con 0x7f64dc109070 2024-04-25T14:30:15.899 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:08.809+0000 7f64e0a00700 1 --2- 172.21.15.53:0/3767040449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64dc109070 0x7f64dc1088f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f64cc000c00 tx=0x7f64cc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:30:15.900 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:08.809+0000 7f64da000700 1 -- 172.21.15.53:0/3767040449 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f64cc003aa0 con 0x7f64dc109070 2024-04-25T14:30:15.900 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:08.809+0000 7f64da000700 1 -- 172.21.15.53:0/3767040449 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64cc01b8f0 con 0x7f64dc109070 2024-04-25T14:30:15.900 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:08.809+0000 7f64da000700 1 -- 172.21.15.53:0/3767040449 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f64cc023870 con 0x7f64dc109070 2024-04-25T14:30:15.900 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:08.809+0000 7f64e3400700 1 -- 172.21.15.53:0/3767040449 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64dc107020 con 0x7f64dc109070 2024-04-25T14:30:15.900 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:08.810+0000 7f64e3400700 1 -- 172.21.15.53:0/3767040449 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64dc107560 con 0x7f64dc109070 2024-04-25T14:30:15.900 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:08.811+0000 7f64da000700 1 -- 172.21.15.53:0/3767040449 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f64cc01b440 con 0x7f64dc109070 2024-04-25T14:30:15.900 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:08.811+0000 7f64da000700 1 --2- 172.21.15.53:0/3767040449 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f64c0039660 0x7f64c003bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:15.900 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:08.811+0000 7f64da000700 1 -- 172.21.15.53:0/3767040449 --> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f64c003c230 con 0x7f64c0039660 2024-04-25T14:30:15.900 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:08.811+0000 7f64da000700 1 -- 172.21.15.53:0/3767040449 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f64cc04df20 con 0x7f64dc109070 2024-04-25T14:30:15.900 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:08.812+0000 7f64d3e00700 1 -- 172.21.15.53:0/3767040449 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f64c0039660 msgr2=0x7f64c003bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/3887954889 2024-04-25T14:30:15.900 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:08.812+0000 7f64d3e00700 1 --2- 172.21.15.53:0/3767040449 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f64c0039660 0x7f64c003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:30:15.900 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:09.013+0000 7f64d3e00700 1 -- 172.21.15.53:0/3767040449 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f64c0039660 msgr2=0x7f64c003bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/3887954889 2024-04-25T14:30:15.900 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:09.013+0000 7f64d3e00700 1 --2- 172.21.15.53:0/3767040449 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f64c0039660 0x7f64c003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T14:30:15.900 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:09.414+0000 7f64d3e00700 1 -- 172.21.15.53:0/3767040449 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f64c0039660 msgr2=0x7f64c003bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/3887954889 2024-04-25T14:30:15.901 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:09.414+0000 7f64d3e00700 1 --2- 172.21.15.53:0/3767040449 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f64c0039660 0x7f64c003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T14:30:15.901 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:10.215+0000 7f64d3e00700 1 -- 172.21.15.53:0/3767040449 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f64c0039660 msgr2=0x7f64c003bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/3887954889 2024-04-25T14:30:15.901 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:10.215+0000 7f64d3e00700 1 --2- 172.21.15.53:0/3767040449 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f64c0039660 0x7f64c003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T14:30:15.901 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:11.817+0000 7f64d3e00700 1 -- 172.21.15.53:0/3767040449 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f64c0039660 msgr2=0x7f64c003bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/3887954889 2024-04-25T14:30:15.901 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:11.817+0000 7f64d3e00700 1 --2- 172.21.15.53:0/3767040449 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f64c0039660 0x7f64c003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T14:30:15.901 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:14.166+0000 7f64da000700 1 -- 172.21.15.53:0/3767040449 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f64cc00b3e0 con 0x7f64dc109070 2024-04-25T14:30:15.901 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:14.167+0000 7f64da000700 1 -- 172.21.15.53:0/3767040449 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f64c0039660 msgr2=0x7f64c003bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:30:15.901 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:14.167+0000 7f64da000700 1 --2- 172.21.15.53:0/3767040449 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f64c0039660 0x7f64c003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:15.901 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:15.169+0000 7f64da000700 1 -- 172.21.15.53:0/3767040449 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f64cc04feb0 con 0x7f64dc109070 2024-04-25T14:30:15.901 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:15.170+0000 7f64da000700 1 --2- 172.21.15.53:0/3767040449 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f64c003cff0 0x7f64c003f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:15.901 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:15.170+0000 7f64da000700 1 -- 172.21.15.53:0/3767040449 --> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f64c003c230 con 0x7f64c003cff0 2024-04-25T14:30:15.901 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:15.171+0000 7f64d3e00700 1 --2- 172.21.15.53:0/3767040449 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f64c003cff0 0x7f64c003f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:30:15.901 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:15.174+0000 7f64d3e00700 1 --2- 172.21.15.53:0/3767040449 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f64c003cff0 0x7f64c003f3e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f64c4003a10 tx=0x7f64c4009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:30:15.901 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:15.175+0000 7f64da000700 1 -- 172.21.15.53:0/3767040449 <== mgr.14152 v2:172.21.15.53:6800/4154933108 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f64c003c230 con 0x7f64c003cff0 2024-04-25T14:30:15.902 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:15.182+0000 7f64e3400700 1 -- 172.21.15.53:0/3767040449 --> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f64dc069ba0 con 0x7f64c003cff0 2024-04-25T14:30:15.902 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:15.183+0000 7f64da000700 1 -- 172.21.15.53:0/3767040449 <== mgr.14152 v2:172.21.15.53:6800/4154933108 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f64dc069ba0 con 0x7f64c003cff0 2024-04-25T14:30:15.902 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:15.184+0000 7f64e3400700 1 -- 172.21.15.53:0/3767040449 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f64c003cff0 msgr2=0x7f64c003f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:15.902 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:15.184+0000 7f64e3400700 1 --2- 172.21.15.53:0/3767040449 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f64c003cff0 0x7f64c003f3e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f64c4003a10 tx=0x7f64c4009270 comp rx=0 tx=0).stop 2024-04-25T14:30:15.902 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:15.184+0000 7f64e3400700 1 -- 172.21.15.53:0/3767040449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64dc109070 msgr2=0x7f64dc1088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:15.902 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:15.184+0000 7f64e3400700 1 --2- 172.21.15.53:0/3767040449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64dc109070 0x7f64dc1088f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f64cc000c00 tx=0x7f64cc011aa0 comp rx=0 tx=0).stop 2024-04-25T14:30:15.902 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:15.184+0000 7f64e3400700 1 -- 172.21.15.53:0/3767040449 shutdown_connections 2024-04-25T14:30:15.902 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:15.184+0000 7f64e3400700 1 --2- 172.21.15.53:0/3767040449 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f64c003cff0 0x7f64c003f3e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:15.902 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:15.184+0000 7f64e3400700 1 --2- 172.21.15.53:0/3767040449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64dc109070 0x7f64dc1088f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:15.902 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:15.184+0000 7f64e3400700 1 --2- 172.21.15.53:0/3767040449 >> [v2:172.21.15.53:6800/3887954889,v1:172.21.15.53:6801/3887954889] conn(0x7f64c0039660 0x7f64c003bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:15.902 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:15.184+0000 7f64e3400700 1 -- 172.21.15.53:0/3767040449 >> 172.21.15.53:0/3767040449 conn(0x7f64dc102a40 msgr2=0x7f64dc103510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:30:15.902 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:15.184+0000 7f64e3400700 1 -- 172.21.15.53:0/3767040449 shutdown_connections 2024-04-25T14:30:15.902 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:15.184+0000 7f64e3400700 1 -- 172.21.15.53:0/3767040449 wait complete. 2024-04-25T14:30:15.902 INFO:teuthology.orchestra.run.smithi053.stdout:mgr epoch 9 is available 2024-04-25T14:30:15.903 INFO:teuthology.orchestra.run.smithi053.stdout:Generating a dashboard self-signed certificate... 2024-04-25T14:30:16.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:16 smithi053 ceph-mon[27773]: [25/Apr/2024:14:30:15] ENGINE Serving on https://172.21.15.53:7150 2024-04-25T14:30:16.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:16 smithi053 ceph-mon[27773]: [25/Apr/2024:14:30:15] ENGINE Serving on http://172.21.15.53:8765 2024-04-25T14:30:16.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:16 smithi053 ceph-mon[27773]: [25/Apr/2024:14:30:15] ENGINE Bus STARTED 2024-04-25T14:30:16.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:16 smithi053 ceph-mon[27773]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T14:30:16.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:16 smithi053 ceph-mon[27773]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T14:30:17.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:17 smithi053 ceph-mon[27773]: mgrmap e12: x(active, since 2s) 2024-04-25T14:30:18.047 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-25T14:30:18.048 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:16.934+0000 7fe94f200700 1 Processor -- start 2024-04-25T14:30:18.048 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:16.934+0000 7fe94f200700 1 -- start start 2024-04-25T14:30:18.048 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:16.935+0000 7fe94f200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe94807de20 0x7fe94807c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:18.048 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:16.935+0000 7fe94f200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe94807c810 con 0x7fe94807de20 2024-04-25T14:30:18.048 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:16.935+0000 7fe947e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe94807de20 0x7fe94807c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:30:18.048 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:16.935+0000 7fe947e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe94807de20 0x7fe94807c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54758/0 (socket says 172.21.15.53:54758) 2024-04-25T14:30:18.048 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:16.935+0000 7fe947e00700 1 -- 172.21.15.53:0/3669019181 learned_addr learned my addr 172.21.15.53:0/3669019181 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:30:18.048 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:16.935+0000 7fe947e00700 1 -- 172.21.15.53:0/3669019181 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe94807c990 con 0x7fe94807de20 2024-04-25T14:30:18.048 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:16.936+0000 7fe947e00700 1 --2- 172.21.15.53:0/3669019181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe94807de20 0x7fe94807c2d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe9380098a0 tx=0x7fe938009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e924d37abce09231 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:30:18.049 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:16.936+0000 7fe947400700 1 -- 172.21.15.53:0/3669019181 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe938004160 con 0x7fe94807de20 2024-04-25T14:30:18.049 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:16.936+0000 7fe947400700 1 -- 172.21.15.53:0/3669019181 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe938004300 con 0x7fe94807de20 2024-04-25T14:30:18.049 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:16.936+0000 7fe947400700 1 -- 172.21.15.53:0/3669019181 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe938011a20 con 0x7fe94807de20 2024-04-25T14:30:18.049 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:16.937+0000 7fe94f200700 1 -- 172.21.15.53:0/3669019181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe94807de20 msgr2=0x7fe94807c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:18.049 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:16.937+0000 7fe94f200700 1 --2- 172.21.15.53:0/3669019181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe94807de20 0x7fe94807c2d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe9380098a0 tx=0x7fe938009bb0 comp rx=0 tx=0).stop 2024-04-25T14:30:18.049 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:16.937+0000 7fe94f200700 1 -- 172.21.15.53:0/3669019181 shutdown_connections 2024-04-25T14:30:18.049 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:16.937+0000 7fe94f200700 1 --2- 172.21.15.53:0/3669019181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe94807de20 0x7fe94807c2d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:18.049 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:16.937+0000 7fe94f200700 1 -- 172.21.15.53:0/3669019181 >> 172.21.15.53:0/3669019181 conn(0x7fe948103b10 msgr2=0x7fe948105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:30:18.049 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:16.937+0000 7fe94f200700 1 -- 172.21.15.53:0/3669019181 shutdown_connections 2024-04-25T14:30:18.049 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:16.937+0000 7fe94f200700 1 -- 172.21.15.53:0/3669019181 wait complete. 2024-04-25T14:30:18.050 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:16.938+0000 7fe94f200700 1 Processor -- start 2024-04-25T14:30:18.050 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:16.938+0000 7fe94f200700 1 -- start start 2024-04-25T14:30:18.050 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:16.939+0000 7fe94f200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe94807de20 0x7fe94819e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:18.050 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:16.939+0000 7fe94f200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe94810ead0 con 0x7fe94807de20 2024-04-25T14:30:18.050 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:16.939+0000 7fe947e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe94807de20 0x7fe94819e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:30:18.050 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:16.939+0000 7fe947e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe94807de20 0x7fe94819e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54772/0 (socket says 172.21.15.53:54772) 2024-04-25T14:30:18.050 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:16.939+0000 7fe947e00700 1 -- 172.21.15.53:0/4030883188 learned_addr learned my addr 172.21.15.53:0/4030883188 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:30:18.050 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:16.939+0000 7fe947e00700 1 -- 172.21.15.53:0/4030883188 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe94819eb70 con 0x7fe94807de20 2024-04-25T14:30:18.051 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:16.940+0000 7fe947e00700 1 --2- 172.21.15.53:0/4030883188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe94807de20 0x7fe94819e630 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe938004660 tx=0x7fe938011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:30:18.051 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:16.940+0000 7fe945600700 1 -- 172.21.15.53:0/4030883188 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe938003aa0 con 0x7fe94807de20 2024-04-25T14:30:18.051 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:16.940+0000 7fe945600700 1 -- 172.21.15.53:0/4030883188 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe93801b8f0 con 0x7fe94807de20 2024-04-25T14:30:18.051 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:16.940+0000 7fe945600700 1 -- 172.21.15.53:0/4030883188 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe938023900 con 0x7fe94807de20 2024-04-25T14:30:18.051 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:16.940+0000 7fe94f200700 1 -- 172.21.15.53:0/4030883188 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe94819ee00 con 0x7fe94807de20 2024-04-25T14:30:18.051 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:16.940+0000 7fe94f200700 1 -- 172.21.15.53:0/4030883188 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9481a1af0 con 0x7fe94807de20 2024-04-25T14:30:18.051 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:16.941+0000 7fe945600700 1 -- 172.21.15.53:0/4030883188 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fe938023b70 con 0x7fe94807de20 2024-04-25T14:30:18.051 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:16.941+0000 7fe94f200700 1 -- 172.21.15.53:0/4030883188 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9480640a0 con 0x7fe94807de20 2024-04-25T14:30:18.051 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:16.942+0000 7fe945600700 1 --2- 172.21.15.53:0/4030883188 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fe92c039660 0x7fe92c03bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:18.051 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:16.942+0000 7fe945600700 1 -- 172.21.15.53:0/4030883188 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe93804df90 con 0x7fe94807de20 2024-04-25T14:30:18.052 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:16.942+0000 7fe93fe00700 1 --2- 172.21.15.53:0/4030883188 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fe92c039660 0x7fe92c03bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:30:18.052 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:16.943+0000 7fe93fe00700 1 --2- 172.21.15.53:0/4030883188 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fe92c039660 0x7fe92c03bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe930006fd0 tx=0x7fe930006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:30:18.052 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:16.948+0000 7fe945600700 1 -- 172.21.15.53:0/4030883188 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe93801f030 con 0x7fe94807de20 2024-04-25T14:30:18.052 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:17.224+0000 7fe94f200700 1 -- 172.21.15.53:0/4030883188 --> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fe948104d80 con 0x7fe92c039660 2024-04-25T14:30:18.052 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:17.383+0000 7fe945600700 1 -- 172.21.15.53:0/4030883188 <== mgr.14152 v2:172.21.15.53:6800/4154933108 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fe948104d80 con 0x7fe92c039660 2024-04-25T14:30:18.052 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:17.386+0000 7fe94f200700 1 -- 172.21.15.53:0/4030883188 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fe92c039660 msgr2=0x7fe92c03bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:18.052 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:17.386+0000 7fe94f200700 1 --2- 172.21.15.53:0/4030883188 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fe92c039660 0x7fe92c03bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe930006fd0 tx=0x7fe930006e40 comp rx=0 tx=0).stop 2024-04-25T14:30:18.052 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:17.386+0000 7fe94f200700 1 -- 172.21.15.53:0/4030883188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe94807de20 msgr2=0x7fe94819e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:18.052 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:17.386+0000 7fe94f200700 1 --2- 172.21.15.53:0/4030883188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe94807de20 0x7fe94819e630 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe938004660 tx=0x7fe938011aa0 comp rx=0 tx=0).stop 2024-04-25T14:30:18.053 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:17.386+0000 7fe94f200700 1 -- 172.21.15.53:0/4030883188 shutdown_connections 2024-04-25T14:30:18.053 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:17.386+0000 7fe94f200700 1 --2- 172.21.15.53:0/4030883188 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fe92c039660 0x7fe92c03bb20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:18.053 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:17.386+0000 7fe94f200700 1 --2- 172.21.15.53:0/4030883188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe94807de20 0x7fe94819e630 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:18.053 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:17.386+0000 7fe94f200700 1 -- 172.21.15.53:0/4030883188 >> 172.21.15.53:0/4030883188 conn(0x7fe948103b10 msgr2=0x7fe948104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:30:18.053 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:17.386+0000 7fe94f200700 1 -- 172.21.15.53:0/4030883188 shutdown_connections 2024-04-25T14:30:18.053 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:17.387+0000 7fe94f200700 1 -- 172.21.15.53:0/4030883188 wait complete. 2024-04-25T14:30:18.053 INFO:teuthology.orchestra.run.smithi053.stdout:Creating initial admin user... 2024-04-25T14:30:18.617 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:18 smithi053 ceph-mon[27773]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:30:18.617 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:18 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:30:18.617 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:18 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:30:18.617 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:18 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:30:19.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:19 smithi053 ceph-mon[27773]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:30:19.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:19 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:30:19.886 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$W9SU0d29OLXMvfsKs1v8peA6xGV3t/rLyiTaqzTIW8rRS65wJPEiW", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714055419, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-25T14:30:19.886 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:18.652+0000 7fa0ca800700 1 Processor -- start 2024-04-25T14:30:19.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:18.652+0000 7fa0ca800700 1 -- start start 2024-04-25T14:30:19.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:18.653+0000 7fa0ca800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0c4108fb0 0x7fa0c410b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:19.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:18.653+0000 7fa0ca800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c4063d60 con 0x7fa0c4108fb0 2024-04-25T14:30:19.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:18.653+0000 7fa0c3400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0c4108fb0 0x7fa0c410b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:30:19.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:18.653+0000 7fa0c3400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0c4108fb0 0x7fa0c410b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54786/0 (socket says 172.21.15.53:54786) 2024-04-25T14:30:19.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:18.653+0000 7fa0c3400700 1 -- 172.21.15.53:0/815712414 learned_addr learned my addr 172.21.15.53:0/815712414 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:30:19.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:18.654+0000 7fa0c3400700 1 -- 172.21.15.53:0/815712414 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c410b8e0 con 0x7fa0c4108fb0 2024-04-25T14:30:19.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:18.654+0000 7fa0c3400700 1 --2- 172.21.15.53:0/815712414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0c4108fb0 0x7fa0c410b3a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa0b40098a0 tx=0x7fa0b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9018f63e6d3b88eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:30:19.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:18.654+0000 7fa0c2000700 1 -- 172.21.15.53:0/815712414 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0b4004160 con 0x7fa0c4108fb0 2024-04-25T14:30:19.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:18.654+0000 7fa0c2000700 1 -- 172.21.15.53:0/815712414 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0b4004300 con 0x7fa0c4108fb0 2024-04-25T14:30:19.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:18.654+0000 7fa0c2000700 1 -- 172.21.15.53:0/815712414 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0b4011a20 con 0x7fa0c4108fb0 2024-04-25T14:30:19.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:18.655+0000 7fa0ca800700 1 -- 172.21.15.53:0/815712414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0c4108fb0 msgr2=0x7fa0c410b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:19.887 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:18.655+0000 7fa0ca800700 1 --2- 172.21.15.53:0/815712414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0c4108fb0 0x7fa0c410b3a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa0b40098a0 tx=0x7fa0b4009bb0 comp rx=0 tx=0).stop 2024-04-25T14:30:19.889 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:18.655+0000 7fa0ca800700 1 -- 172.21.15.53:0/815712414 shutdown_connections 2024-04-25T14:30:19.889 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:18.655+0000 7fa0ca800700 1 --2- 172.21.15.53:0/815712414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0c4108fb0 0x7fa0c410b3a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:19.889 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:18.655+0000 7fa0ca800700 1 -- 172.21.15.53:0/815712414 >> 172.21.15.53:0/815712414 conn(0x7fa0c4102a40 msgr2=0x7fa0c4104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:30:19.889 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:18.655+0000 7fa0ca800700 1 -- 172.21.15.53:0/815712414 shutdown_connections 2024-04-25T14:30:19.889 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:18.656+0000 7fa0ca800700 1 -- 172.21.15.53:0/815712414 wait complete. 2024-04-25T14:30:19.889 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:18.657+0000 7fa0ca800700 1 Processor -- start 2024-04-25T14:30:19.889 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:18.657+0000 7fa0ca800700 1 -- start start 2024-04-25T14:30:19.889 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:18.657+0000 7fa0ca800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0c4108fb0 0x7fa0c4108830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:19.889 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:18.657+0000 7fa0ca800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c410c8f0 con 0x7fa0c4108fb0 2024-04-25T14:30:19.889 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:18.658+0000 7fa0c3400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0c4108fb0 0x7fa0c4108830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:30:19.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:18.658+0000 7fa0c3400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0c4108fb0 0x7fa0c4108830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54802/0 (socket says 172.21.15.53:54802) 2024-04-25T14:30:19.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:18.658+0000 7fa0c3400700 1 -- 172.21.15.53:0/567058511 learned_addr learned my addr 172.21.15.53:0/567058511 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:30:19.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:18.658+0000 7fa0c3400700 1 -- 172.21.15.53:0/567058511 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c4108d70 con 0x7fa0c4108fb0 2024-04-25T14:30:19.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:18.658+0000 7fa0c3400700 1 --2- 172.21.15.53:0/567058511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0c4108fb0 0x7fa0c4108830 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4000c00 tx=0x7fa0b4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:30:19.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:18.659+0000 7fa0bbe00700 1 -- 172.21.15.53:0/567058511 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0b4003aa0 con 0x7fa0c4108fb0 2024-04-25T14:30:19.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:18.659+0000 7fa0bbe00700 1 -- 172.21.15.53:0/567058511 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0b401b8f0 con 0x7fa0c4108fb0 2024-04-25T14:30:19.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:18.659+0000 7fa0ca800700 1 -- 172.21.15.53:0/567058511 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0c4106f60 con 0x7fa0c4108fb0 2024-04-25T14:30:19.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:18.659+0000 7fa0bbe00700 1 -- 172.21.15.53:0/567058511 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0b4023870 con 0x7fa0c4108fb0 2024-04-25T14:30:19.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:18.659+0000 7fa0ca800700 1 -- 172.21.15.53:0/567058511 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0c41074a0 con 0x7fa0c4108fb0 2024-04-25T14:30:19.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:18.660+0000 7fa0bbe00700 1 -- 172.21.15.53:0/567058511 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fa0b4023b70 con 0x7fa0c4108fb0 2024-04-25T14:30:19.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:18.660+0000 7fa0ca800700 1 -- 172.21.15.53:0/567058511 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0c40640a0 con 0x7fa0c4108fb0 2024-04-25T14:30:19.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:18.661+0000 7fa0bbe00700 1 --2- 172.21.15.53:0/567058511 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fa0a8039600 0x7fa0a803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:19.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:18.661+0000 7fa0bbe00700 1 -- 172.21.15.53:0/567058511 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa0b404e060 con 0x7fa0c4108fb0 2024-04-25T14:30:19.890 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:18.661+0000 7fa0c2a00700 1 --2- 172.21.15.53:0/567058511 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fa0a8039600 0x7fa0a803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:30:19.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:18.662+0000 7fa0c2a00700 1 --2- 172.21.15.53:0/567058511 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fa0a8039600 0x7fa0a803bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0006fd0 tx=0x7fa0b0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:30:19.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:18.667+0000 7fa0bbe00700 1 -- 172.21.15.53:0/567058511 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa0b401f030 con 0x7fa0c4108fb0 2024-04-25T14:30:19.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:18.923+0000 7fa0ca800700 1 -- 172.21.15.53:0/567058511 --> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fa0c4107720 con 0x7fa0a8039600 2024-04-25T14:30:19.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:19.203+0000 7fa0bbe00700 1 -- 172.21.15.53:0/567058511 <== mgr.14152 v2:172.21.15.53:6800/4154933108 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fa0c4107720 con 0x7fa0a8039600 2024-04-25T14:30:19.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:19.206+0000 7fa0ca800700 1 -- 172.21.15.53:0/567058511 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fa0a8039600 msgr2=0x7fa0a803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:19.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:19.206+0000 7fa0ca800700 1 --2- 172.21.15.53:0/567058511 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fa0a8039600 0x7fa0a803bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0006fd0 tx=0x7fa0b0006e40 comp rx=0 tx=0).stop 2024-04-25T14:30:19.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:19.206+0000 7fa0ca800700 1 -- 172.21.15.53:0/567058511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0c4108fb0 msgr2=0x7fa0c4108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:19.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:19.206+0000 7fa0ca800700 1 --2- 172.21.15.53:0/567058511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0c4108fb0 0x7fa0c4108830 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4000c00 tx=0x7fa0b4011aa0 comp rx=0 tx=0).stop 2024-04-25T14:30:19.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:19.206+0000 7fa0ca800700 1 -- 172.21.15.53:0/567058511 shutdown_connections 2024-04-25T14:30:19.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:19.206+0000 7fa0ca800700 1 --2- 172.21.15.53:0/567058511 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fa0a8039600 0x7fa0a803bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:19.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:19.206+0000 7fa0ca800700 1 --2- 172.21.15.53:0/567058511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0c4108fb0 0x7fa0c4108830 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:19.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:19.206+0000 7fa0ca800700 1 -- 172.21.15.53:0/567058511 >> 172.21.15.53:0/567058511 conn(0x7fa0c4102a40 msgr2=0x7fa0c4103490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:30:19.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:19.206+0000 7fa0ca800700 1 -- 172.21.15.53:0/567058511 shutdown_connections 2024-04-25T14:30:19.891 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:19.206+0000 7fa0ca800700 1 -- 172.21.15.53:0/567058511 wait complete. 2024-04-25T14:30:19.891 INFO:teuthology.orchestra.run.smithi053.stdout:Fetching dashboard port number... 2024-04-25T14:30:20.814 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:20 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:30:21.997 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:21 smithi053 ceph-mon[27773]: mgrmap e13: x(active, since 6s) 2024-04-25T14:30:21.997 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:21 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/2202123070' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-25T14:30:22.518 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stdout 8443 2024-04-25T14:30:22.518 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.536+0000 7f2e7ae00700 1 Processor -- start 2024-04-25T14:30:22.519 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.537+0000 7f2e7ae00700 1 -- start start 2024-04-25T14:30:22.519 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.537+0000 7f2e7ae00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e7407de40 0x7f2e7407c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:22.520 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.537+0000 7f2e7ae00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2e7407c830 con 0x7f2e7407de40 2024-04-25T14:30:22.520 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.538+0000 7f2e73e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e7407de40 0x7f2e7407c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:30:22.520 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.538+0000 7f2e73e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e7407de40 0x7f2e7407c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54804/0 (socket says 172.21.15.53:54804) 2024-04-25T14:30:22.520 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.538+0000 7f2e73e00700 1 -- 172.21.15.53:0/191064283 learned_addr learned my addr 172.21.15.53:0/191064283 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:30:22.520 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.538+0000 7f2e73e00700 1 -- 172.21.15.53:0/191064283 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e7407c9b0 con 0x7f2e7407de40 2024-04-25T14:30:22.520 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.539+0000 7f2e73e00700 1 --2- 172.21.15.53:0/191064283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e7407de40 0x7f2e7407c2f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2e640098a0 tx=0x7f2e64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6591e8490640c28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:30:22.520 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.539+0000 7f2e72a00700 1 -- 172.21.15.53:0/191064283 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e64004160 con 0x7f2e7407de40 2024-04-25T14:30:22.520 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.539+0000 7f2e72a00700 1 -- 172.21.15.53:0/191064283 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e64004300 con 0x7f2e7407de40 2024-04-25T14:30:22.520 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.539+0000 7f2e72a00700 1 -- 172.21.15.53:0/191064283 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e64011a20 con 0x7f2e7407de40 2024-04-25T14:30:22.521 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.540+0000 7f2e7ae00700 1 -- 172.21.15.53:0/191064283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e7407de40 msgr2=0x7f2e7407c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:22.523 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.540+0000 7f2e7ae00700 1 --2- 172.21.15.53:0/191064283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e7407de40 0x7f2e7407c2f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2e640098a0 tx=0x7f2e64009bb0 comp rx=0 tx=0).stop 2024-04-25T14:30:22.523 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.540+0000 7f2e7ae00700 1 -- 172.21.15.53:0/191064283 shutdown_connections 2024-04-25T14:30:22.523 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.540+0000 7f2e7ae00700 1 --2- 172.21.15.53:0/191064283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e7407de40 0x7f2e7407c2f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:22.523 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.540+0000 7f2e7ae00700 1 -- 172.21.15.53:0/191064283 >> 172.21.15.53:0/191064283 conn(0x7f2e74103b40 msgr2=0x7f2e74105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:30:22.523 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.540+0000 7f2e7ae00700 1 -- 172.21.15.53:0/191064283 shutdown_connections 2024-04-25T14:30:22.523 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.540+0000 7f2e7ae00700 1 -- 172.21.15.53:0/191064283 wait complete. 2024-04-25T14:30:22.523 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.541+0000 7f2e7ae00700 1 Processor -- start 2024-04-25T14:30:22.523 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.542+0000 7f2e7ae00700 1 -- start start 2024-04-25T14:30:22.523 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.542+0000 7f2e7ae00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e7407de40 0x7f2e7419e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:22.523 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.542+0000 7f2e7ae00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2e7410eac0 con 0x7f2e7407de40 2024-04-25T14:30:22.523 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.542+0000 7f2e73e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e7407de40 0x7f2e7419e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:30:22.524 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.542+0000 7f2e73e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e7407de40 0x7f2e7419e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54806/0 (socket says 172.21.15.53:54806) 2024-04-25T14:30:22.524 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.542+0000 7f2e73e00700 1 -- 172.21.15.53:0/2202123070 learned_addr learned my addr 172.21.15.53:0/2202123070 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:30:22.524 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.543+0000 7f2e73e00700 1 -- 172.21.15.53:0/2202123070 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e7419eba0 con 0x7f2e7407de40 2024-04-25T14:30:22.524 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.543+0000 7f2e73e00700 1 --2- 172.21.15.53:0/2202123070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e7407de40 0x7f2e7419e660 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2e64011a60 tx=0x7f2e64011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:30:22.524 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.543+0000 7f2e70c00700 1 -- 172.21.15.53:0/2202123070 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e64003b20 con 0x7f2e7407de40 2024-04-25T14:30:22.524 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.544+0000 7f2e70c00700 1 -- 172.21.15.53:0/2202123070 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e6401b8f0 con 0x7f2e7407de40 2024-04-25T14:30:22.524 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.544+0000 7f2e70c00700 1 -- 172.21.15.53:0/2202123070 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e64023900 con 0x7f2e7407de40 2024-04-25T14:30:22.524 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.544+0000 7f2e7ae00700 1 -- 172.21.15.53:0/2202123070 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e7419ee30 con 0x7f2e7407de40 2024-04-25T14:30:22.524 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.544+0000 7f2e7ae00700 1 -- 172.21.15.53:0/2202123070 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e741a1b20 con 0x7f2e7407de40 2024-04-25T14:30:22.526 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.545+0000 7f2e7ae00700 1 -- 172.21.15.53:0/2202123070 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e740640a0 con 0x7f2e7407de40 2024-04-25T14:30:22.526 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.546+0000 7f2e70c00700 1 -- 172.21.15.53:0/2202123070 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f2e6401b440 con 0x7f2e7407de40 2024-04-25T14:30:22.526 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.546+0000 7f2e70c00700 1 --2- 172.21.15.53:0/2202123070 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f2e58039660 0x7f2e5803bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:22.526 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.546+0000 7f2e70c00700 1 -- 172.21.15.53:0/2202123070 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2e6404e120 con 0x7f2e7407de40 2024-04-25T14:30:22.526 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.547+0000 7f2e73400700 1 --2- 172.21.15.53:0/2202123070 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f2e58039660 0x7f2e5803bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:30:22.527 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.547+0000 7f2e73400700 1 --2- 172.21.15.53:0/2202123070 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f2e58039660 0x7f2e5803bb20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2e60006fd0 tx=0x7f2e60006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:30:22.527 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.552+0000 7f2e70c00700 1 -- 172.21.15.53:0/2202123070 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2e6401f030 con 0x7f2e7407de40 2024-04-25T14:30:22.527 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.590+0000 7f2e70c00700 1 -- 172.21.15.53:0/2202123070 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f2e6402c770 con 0x7f2e7407de40 2024-04-25T14:30:22.527 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.814+0000 7f2e7ae00700 1 -- 172.21.15.53:0/2202123070 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f2e741a1cf0 con 0x7f2e7407de40 2024-04-25T14:30:22.527 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.815+0000 7f2e70c00700 1 -- 172.21.15.53:0/2202123070 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f2e64010380 con 0x7f2e7407de40 2024-04-25T14:30:22.527 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.818+0000 7f2e7ae00700 1 -- 172.21.15.53:0/2202123070 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f2e58039660 msgr2=0x7f2e5803bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:22.527 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.818+0000 7f2e7ae00700 1 --2- 172.21.15.53:0/2202123070 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f2e58039660 0x7f2e5803bb20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2e60006fd0 tx=0x7f2e60006e40 comp rx=0 tx=0).stop 2024-04-25T14:30:22.527 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.818+0000 7f2e7ae00700 1 -- 172.21.15.53:0/2202123070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e7407de40 msgr2=0x7f2e7419e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:22.527 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.818+0000 7f2e7ae00700 1 --2- 172.21.15.53:0/2202123070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e7407de40 0x7f2e7419e660 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2e64011a60 tx=0x7f2e64011b40 comp rx=0 tx=0).stop 2024-04-25T14:30:22.527 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.819+0000 7f2e7ae00700 1 -- 172.21.15.53:0/2202123070 shutdown_connections 2024-04-25T14:30:22.527 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.819+0000 7f2e7ae00700 1 --2- 172.21.15.53:0/2202123070 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f2e58039660 0x7f2e5803bb20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:22.527 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.819+0000 7f2e7ae00700 1 --2- 172.21.15.53:0/2202123070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e7407de40 0x7f2e7419e660 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:22.527 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.819+0000 7f2e7ae00700 1 -- 172.21.15.53:0/2202123070 >> 172.21.15.53:0/2202123070 conn(0x7f2e74103b40 msgr2=0x7f2e74104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:30:22.528 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.819+0000 7f2e7ae00700 1 -- 172.21.15.53:0/2202123070 shutdown_connections 2024-04-25T14:30:22.528 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:20.819+0000 7f2e7ae00700 1 -- 172.21.15.53:0/2202123070 wait complete. 2024-04-25T14:30:22.553 INFO:teuthology.orchestra.run.smithi053.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T14:30:22.553 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stdout disabled 2024-04-25T14:30:22.574 INFO:teuthology.orchestra.run.smithi053.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T14:30:22.574 INFO:teuthology.orchestra.run.smithi053.stdout:systemctl: stdout inactive 2024-04-25T14:30:22.575 INFO:teuthology.orchestra.run.smithi053.stdout:firewalld.service is not enabled 2024-04-25T14:30:22.575 INFO:teuthology.orchestra.run.smithi053.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-25T14:30:22.582 INFO:teuthology.orchestra.run.smithi053.stdout:Ceph Dashboard is now available at: 2024-04-25T14:30:22.582 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:30:22.582 INFO:teuthology.orchestra.run.smithi053.stdout: URL: https://smithi053.front.sepia.ceph.com:8443/ 2024-04-25T14:30:22.582 INFO:teuthology.orchestra.run.smithi053.stdout: User: admin 2024-04-25T14:30:22.582 INFO:teuthology.orchestra.run.smithi053.stdout: Password: qjccscwjdl 2024-04-25T14:30:22.582 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:30:22.584 INFO:teuthology.orchestra.run.smithi053.stdout:Saving cluster configuration to /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config directory 2024-04-25T14:30:22.586 INFO:teuthology.orchestra.run.smithi053.stdout:Enabling autotune for osd_memory_target 2024-04-25T14:30:24.174 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.201+0000 7fc4c9400700 1 Processor -- start 2024-04-25T14:30:24.174 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.201+0000 7fc4c9400700 1 -- start start 2024-04-25T14:30:24.174 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.201+0000 7fc4c9400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4c4108fb0 0x7fc4c410b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:24.174 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.201+0000 7fc4c9400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c4063d60 con 0x7fc4c4108fb0 2024-04-25T14:30:24.174 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.202+0000 7fc4c2800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4c4108fb0 0x7fc4c410b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:30:24.174 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.202+0000 7fc4c2800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4c4108fb0 0x7fc4c410b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57164/0 (socket says 172.21.15.53:57164) 2024-04-25T14:30:24.174 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.202+0000 7fc4c2800700 1 -- 172.21.15.53:0/2778646140 learned_addr learned my addr 172.21.15.53:0/2778646140 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:30:24.175 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.202+0000 7fc4c2800700 1 -- 172.21.15.53:0/2778646140 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c410b8e0 con 0x7fc4c4108fb0 2024-04-25T14:30:24.175 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.203+0000 7fc4c2800700 1 --2- 172.21.15.53:0/2778646140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4c4108fb0 0x7fc4c410b3a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4009e80 tx=0x7fc4b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5d6f02aed0b2b53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:30:24.175 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.203+0000 7fc4c1e00700 1 -- 172.21.15.53:0/2778646140 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4b4017070 con 0x7fc4c4108fb0 2024-04-25T14:30:24.175 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.203+0000 7fc4c1e00700 1 -- 172.21.15.53:0/2778646140 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4b400b8c0 con 0x7fc4c4108fb0 2024-04-25T14:30:24.175 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.203+0000 7fc4c1e00700 1 -- 172.21.15.53:0/2778646140 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4b4004490 con 0x7fc4c4108fb0 2024-04-25T14:30:24.175 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.204+0000 7fc4c9400700 1 -- 172.21.15.53:0/2778646140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4c4108fb0 msgr2=0x7fc4c410b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:24.175 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.204+0000 7fc4c9400700 1 --2- 172.21.15.53:0/2778646140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4c4108fb0 0x7fc4c410b3a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4009e80 tx=0x7fc4b400b330 comp rx=0 tx=0).stop 2024-04-25T14:30:24.175 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.204+0000 7fc4c9400700 1 -- 172.21.15.53:0/2778646140 shutdown_connections 2024-04-25T14:30:24.175 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.204+0000 7fc4c9400700 1 --2- 172.21.15.53:0/2778646140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4c4108fb0 0x7fc4c410b3a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:24.175 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.204+0000 7fc4c9400700 1 -- 172.21.15.53:0/2778646140 >> 172.21.15.53:0/2778646140 conn(0x7fc4c4102a40 msgr2=0x7fc4c4104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:30:24.175 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.204+0000 7fc4c9400700 1 -- 172.21.15.53:0/2778646140 shutdown_connections 2024-04-25T14:30:24.175 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.204+0000 7fc4c9400700 1 -- 172.21.15.53:0/2778646140 wait complete. 2024-04-25T14:30:24.175 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.205+0000 7fc4c9400700 1 Processor -- start 2024-04-25T14:30:24.175 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.205+0000 7fc4c9400700 1 -- start start 2024-04-25T14:30:24.176 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.206+0000 7fc4c9400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4c4108fb0 0x7fc4c419e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:24.176 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.206+0000 7fc4c9400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c410c8f0 con 0x7fc4c4108fb0 2024-04-25T14:30:24.176 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.206+0000 7fc4c2800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4c4108fb0 0x7fc4c419e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:30:24.176 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.206+0000 7fc4c2800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4c4108fb0 0x7fc4c419e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57168/0 (socket says 172.21.15.53:57168) 2024-04-25T14:30:24.176 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.206+0000 7fc4c2800700 1 -- 172.21.15.53:0/1698660106 learned_addr learned my addr 172.21.15.53:0/1698660106 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:30:24.176 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.207+0000 7fc4c2800700 1 -- 172.21.15.53:0/1698660106 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c419eb50 con 0x7fc4c4108fb0 2024-04-25T14:30:24.176 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.207+0000 7fc4c2800700 1 --2- 172.21.15.53:0/1698660106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4c4108fb0 0x7fc4c419e610 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4004220 tx=0x7fc4b40093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:30:24.176 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.207+0000 7fc4bbe00700 1 -- 172.21.15.53:0/1698660106 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4b4020020 con 0x7fc4c4108fb0 2024-04-25T14:30:24.176 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.207+0000 7fc4bbe00700 1 -- 172.21.15.53:0/1698660106 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4b4003b50 con 0x7fc4c4108fb0 2024-04-25T14:30:24.176 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.207+0000 7fc4bbe00700 1 -- 172.21.15.53:0/1698660106 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4b4017040 con 0x7fc4c4108fb0 2024-04-25T14:30:24.176 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.207+0000 7fc4c9400700 1 -- 172.21.15.53:0/1698660106 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4c419ede0 con 0x7fc4c4108fb0 2024-04-25T14:30:24.176 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.207+0000 7fc4c9400700 1 -- 172.21.15.53:0/1698660106 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4c41a1ad0 con 0x7fc4c4108fb0 2024-04-25T14:30:24.176 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.209+0000 7fc4c9400700 1 -- 172.21.15.53:0/1698660106 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4c40640a0 con 0x7fc4c4108fb0 2024-04-25T14:30:24.176 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.209+0000 7fc4bbe00700 1 -- 172.21.15.53:0/1698660106 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fc4b4023020 con 0x7fc4c4108fb0 2024-04-25T14:30:24.177 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.209+0000 7fc4bbe00700 1 --2- 172.21.15.53:0/1698660106 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fc4a80396f0 0x7fc4a803bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:24.177 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.209+0000 7fc4bbe00700 1 -- 172.21.15.53:0/1698660106 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc4b400f050 con 0x7fc4c4108fb0 2024-04-25T14:30:24.177 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.210+0000 7fc4b9e00700 1 --2- 172.21.15.53:0/1698660106 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fc4a80396f0 0x7fc4a803bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:30:24.177 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.210+0000 7fc4b9e00700 1 --2- 172.21.15.53:0/1698660106 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fc4a80396f0 0x7fc4a803bbb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac006fd0 tx=0x7fc4ac006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:30:24.177 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.215+0000 7fc4bbe00700 1 -- 172.21.15.53:0/1698660106 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc4b402a020 con 0x7fc4c4108fb0 2024-04-25T14:30:24.177 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.453+0000 7fc4c9400700 1 -- 172.21.15.53:0/1698660106 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fc4c41a20b0 con 0x7fc4c4108fb0 2024-04-25T14:30:24.177 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.454+0000 7fc4bbe00700 1 -- 172.21.15.53:0/1698660106 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fc4b402c300 con 0x7fc4c4108fb0 2024-04-25T14:30:24.177 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.458+0000 7fc4c9400700 1 -- 172.21.15.53:0/1698660106 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fc4a80396f0 msgr2=0x7fc4a803bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:24.177 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.458+0000 7fc4c9400700 1 --2- 172.21.15.53:0/1698660106 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fc4a80396f0 0x7fc4a803bbb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac006fd0 tx=0x7fc4ac006e40 comp rx=0 tx=0).stop 2024-04-25T14:30:24.177 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.458+0000 7fc4c9400700 1 -- 172.21.15.53:0/1698660106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4c4108fb0 msgr2=0x7fc4c419e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:24.177 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.458+0000 7fc4c9400700 1 --2- 172.21.15.53:0/1698660106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4c4108fb0 0x7fc4c419e610 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4004220 tx=0x7fc4b40093e0 comp rx=0 tx=0).stop 2024-04-25T14:30:24.177 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.458+0000 7fc4c9400700 1 -- 172.21.15.53:0/1698660106 shutdown_connections 2024-04-25T14:30:24.177 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.458+0000 7fc4c9400700 1 --2- 172.21.15.53:0/1698660106 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fc4a80396f0 0x7fc4a803bbb0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:24.177 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.458+0000 7fc4c9400700 1 --2- 172.21.15.53:0/1698660106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4c4108fb0 0x7fc4c419e610 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:24.178 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.458+0000 7fc4c9400700 1 -- 172.21.15.53:0/1698660106 >> 172.21.15.53:0/1698660106 conn(0x7fc4c4102a40 msgr2=0x7fc4c41034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:30:24.178 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.458+0000 7fc4c9400700 1 -- 172.21.15.53:0/1698660106 shutdown_connections 2024-04-25T14:30:24.178 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:23.458+0000 7fc4c9400700 1 -- 172.21.15.53:0/1698660106 wait complete. 2024-04-25T14:30:25.773 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:24.793+0000 7f940b800700 1 Processor -- start 2024-04-25T14:30:25.773 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:24.793+0000 7f940b800700 1 -- start start 2024-04-25T14:30:25.773 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:24.793+0000 7f940b800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9404110220 0x7f9404110640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:25.773 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:24.793+0000 7f940b800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9404110c10 con 0x7f9404110220 2024-04-25T14:30:25.773 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:24.793+0000 7f9408e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9404110220 0x7f9404110640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:30:25.773 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:24.794+0000 7f9408e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9404110220 0x7f9404110640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57184/0 (socket says 172.21.15.53:57184) 2024-04-25T14:30:25.773 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:24.794+0000 7f9408e00700 1 -- 172.21.15.53:0/3760721358 learned_addr learned my addr 172.21.15.53:0/3760721358 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:30:25.773 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:24.794+0000 7f9408e00700 1 -- 172.21.15.53:0/3760721358 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f940410ece0 con 0x7f9404110220 2024-04-25T14:30:25.773 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:24.794+0000 7f9408e00700 1 --2- 172.21.15.53:0/3760721358 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9404110220 0x7f9404110640 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f93f40098a0 tx=0x7f93f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=508bb198c443fa72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:30:25.773 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:24.795+0000 7f9403e00700 1 -- 172.21.15.53:0/3760721358 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93f4004160 con 0x7f9404110220 2024-04-25T14:30:25.773 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:24.795+0000 7f9403e00700 1 -- 172.21.15.53:0/3760721358 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93f4004300 con 0x7f9404110220 2024-04-25T14:30:25.774 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:24.795+0000 7f9403e00700 1 -- 172.21.15.53:0/3760721358 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93f4011a20 con 0x7f9404110220 2024-04-25T14:30:25.774 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:24.795+0000 7f940b800700 1 -- 172.21.15.53:0/3760721358 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9404110220 msgr2=0x7f9404110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:25.778 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:24.795+0000 7f940b800700 1 --2- 172.21.15.53:0/3760721358 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9404110220 0x7f9404110640 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f93f40098a0 tx=0x7f93f4009bb0 comp rx=0 tx=0).stop 2024-04-25T14:30:25.778 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:24.795+0000 7f940b800700 1 -- 172.21.15.53:0/3760721358 shutdown_connections 2024-04-25T14:30:25.778 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:24.795+0000 7f940b800700 1 --2- 172.21.15.53:0/3760721358 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9404110220 0x7f9404110640 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:25.778 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:24.795+0000 7f940b800700 1 -- 172.21.15.53:0/3760721358 >> 172.21.15.53:0/3760721358 conn(0x7f940407de20 msgr2=0x7f9404080240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:30:25.778 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:24.796+0000 7f940b800700 1 -- 172.21.15.53:0/3760721358 shutdown_connections 2024-04-25T14:30:25.779 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:24.796+0000 7f940b800700 1 -- 172.21.15.53:0/3760721358 wait complete. 2024-04-25T14:30:25.779 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:24.797+0000 7f940b800700 1 Processor -- start 2024-04-25T14:30:25.779 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:24.797+0000 7f940b800700 1 -- start start 2024-04-25T14:30:25.779 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:24.797+0000 7f940b800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9404110220 0x7f94041a2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:25.779 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:24.797+0000 7f940b800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9404113140 con 0x7f9404110220 2024-04-25T14:30:25.779 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:24.798+0000 7f9408e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9404110220 0x7f94041a2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:30:25.779 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:24.798+0000 7f9408e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9404110220 0x7f94041a2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57192/0 (socket says 172.21.15.53:57192) 2024-04-25T14:30:25.779 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:24.798+0000 7f9408e00700 1 -- 172.21.15.53:0/2827083929 learned_addr learned my addr 172.21.15.53:0/2827083929 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:30:25.779 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:24.798+0000 7f9408e00700 1 -- 172.21.15.53:0/2827083929 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94041a3120 con 0x7f9404110220 2024-04-25T14:30:25.779 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:24.798+0000 7f9408e00700 1 --2- 172.21.15.53:0/2827083929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9404110220 0x7f94041a2be0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f93f40119f0 tx=0x7f93f4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:30:25.779 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:24.799+0000 7f9402000700 1 -- 172.21.15.53:0/2827083929 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93f40039e0 con 0x7f9404110220 2024-04-25T14:30:25.779 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:24.799+0000 7f9402000700 1 -- 172.21.15.53:0/2827083929 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93f401b8f0 con 0x7f9404110220 2024-04-25T14:30:25.779 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:24.799+0000 7f940b800700 1 -- 172.21.15.53:0/2827083929 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94041a33b0 con 0x7f9404110220 2024-04-25T14:30:25.779 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:24.799+0000 7f9402000700 1 -- 172.21.15.53:0/2827083929 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93f40237a0 con 0x7f9404110220 2024-04-25T14:30:25.779 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:24.799+0000 7f940b800700 1 -- 172.21.15.53:0/2827083929 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94041a3810 con 0x7f9404110220 2024-04-25T14:30:25.779 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:24.801+0000 7f9402000700 1 -- 172.21.15.53:0/2827083929 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f93f401b440 con 0x7f9404110220 2024-04-25T14:30:25.780 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:24.801+0000 7f940b800700 1 -- 172.21.15.53:0/2827083929 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94040640a0 con 0x7f9404110220 2024-04-25T14:30:25.780 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:24.801+0000 7f9402000700 1 --2- 172.21.15.53:0/2827083929 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f93e8039650 0x7f93e803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:25.780 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:24.801+0000 7f9402000700 1 -- 172.21.15.53:0/2827083929 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f93f404e320 con 0x7f9404110220 2024-04-25T14:30:25.780 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:24.801+0000 7f93fbe00700 1 --2- 172.21.15.53:0/2827083929 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f93e8039650 0x7f93e803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:30:25.780 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:24.802+0000 7f93fbe00700 1 --2- 172.21.15.53:0/2827083929 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f93e8039650 0x7f93e803bb10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f93ec006fd0 tx=0x7f93ec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:30:25.780 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:24.807+0000 7f9402000700 1 -- 172.21.15.53:0/2827083929 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f93f401f030 con 0x7f9404110220 2024-04-25T14:30:25.780 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:25.156+0000 7f940b800700 1 -- 172.21.15.53:0/2827083929 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f94041a60a0 con 0x7f9404110220 2024-04-25T14:30:25.780 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:25.159+0000 7f9402000700 1 -- 172.21.15.53:0/2827083929 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f94041a60a0 con 0x7f9404110220 2024-04-25T14:30:25.780 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:25.162+0000 7f940b800700 1 -- 172.21.15.53:0/2827083929 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f93e8039650 msgr2=0x7f93e803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:25.780 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:25.162+0000 7f940b800700 1 --2- 172.21.15.53:0/2827083929 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f93e8039650 0x7f93e803bb10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f93ec006fd0 tx=0x7f93ec006e40 comp rx=0 tx=0).stop 2024-04-25T14:30:25.780 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:25.162+0000 7f940b800700 1 -- 172.21.15.53:0/2827083929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9404110220 msgr2=0x7f94041a2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:25.780 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:25.162+0000 7f940b800700 1 --2- 172.21.15.53:0/2827083929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9404110220 0x7f94041a2be0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f93f40119f0 tx=0x7f93f4011ad0 comp rx=0 tx=0).stop 2024-04-25T14:30:25.780 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:25.163+0000 7f940b800700 1 -- 172.21.15.53:0/2827083929 shutdown_connections 2024-04-25T14:30:25.780 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:25.163+0000 7f940b800700 1 --2- 172.21.15.53:0/2827083929 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f93e8039650 0x7f93e803bb10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:25.781 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:25.163+0000 7f940b800700 1 --2- 172.21.15.53:0/2827083929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9404110220 0x7f94041a2be0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:25.781 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:25.163+0000 7f940b800700 1 -- 172.21.15.53:0/2827083929 >> 172.21.15.53:0/2827083929 conn(0x7f940407de20 msgr2=0x7f940407f980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:30:25.781 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:25.163+0000 7f940b800700 1 -- 172.21.15.53:0/2827083929 shutdown_connections 2024-04-25T14:30:25.781 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr 2024-04-25T14:30:25.163+0000 7f940b800700 1 -- 172.21.15.53:0/2827083929 wait complete. 2024-04-25T14:30:25.781 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-25T14:30:25.781 INFO:teuthology.orchestra.run.smithi053.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-25T14:30:25.781 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:30:25.781 INFO:teuthology.orchestra.run.smithi053.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-25T14:30:25.781 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:30:25.781 INFO:teuthology.orchestra.run.smithi053.stdout:Or, if you are only running a single cluster on this host: 2024-04-25T14:30:25.781 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:30:25.781 INFO:teuthology.orchestra.run.smithi053.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-25T14:30:25.781 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:30:25.781 INFO:teuthology.orchestra.run.smithi053.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-25T14:30:25.782 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:30:25.782 INFO:teuthology.orchestra.run.smithi053.stdout: ceph telemetry on 2024-04-25T14:30:25.782 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:30:25.782 INFO:teuthology.orchestra.run.smithi053.stdout:For more information see: 2024-04-25T14:30:25.782 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:30:25.782 INFO:teuthology.orchestra.run.smithi053.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-25T14:30:25.782 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:30:25.782 INFO:teuthology.orchestra.run.smithi053.stdout:Bootstrap complete. 2024-04-25T14:30:25.868 INFO:tasks.cephadm:Fetching config... 2024-04-25T14:30:25.868 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-25T14:30:25.869 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-25T14:30:25.918 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-25T14:30:25.918 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-25T14:30:25.919 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-25T14:30:25.962 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-25T14:30:25.962 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-25T14:30:25.962 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-25T14:30:26.080 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-25T14:30:26.080 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-25T14:30:26.080 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-25T14:30:26.175 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-25T14:30:26.175 DEBUG:teuthology.orchestra.run.smithi053:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrCLUe/I7I53ZLzB43ipFc6lpRBE7ADQ18JN0n9Fsf1x8tiaJkmaNaFVBeK2GDjmE1b65JnV4cyo8PBNntqrUH0f8fi5xx0is/r5OuMN5o34dnu3wBI4VqTonNTnAFKp6eT9pIu3vSFr8uy4Yi9Sk0QuGgi6JFNgudd+yGNNt7KUOGYH+PmMiC8IjCr6wyX27zAZXc24ldDsuC4f4Kl80zL0c5LtyCeLTAtT8e0jcJb9MUJCOif7gjUYIMPNizVoa/fdWMpjCZqFBgxEFQWJtY7lDk5xoWU+0E0h4eZDD5xOBCb9iR9x2QLzWZE7C/AkhmZ8XntHapXlxTlrVx8UuBLz5WE8ptcPULceUd2z3XEB1xtv9S48Jm+rS2gVGnVTqjIuMrvrqvoqK430OpSTKmhveepVmJkPMbmZkO4fu3SViLP5/Jl9/u36MPLGng2shnTp4TqgvcibofLE09iTImto7YiBG1ub6ASmlTnV+nKSKSRlxwGh8L8vyApLL+tHE= ceph-fd30e6ce-030f-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T14:30:26.284 INFO:teuthology.orchestra.run.smithi053.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrCLUe/I7I53ZLzB43ipFc6lpRBE7ADQ18JN0n9Fsf1x8tiaJkmaNaFVBeK2GDjmE1b65JnV4cyo8PBNntqrUH0f8fi5xx0is/r5OuMN5o34dnu3wBI4VqTonNTnAFKp6eT9pIu3vSFr8uy4Yi9Sk0QuGgi6JFNgudd+yGNNt7KUOGYH+PmMiC8IjCr6wyX27zAZXc24ldDsuC4f4Kl80zL0c5LtyCeLTAtT8e0jcJb9MUJCOif7gjUYIMPNizVoa/fdWMpjCZqFBgxEFQWJtY7lDk5xoWU+0E0h4eZDD5xOBCb9iR9x2QLzWZE7C/AkhmZ8XntHapXlxTlrVx8UuBLz5WE8ptcPULceUd2z3XEB1xtv9S48Jm+rS2gVGnVTqjIuMrvrqvoqK430OpSTKmhveepVmJkPMbmZkO4fu3SViLP5/Jl9/u36MPLGng2shnTp4TqgvcibofLE09iTImto7YiBG1ub6ASmlTnV+nKSKSRlxwGh8L8vyApLL+tHE= ceph-fd30e6ce-030f-11ef-bc93-c7b262605968 2024-04-25T14:30:26.322 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:26 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/2827083929' entity='client.admin' 2024-04-25T14:30:26.325 DEBUG:teuthology.orchestra.run.smithi150:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrCLUe/I7I53ZLzB43ipFc6lpRBE7ADQ18JN0n9Fsf1x8tiaJkmaNaFVBeK2GDjmE1b65JnV4cyo8PBNntqrUH0f8fi5xx0is/r5OuMN5o34dnu3wBI4VqTonNTnAFKp6eT9pIu3vSFr8uy4Yi9Sk0QuGgi6JFNgudd+yGNNt7KUOGYH+PmMiC8IjCr6wyX27zAZXc24ldDsuC4f4Kl80zL0c5LtyCeLTAtT8e0jcJb9MUJCOif7gjUYIMPNizVoa/fdWMpjCZqFBgxEFQWJtY7lDk5xoWU+0E0h4eZDD5xOBCb9iR9x2QLzWZE7C/AkhmZ8XntHapXlxTlrVx8UuBLz5WE8ptcPULceUd2z3XEB1xtv9S48Jm+rS2gVGnVTqjIuMrvrqvoqK430OpSTKmhveepVmJkPMbmZkO4fu3SViLP5/Jl9/u36MPLGng2shnTp4TqgvcibofLE09iTImto7YiBG1ub6ASmlTnV+nKSKSRlxwGh8L8vyApLL+tHE= ceph-fd30e6ce-030f-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T14:30:26.366 INFO:teuthology.orchestra.run.smithi150.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrCLUe/I7I53ZLzB43ipFc6lpRBE7ADQ18JN0n9Fsf1x8tiaJkmaNaFVBeK2GDjmE1b65JnV4cyo8PBNntqrUH0f8fi5xx0is/r5OuMN5o34dnu3wBI4VqTonNTnAFKp6eT9pIu3vSFr8uy4Yi9Sk0QuGgi6JFNgudd+yGNNt7KUOGYH+PmMiC8IjCr6wyX27zAZXc24ldDsuC4f4Kl80zL0c5LtyCeLTAtT8e0jcJb9MUJCOif7gjUYIMPNizVoa/fdWMpjCZqFBgxEFQWJtY7lDk5xoWU+0E0h4eZDD5xOBCb9iR9x2QLzWZE7C/AkhmZ8XntHapXlxTlrVx8UuBLz5WE8ptcPULceUd2z3XEB1xtv9S48Jm+rS2gVGnVTqjIuMrvrqvoqK430OpSTKmhveepVmJkPMbmZkO4fu3SViLP5/Jl9/u36MPLGng2shnTp4TqgvcibofLE09iTImto7YiBG1ub6ASmlTnV+nKSKSRlxwGh8L8vyApLL+tHE= ceph-fd30e6ce-030f-11ef-bc93-c7b262605968 2024-04-25T14:30:26.378 DEBUG:teuthology.orchestra.run.smithi183:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrCLUe/I7I53ZLzB43ipFc6lpRBE7ADQ18JN0n9Fsf1x8tiaJkmaNaFVBeK2GDjmE1b65JnV4cyo8PBNntqrUH0f8fi5xx0is/r5OuMN5o34dnu3wBI4VqTonNTnAFKp6eT9pIu3vSFr8uy4Yi9Sk0QuGgi6JFNgudd+yGNNt7KUOGYH+PmMiC8IjCr6wyX27zAZXc24ldDsuC4f4Kl80zL0c5LtyCeLTAtT8e0jcJb9MUJCOif7gjUYIMPNizVoa/fdWMpjCZqFBgxEFQWJtY7lDk5xoWU+0E0h4eZDD5xOBCb9iR9x2QLzWZE7C/AkhmZ8XntHapXlxTlrVx8UuBLz5WE8ptcPULceUd2z3XEB1xtv9S48Jm+rS2gVGnVTqjIuMrvrqvoqK430OpSTKmhveepVmJkPMbmZkO4fu3SViLP5/Jl9/u36MPLGng2shnTp4TqgvcibofLE09iTImto7YiBG1ub6ASmlTnV+nKSKSRlxwGh8L8vyApLL+tHE= ceph-fd30e6ce-030f-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T14:30:26.418 INFO:teuthology.orchestra.run.smithi183.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrCLUe/I7I53ZLzB43ipFc6lpRBE7ADQ18JN0n9Fsf1x8tiaJkmaNaFVBeK2GDjmE1b65JnV4cyo8PBNntqrUH0f8fi5xx0is/r5OuMN5o34dnu3wBI4VqTonNTnAFKp6eT9pIu3vSFr8uy4Yi9Sk0QuGgi6JFNgudd+yGNNt7KUOGYH+PmMiC8IjCr6wyX27zAZXc24ldDsuC4f4Kl80zL0c5LtyCeLTAtT8e0jcJb9MUJCOif7gjUYIMPNizVoa/fdWMpjCZqFBgxEFQWJtY7lDk5xoWU+0E0h4eZDD5xOBCb9iR9x2QLzWZE7C/AkhmZ8XntHapXlxTlrVx8UuBLz5WE8ptcPULceUd2z3XEB1xtv9S48Jm+rS2gVGnVTqjIuMrvrqvoqK430OpSTKmhveepVmJkPMbmZkO4fu3SViLP5/Jl9/u36MPLGng2shnTp4TqgvcibofLE09iTImto7YiBG1ub6ASmlTnV+nKSKSRlxwGh8L8vyApLL+tHE= ceph-fd30e6ce-030f-11ef-bc93-c7b262605968 2024-04-25T14:30:26.430 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-25T14:30:26.858 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:30:29.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:29.760+0000 7ff379a00700 1 -- 172.21.15.53:0/1567775508 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff374100d80 msgr2=0x7ff3741011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:29.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:29.760+0000 7ff379a00700 1 --2- 172.21.15.53:0/1567775508 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff374100d80 0x7ff3741011a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff364009910 tx=0x7ff364009c20 comp rx=0 tx=0).stop 2024-04-25T14:30:29.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:29.761+0000 7ff379a00700 1 -- 172.21.15.53:0/1567775508 shutdown_connections 2024-04-25T14:30:29.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:29.761+0000 7ff379a00700 1 --2- 172.21.15.53:0/1567775508 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff374100d80 0x7ff3741011a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:29.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:29.761+0000 7ff379a00700 1 -- 172.21.15.53:0/1567775508 >> 172.21.15.53:0/1567775508 conn(0x7ff3740fc940 msgr2=0x7ff3740fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:30:29.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:29.761+0000 7ff379a00700 1 -- 172.21.15.53:0/1567775508 shutdown_connections 2024-04-25T14:30:29.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:29.761+0000 7ff379a00700 1 -- 172.21.15.53:0/1567775508 wait complete. 2024-04-25T14:30:29.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:29.762+0000 7ff379a00700 1 Processor -- start 2024-04-25T14:30:29.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:29.762+0000 7ff379a00700 1 -- start start 2024-04-25T14:30:29.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:29.762+0000 7ff379a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff374100d80 0x7ff374199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:29.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:29.762+0000 7ff379a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff37410a5d0 con 0x7ff374100d80 2024-04-25T14:30:29.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:29.763+0000 7ff372a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff374100d80 0x7ff374199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:30:29.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:29.763+0000 7ff372a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff374100d80 0x7ff374199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57212/0 (socket says 172.21.15.53:57212) 2024-04-25T14:30:29.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:29.763+0000 7ff372a00700 1 -- 172.21.15.53:0/4191776054 learned_addr learned my addr 172.21.15.53:0/4191776054 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:30:29.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:29.763+0000 7ff372a00700 1 -- 172.21.15.53:0/4191776054 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff37419a200 con 0x7ff374100d80 2024-04-25T14:30:29.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:29.764+0000 7ff372a00700 1 --2- 172.21.15.53:0/4191776054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff374100d80 0x7ff374199cc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff364005190 tx=0x7ff364004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:30:29.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:29.764+0000 7ff36b400700 1 -- 172.21.15.53:0/4191776054 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff364027070 con 0x7ff374100d80 2024-04-25T14:30:29.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:29.764+0000 7ff36b400700 1 -- 172.21.15.53:0/4191776054 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff364022470 con 0x7ff374100d80 2024-04-25T14:30:29.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:29.764+0000 7ff379a00700 1 -- 172.21.15.53:0/4191776054 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff37419a490 con 0x7ff374100d80 2024-04-25T14:30:29.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:29.764+0000 7ff36b400700 1 -- 172.21.15.53:0/4191776054 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff36400f5f0 con 0x7ff374100d80 2024-04-25T14:30:29.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:29.764+0000 7ff379a00700 1 -- 172.21.15.53:0/4191776054 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff37419d180 con 0x7ff374100d80 2024-04-25T14:30:29.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:29.765+0000 7ff36b400700 1 -- 172.21.15.53:0/4191776054 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7ff36402a070 con 0x7ff374100d80 2024-04-25T14:30:29.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:29.765+0000 7ff379a00700 1 -- 172.21.15.53:0/4191776054 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff37410ad50 con 0x7ff374100d80 2024-04-25T14:30:29.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:29.766+0000 7ff36b400700 1 --2- 172.21.15.53:0/4191776054 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff3580396b0 0x7ff35803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:29.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:29.766+0000 7ff36b400700 1 -- 172.21.15.53:0/4191776054 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff36401e030 con 0x7ff374100d80 2024-04-25T14:30:29.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:29.766+0000 7ff372000700 1 --2- 172.21.15.53:0/4191776054 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff3580396b0 0x7ff35803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:30:29.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:29.767+0000 7ff372000700 1 --2- 172.21.15.53:0/4191776054 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff3580396b0 0x7ff35803bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff360006fd0 tx=0x7ff360006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:30:29.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:29.772+0000 7ff36b400700 1 -- 172.21.15.53:0/4191776054 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff364013070 con 0x7ff374100d80 2024-04-25T14:30:30.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:30.001+0000 7ff379a00700 1 -- 172.21.15.53:0/4191776054 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7ff37402e090 con 0x7ff374100d80 2024-04-25T14:30:30.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:30.006+0000 7ff36b400700 1 -- 172.21.15.53:0/4191776054 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7ff364031020 con 0x7ff374100d80 2024-04-25T14:30:30.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:30.009+0000 7ff379a00700 1 -- 172.21.15.53:0/4191776054 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff3580396b0 msgr2=0x7ff35803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:30.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:30.009+0000 7ff379a00700 1 --2- 172.21.15.53:0/4191776054 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff3580396b0 0x7ff35803bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff360006fd0 tx=0x7ff360006e40 comp rx=0 tx=0).stop 2024-04-25T14:30:30.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:30.009+0000 7ff379a00700 1 -- 172.21.15.53:0/4191776054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff374100d80 msgr2=0x7ff374199cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:30.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:30.009+0000 7ff379a00700 1 --2- 172.21.15.53:0/4191776054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff374100d80 0x7ff374199cc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff364005190 tx=0x7ff364004f80 comp rx=0 tx=0).stop 2024-04-25T14:30:30.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:30.010+0000 7ff379a00700 1 -- 172.21.15.53:0/4191776054 shutdown_connections 2024-04-25T14:30:30.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:30.010+0000 7ff379a00700 1 --2- 172.21.15.53:0/4191776054 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff3580396b0 0x7ff35803bb70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:30.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:30.010+0000 7ff379a00700 1 --2- 172.21.15.53:0/4191776054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff374100d80 0x7ff374199cc0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:30.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:30.010+0000 7ff379a00700 1 -- 172.21.15.53:0/4191776054 >> 172.21.15.53:0/4191776054 conn(0x7ff3740fc940 msgr2=0x7ff3740fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:30:30.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:30.010+0000 7ff379a00700 1 -- 172.21.15.53:0/4191776054 shutdown_connections 2024-04-25T14:30:30.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:30.010+0000 7ff379a00700 1 -- 172.21.15.53:0/4191776054 wait complete. 2024-04-25T14:30:30.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:30 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:30:30.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:30 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:30:30.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:30 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi053", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:30:30.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:30 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:30:30.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:30 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:30:30.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:30 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:30:30.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:30 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/4191776054' entity='client.admin' 2024-04-25T14:30:30.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:30 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:30:30.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:30 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:30:30.801 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-25T14:30:30.801 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-25T14:30:31.212 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:30:33.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:33.838+0000 7f90efe00700 1 -- 172.21.15.53:0/1720354051 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90e81046d0 msgr2=0x7f90e8104af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:33.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:33.838+0000 7f90efe00700 1 --2- 172.21.15.53:0/1720354051 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90e81046d0 0x7f90e8104af0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f90dc009910 tx=0x7f90dc009c20 comp rx=0 tx=0).stop 2024-04-25T14:30:33.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:33.838+0000 7f90efe00700 1 -- 172.21.15.53:0/1720354051 shutdown_connections 2024-04-25T14:30:33.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:33.838+0000 7f90efe00700 1 --2- 172.21.15.53:0/1720354051 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90e81046d0 0x7f90e8104af0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:33.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:33.838+0000 7f90efe00700 1 -- 172.21.15.53:0/1720354051 >> 172.21.15.53:0/1720354051 conn(0x7f90e80ffc90 msgr2=0x7f90e81020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:30:33.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:33.838+0000 7f90efe00700 1 -- 172.21.15.53:0/1720354051 shutdown_connections 2024-04-25T14:30:33.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:33.839+0000 7f90efe00700 1 -- 172.21.15.53:0/1720354051 wait complete. 2024-04-25T14:30:33.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:33.840+0000 7f90efe00700 1 Processor -- start 2024-04-25T14:30:33.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:33.840+0000 7f90efe00700 1 -- start start 2024-04-25T14:30:33.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:33.840+0000 7f90efe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90e81046d0 0x7f90e8199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:33.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:33.840+0000 7f90efe00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f90e810a610 con 0x7f90e81046d0 2024-04-25T14:30:33.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:33.840+0000 7f90ed400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90e81046d0 0x7f90e8199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:30:33.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:33.840+0000 7f90ed400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90e81046d0 0x7f90e8199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45546/0 (socket says 172.21.15.53:45546) 2024-04-25T14:30:33.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:33.840+0000 7f90ed400700 1 -- 172.21.15.53:0/2614258589 learned_addr learned my addr 172.21.15.53:0/2614258589 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:30:33.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:33.841+0000 7f90ed400700 1 -- 172.21.15.53:0/2614258589 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e819a200 con 0x7f90e81046d0 2024-04-25T14:30:33.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:33.841+0000 7f90ed400700 1 --2- 172.21.15.53:0/2614258589 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90e81046d0 0x7f90e8199cc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f90dc005800 tx=0x7f90dc009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:30:33.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:33.841+0000 7f90e6000700 1 -- 172.21.15.53:0/2614258589 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90dc027070 con 0x7f90e81046d0 2024-04-25T14:30:33.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:33.842+0000 7f90e6000700 1 -- 172.21.15.53:0/2614258589 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90dc00be10 con 0x7f90e81046d0 2024-04-25T14:30:33.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:33.842+0000 7f90efe00700 1 -- 172.21.15.53:0/2614258589 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90e819a490 con 0x7f90e81046d0 2024-04-25T14:30:33.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:33.842+0000 7f90e6000700 1 -- 172.21.15.53:0/2614258589 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90dc00f460 con 0x7f90e81046d0 2024-04-25T14:30:33.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:33.842+0000 7f90efe00700 1 -- 172.21.15.53:0/2614258589 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90e819d180 con 0x7f90e81046d0 2024-04-25T14:30:33.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:33.843+0000 7f90efe00700 1 -- 172.21.15.53:0/2614258589 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90e810ad90 con 0x7f90e81046d0 2024-04-25T14:30:33.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:33.843+0000 7f90e6000700 1 -- 172.21.15.53:0/2614258589 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f90dc02a070 con 0x7f90e81046d0 2024-04-25T14:30:33.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:33.843+0000 7f90e6000700 1 --2- 172.21.15.53:0/2614258589 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f90d0039650 0x7f90d003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:33.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:33.843+0000 7f90e6000700 1 -- 172.21.15.53:0/2614258589 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f90dc01e030 con 0x7f90e81046d0 2024-04-25T14:30:33.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:33.844+0000 7f90eca00700 1 --2- 172.21.15.53:0/2614258589 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f90d0039650 0x7f90d003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:30:33.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:33.844+0000 7f90eca00700 1 --2- 172.21.15.53:0/2614258589 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f90d0039650 0x7f90d003bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f90d4006fd0 tx=0x7f90d4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:30:33.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:33.849+0000 7f90e6000700 1 -- 172.21.15.53:0/2614258589 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f90dc013070 con 0x7f90e81046d0 2024-04-25T14:30:34.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:34.084+0000 7f90efe00700 1 -- 172.21.15.53:0/2614258589 --> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f90e802e120 con 0x7f90d0039650 2024-04-25T14:30:34.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:34.089+0000 7f90e6000700 1 -- 172.21.15.53:0/2614258589 <== mgr.14152 v2:172.21.15.53:6800/4154933108 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f90e802e120 con 0x7f90d0039650 2024-04-25T14:30:34.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:34.093+0000 7f90efe00700 1 -- 172.21.15.53:0/2614258589 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f90d0039650 msgr2=0x7f90d003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:34.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:34.093+0000 7f90efe00700 1 --2- 172.21.15.53:0/2614258589 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f90d0039650 0x7f90d003bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f90d4006fd0 tx=0x7f90d4006e40 comp rx=0 tx=0).stop 2024-04-25T14:30:34.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:34.093+0000 7f90efe00700 1 -- 172.21.15.53:0/2614258589 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90e81046d0 msgr2=0x7f90e8199cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:34.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:34.093+0000 7f90efe00700 1 --2- 172.21.15.53:0/2614258589 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90e81046d0 0x7f90e8199cc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f90dc005800 tx=0x7f90dc009f60 comp rx=0 tx=0).stop 2024-04-25T14:30:34.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:34.093+0000 7f90efe00700 1 -- 172.21.15.53:0/2614258589 shutdown_connections 2024-04-25T14:30:34.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:34.093+0000 7f90efe00700 1 --2- 172.21.15.53:0/2614258589 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f90d0039650 0x7f90d003bb10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:34.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:34.093+0000 7f90efe00700 1 --2- 172.21.15.53:0/2614258589 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90e81046d0 0x7f90e8199cc0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:34.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:34.093+0000 7f90efe00700 1 -- 172.21.15.53:0/2614258589 >> 172.21.15.53:0/2614258589 conn(0x7f90e80ffc90 msgr2=0x7f90e8100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:30:34.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:34.093+0000 7f90efe00700 1 -- 172.21.15.53:0/2614258589 shutdown_connections 2024-04-25T14:30:34.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:34.093+0000 7f90efe00700 1 -- 172.21.15.53:0/2614258589 wait complete. 2024-04-25T14:30:34.933 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi150 2024-04-25T14:30:34.934 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T14:30:34.934 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/etc/ceph/ceph.conf 2024-04-25T14:30:34.952 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T14:30:34.952 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:30:35.011 INFO:tasks.cephadm:Adding host smithi150 to orchestrator... 2024-04-25T14:30:35.011 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph orch host add smithi150 2024-04-25T14:30:35.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:35 smithi053 ceph-mon[27773]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:30:35.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:35 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:30:35.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:35 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:30:35.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:35 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:30:35.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:35 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:30:35.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:35 smithi053 ceph-mon[27773]: Updating smithi053:/etc/ceph/ceph.conf 2024-04-25T14:30:35.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:35 smithi053 ceph-mon[27773]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:30:35.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:35 smithi053 ceph-mon[27773]: Updating smithi053:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:30:35.440 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:30:36.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:36 smithi053 ceph-mon[27773]: Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:30:37.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:37.553+0000 7fb98da00700 1 -- 172.21.15.53:0/328585308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb988076f70 msgr2=0x7fb988077350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:37.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:37.553+0000 7fb98da00700 1 --2- 172.21.15.53:0/328585308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb988076f70 0x7fb988077350 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb97800b0a0 tx=0x7fb97800b3b0 comp rx=0 tx=0).stop 2024-04-25T14:30:37.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:37.553+0000 7fb98da00700 1 -- 172.21.15.53:0/328585308 shutdown_connections 2024-04-25T14:30:37.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:37.553+0000 7fb98da00700 1 --2- 172.21.15.53:0/328585308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb988076f70 0x7fb988077350 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:37.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:37.553+0000 7fb98da00700 1 -- 172.21.15.53:0/328585308 >> 172.21.15.53:0/328585308 conn(0x7fb988100160 msgr2=0x7fb988102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:30:37.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:37.553+0000 7fb98da00700 1 -- 172.21.15.53:0/328585308 shutdown_connections 2024-04-25T14:30:37.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:37.553+0000 7fb98da00700 1 -- 172.21.15.53:0/328585308 wait complete. 2024-04-25T14:30:37.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:37.555+0000 7fb98da00700 1 Processor -- start 2024-04-25T14:30:37.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:37.555+0000 7fb98da00700 1 -- start start 2024-04-25T14:30:37.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:37.556+0000 7fb98da00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb988076f70 0x7fb98819e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:37.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:37.556+0000 7fb98da00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb988112f90 con 0x7fb988076f70 2024-04-25T14:30:37.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:37.556+0000 7fb986a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb988076f70 0x7fb98819e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:30:37.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:37.556+0000 7fb986a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb988076f70 0x7fb98819e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45566/0 (socket says 172.21.15.53:45566) 2024-04-25T14:30:37.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:37.556+0000 7fb986a00700 1 -- 172.21.15.53:0/1008703948 learned_addr learned my addr 172.21.15.53:0/1008703948 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:30:37.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:37.556+0000 7fb986a00700 1 -- 172.21.15.53:0/1008703948 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb98819e950 con 0x7fb988076f70 2024-04-25T14:30:37.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:37.557+0000 7fb986a00700 1 --2- 172.21.15.53:0/1008703948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb988076f70 0x7fb98819e410 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb97800d010 tx=0x7fb978003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:30:37.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:37.557+0000 7fb97fe00700 1 -- 172.21.15.53:0/1008703948 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb978021020 con 0x7fb988076f70 2024-04-25T14:30:37.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:37.557+0000 7fb97fe00700 1 -- 172.21.15.53:0/1008703948 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9780096a0 con 0x7fb988076f70 2024-04-25T14:30:37.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:37.557+0000 7fb97fe00700 1 -- 172.21.15.53:0/1008703948 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb978017040 con 0x7fb988076f70 2024-04-25T14:30:37.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:37.557+0000 7fb98da00700 1 -- 172.21.15.53:0/1008703948 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb98819ebe0 con 0x7fb988076f70 2024-04-25T14:30:37.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:37.557+0000 7fb98da00700 1 -- 172.21.15.53:0/1008703948 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9881a2ce0 con 0x7fb988076f70 2024-04-25T14:30:37.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:37.558+0000 7fb97fe00700 1 -- 172.21.15.53:0/1008703948 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fb978024020 con 0x7fb988076f70 2024-04-25T14:30:37.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:37.559+0000 7fb98da00700 1 -- 172.21.15.53:0/1008703948 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9880786b0 con 0x7fb988076f70 2024-04-25T14:30:37.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:37.559+0000 7fb97fe00700 1 --2- 172.21.15.53:0/1008703948 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fb96c039700 0x7fb96c03bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:37.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:37.559+0000 7fb97fe00700 1 -- 172.21.15.53:0/1008703948 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb97800f050 con 0x7fb988076f70 2024-04-25T14:30:37.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:37.559+0000 7fb97e000700 1 --2- 172.21.15.53:0/1008703948 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fb96c039700 0x7fb96c03bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:30:37.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:37.560+0000 7fb97e000700 1 --2- 172.21.15.53:0/1008703948 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fb96c039700 0x7fb96c03bbc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb970006fd0 tx=0x7fb970006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:30:37.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:37.565+0000 7fb97fe00700 1 -- 172.21.15.53:0/1008703948 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb97802b020 con 0x7fb988076f70 2024-04-25T14:30:37.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:37.805+0000 7fb98da00700 1 -- 172.21.15.53:0/1008703948 --> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi150", "target": ["mon-mgr", ""]}) -- 0x7fb98819f770 con 0x7fb96c039700 2024-04-25T14:30:38.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:37 smithi053 ceph-mon[27773]: Updating smithi053:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T14:30:38.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:37 smithi053 ceph-mon[27773]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:30:38.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:37 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:30:38.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:37 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:30:38.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:37 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:30:39.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:38 smithi053 ceph-mon[27773]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi150", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:30:40.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:39 smithi053 ceph-mon[27773]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:30:40.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:39 smithi053 ceph-mon[27773]: Deploying cephadm binary to smithi150 2024-04-25T14:30:42.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:41 smithi053 ceph-mon[27773]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:30:42.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:42.678+0000 7fb97fe00700 1 -- 172.21.15.53:0/1008703948 <== mgr.14152 v2:172.21.15.53:6800/4154933108 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fb98819f770 con 0x7fb96c039700 2024-04-25T14:30:42.682 INFO:teuthology.orchestra.run.smithi053.stdout:Added host 'smithi150' with addr '172.21.15.150' 2024-04-25T14:30:42.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:42.682+0000 7fb98da00700 1 -- 172.21.15.53:0/1008703948 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fb96c039700 msgr2=0x7fb96c03bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:42.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:42.682+0000 7fb98da00700 1 --2- 172.21.15.53:0/1008703948 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fb96c039700 0x7fb96c03bbc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb970006fd0 tx=0x7fb970006e40 comp rx=0 tx=0).stop 2024-04-25T14:30:42.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:42.682+0000 7fb98da00700 1 -- 172.21.15.53:0/1008703948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb988076f70 msgr2=0x7fb98819e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:42.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:42.682+0000 7fb98da00700 1 --2- 172.21.15.53:0/1008703948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb988076f70 0x7fb98819e410 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb97800d010 tx=0x7fb978003ce0 comp rx=0 tx=0).stop 2024-04-25T14:30:42.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:42.682+0000 7fb98da00700 1 -- 172.21.15.53:0/1008703948 shutdown_connections 2024-04-25T14:30:42.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:42.682+0000 7fb98da00700 1 --2- 172.21.15.53:0/1008703948 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fb96c039700 0x7fb96c03bbc0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:42.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:42.682+0000 7fb98da00700 1 --2- 172.21.15.53:0/1008703948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb988076f70 0x7fb98819e410 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:42.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:42.682+0000 7fb98da00700 1 -- 172.21.15.53:0/1008703948 >> 172.21.15.53:0/1008703948 conn(0x7fb988100160 msgr2=0x7fb9881019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:30:42.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:42.683+0000 7fb98da00700 1 -- 172.21.15.53:0/1008703948 shutdown_connections 2024-04-25T14:30:42.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:42.683+0000 7fb98da00700 1 -- 172.21.15.53:0/1008703948 wait complete. 2024-04-25T14:30:43.519 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T14:30:43.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:43 smithi053 ceph-mon[27773]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:30:43.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:43 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:30:43.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:43 smithi053 ceph-mon[27773]: Added host smithi150 2024-04-25T14:30:43.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:43 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:30:43.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:43 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:30:43.976 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:30:44.797 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:44 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:30:45.981 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:45 smithi053 ceph-mon[27773]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:30:45.981 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:45 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:30:46.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:46.605+0000 7ff7f0200700 1 -- 172.21.15.53:0/1748663191 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7e810d960 msgr2=0x7ff7e810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:46.607 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:46.605+0000 7ff7f0200700 1 --2- 172.21.15.53:0/1748663191 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7e810d960 0x7ff7e810dd40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc009910 tx=0x7ff7dc009c20 comp rx=0 tx=0).stop 2024-04-25T14:30:46.607 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:46.605+0000 7ff7f0200700 1 -- 172.21.15.53:0/1748663191 shutdown_connections 2024-04-25T14:30:46.607 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:46.605+0000 7ff7f0200700 1 --2- 172.21.15.53:0/1748663191 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7e810d960 0x7ff7e810dd40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:46.607 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:46.605+0000 7ff7f0200700 1 -- 172.21.15.53:0/1748663191 >> 172.21.15.53:0/1748663191 conn(0x7ff7e80781e0 msgr2=0x7ff7e807a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:30:46.607 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:46.605+0000 7ff7f0200700 1 -- 172.21.15.53:0/1748663191 shutdown_connections 2024-04-25T14:30:46.607 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:46.606+0000 7ff7f0200700 1 -- 172.21.15.53:0/1748663191 wait complete. 2024-04-25T14:30:46.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:46.607+0000 7ff7f0200700 1 Processor -- start 2024-04-25T14:30:46.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:46.607+0000 7ff7f0200700 1 -- start start 2024-04-25T14:30:46.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:46.607+0000 7ff7f0200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7e810d960 0x7ff7e819e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:46.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:46.607+0000 7ff7f0200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e8112f60 con 0x7ff7e810d960 2024-04-25T14:30:46.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:46.607+0000 7ff7ed800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7e810d960 0x7ff7e819e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:30:46.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:46.608+0000 7ff7ed800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7e810d960 0x7ff7e819e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44050/0 (socket says 172.21.15.53:44050) 2024-04-25T14:30:46.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:46.608+0000 7ff7ed800700 1 -- 172.21.15.53:0/622977944 learned_addr learned my addr 172.21.15.53:0/622977944 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:30:46.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:46.608+0000 7ff7ed800700 1 -- 172.21.15.53:0/622977944 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7e819e9b0 con 0x7ff7e810d960 2024-04-25T14:30:46.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:46.608+0000 7ff7ed800700 1 --2- 172.21.15.53:0/622977944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7e810d960 0x7ff7e819e470 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc006010 tx=0x7ff7dc005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:30:46.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:46.609+0000 7ff7e6000700 1 -- 172.21.15.53:0/622977944 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7dc026070 con 0x7ff7e810d960 2024-04-25T14:30:46.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:46.609+0000 7ff7e6000700 1 -- 172.21.15.53:0/622977944 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7dc00bba0 con 0x7ff7e810d960 2024-04-25T14:30:46.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:46.609+0000 7ff7f0200700 1 -- 172.21.15.53:0/622977944 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7e819eca0 con 0x7ff7e810d960 2024-04-25T14:30:46.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:46.609+0000 7ff7e6000700 1 -- 172.21.15.53:0/622977944 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7dc003930 con 0x7ff7e810d960 2024-04-25T14:30:46.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:46.609+0000 7ff7f0200700 1 -- 172.21.15.53:0/622977944 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7e81a2d40 con 0x7ff7e810d960 2024-04-25T14:30:46.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:46.610+0000 7ff7f0200700 1 -- 172.21.15.53:0/622977944 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7e810a180 con 0x7ff7e810d960 2024-04-25T14:30:46.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:46.610+0000 7ff7e6000700 1 -- 172.21.15.53:0/622977944 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7ff7dc029070 con 0x7ff7e810d960 2024-04-25T14:30:46.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:46.611+0000 7ff7e6000700 1 --2- 172.21.15.53:0/622977944 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff7d00396b0 0x7ff7d003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:46.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:46.611+0000 7ff7e6000700 1 -- 172.21.15.53:0/622977944 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff7dc01e070 con 0x7ff7e810d960 2024-04-25T14:30:46.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:46.611+0000 7ff7ece00700 1 --2- 172.21.15.53:0/622977944 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff7d00396b0 0x7ff7d003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:30:46.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:46.612+0000 7ff7ece00700 1 --2- 172.21.15.53:0/622977944 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff7d00396b0 0x7ff7d003bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4006fd0 tx=0x7ff7d4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:30:46.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:46.617+0000 7ff7e6000700 1 -- 172.21.15.53:0/622977944 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff7dc013070 con 0x7ff7e810d960 2024-04-25T14:30:46.852 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:46.850+0000 7ff7f0200700 1 -- 172.21.15.53:0/622977944 --> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff7e819f7d0 con 0x7ff7d00396b0 2024-04-25T14:30:46.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:46.854+0000 7ff7e6000700 1 -- 172.21.15.53:0/622977944 <== mgr.14152 v2:172.21.15.53:6800/4154933108 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7ff7e819f7d0 con 0x7ff7d00396b0 2024-04-25T14:30:46.855 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:30:46.855 INFO:teuthology.orchestra.run.smithi053.stdout:[{"addr": "172.21.15.53", "hostname": "smithi053", "labels": [], "status": ""}, {"addr": "172.21.15.150", "hostname": "smithi150", "labels": [], "status": ""}] 2024-04-25T14:30:46.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:46.856+0000 7ff7f0200700 1 -- 172.21.15.53:0/622977944 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff7d00396b0 msgr2=0x7ff7d003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:46.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:46.856+0000 7ff7f0200700 1 --2- 172.21.15.53:0/622977944 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff7d00396b0 0x7ff7d003bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4006fd0 tx=0x7ff7d4006e40 comp rx=0 tx=0).stop 2024-04-25T14:30:46.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:46.856+0000 7ff7f0200700 1 -- 172.21.15.53:0/622977944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7e810d960 msgr2=0x7ff7e819e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:46.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:46.856+0000 7ff7f0200700 1 --2- 172.21.15.53:0/622977944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7e810d960 0x7ff7e819e470 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc006010 tx=0x7ff7dc005780 comp rx=0 tx=0).stop 2024-04-25T14:30:46.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:46.857+0000 7ff7f0200700 1 -- 172.21.15.53:0/622977944 shutdown_connections 2024-04-25T14:30:46.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:46.857+0000 7ff7f0200700 1 --2- 172.21.15.53:0/622977944 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff7d00396b0 0x7ff7d003bb70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:46.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:46.857+0000 7ff7f0200700 1 --2- 172.21.15.53:0/622977944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7e810d960 0x7ff7e819e470 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:46.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:46.857+0000 7ff7f0200700 1 -- 172.21.15.53:0/622977944 >> 172.21.15.53:0/622977944 conn(0x7ff7e80781e0 msgr2=0x7ff7e8078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:30:46.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:46.857+0000 7ff7f0200700 1 -- 172.21.15.53:0/622977944 shutdown_connections 2024-04-25T14:30:46.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:46.857+0000 7ff7f0200700 1 -- 172.21.15.53:0/622977944 wait complete. 2024-04-25T14:30:47.964 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi183 2024-04-25T14:30:47.964 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-25T14:30:47.964 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/etc/ceph/ceph.conf 2024-04-25T14:30:47.983 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-25T14:30:47.983 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:30:48.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:47 smithi053 ceph-mon[27773]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:30:48.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:47 smithi053 ceph-mon[27773]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:30:48.040 INFO:tasks.cephadm:Adding host smithi183 to orchestrator... 2024-04-25T14:30:48.040 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph orch host add smithi183 2024-04-25T14:30:48.498 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:30:50.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:49 smithi053 ceph-mon[27773]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:30:50.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:49 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:30:50.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:49 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:30:50.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:49 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:30:50.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:49 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:30:50.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:49 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi150", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:30:50.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:49 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:30:50.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:49 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:30:50.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:49 smithi053 ceph-mon[27773]: Updating smithi150:/etc/ceph/ceph.conf 2024-04-25T14:30:50.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:49 smithi053 ceph-mon[27773]: Updating smithi150:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:30:50.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:50.864+0000 7f460ca00700 1 -- 172.21.15.53:0/2493371637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4608074e70 msgr2=0x7f4608075250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:50.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:50.864+0000 7f460ca00700 1 --2- 172.21.15.53:0/2493371637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4608074e70 0x7f4608075250 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f45f8009910 tx=0x7f45f8009c20 comp rx=0 tx=0).stop 2024-04-25T14:30:50.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:50.865+0000 7f460ca00700 1 -- 172.21.15.53:0/2493371637 shutdown_connections 2024-04-25T14:30:50.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:50.865+0000 7f460ca00700 1 --2- 172.21.15.53:0/2493371637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4608074e70 0x7f4608075250 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:50.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:50.865+0000 7f460ca00700 1 -- 172.21.15.53:0/2493371637 >> 172.21.15.53:0/2493371637 conn(0x7f46080fdf80 msgr2=0x7f46081003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:30:50.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:50.865+0000 7f460ca00700 1 -- 172.21.15.53:0/2493371637 shutdown_connections 2024-04-25T14:30:50.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:50.865+0000 7f460ca00700 1 -- 172.21.15.53:0/2493371637 wait complete. 2024-04-25T14:30:50.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:50.866+0000 7f460ca00700 1 Processor -- start 2024-04-25T14:30:50.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:50.866+0000 7f460ca00700 1 -- start start 2024-04-25T14:30:50.868 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:50.867+0000 7f460ca00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4608074e70 0x7f460819a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:50.868 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:50.867+0000 7f460ca00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f460810ebb0 con 0x7f4608074e70 2024-04-25T14:30:50.868 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:50.867+0000 7f4605e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4608074e70 0x7f460819a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:30:50.868 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:50.867+0000 7f4605e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4608074e70 0x7f460819a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44076/0 (socket says 172.21.15.53:44076) 2024-04-25T14:30:50.868 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:50.867+0000 7f4605e00700 1 -- 172.21.15.53:0/3065762022 learned_addr learned my addr 172.21.15.53:0/3065762022 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:30:50.868 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:50.868+0000 7f4605e00700 1 -- 172.21.15.53:0/3065762022 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f460819a640 con 0x7f4608074e70 2024-04-25T14:30:50.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:50.868+0000 7f4605e00700 1 --2- 172.21.15.53:0/3065762022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4608074e70 0x7f460819a100 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f45f8006010 tx=0x7f45f8005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:30:50.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:50.868+0000 7f45ff400700 1 -- 172.21.15.53:0/3065762022 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45f8026070 con 0x7f4608074e70 2024-04-25T14:30:50.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:50.868+0000 7f45ff400700 1 -- 172.21.15.53:0/3065762022 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45f800bc00 con 0x7f4608074e70 2024-04-25T14:30:50.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:50.868+0000 7f45ff400700 1 -- 172.21.15.53:0/3065762022 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45f8003910 con 0x7f4608074e70 2024-04-25T14:30:50.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:50.868+0000 7f460ca00700 1 -- 172.21.15.53:0/3065762022 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f460819a8d0 con 0x7f4608074e70 2024-04-25T14:30:50.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:50.869+0000 7f460ca00700 1 -- 172.21.15.53:0/3065762022 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f460819ad30 con 0x7f4608074e70 2024-04-25T14:30:50.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:50.870+0000 7f45ff400700 1 -- 172.21.15.53:0/3065762022 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f45f8029070 con 0x7f4608074e70 2024-04-25T14:30:50.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:50.870+0000 7f460ca00700 1 -- 172.21.15.53:0/3065762022 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4608076520 con 0x7f4608074e70 2024-04-25T14:30:50.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:50.870+0000 7f45ff400700 1 --2- 172.21.15.53:0/3065762022 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f45ec0396b0 0x7f45ec03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:50.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:50.870+0000 7f45ff400700 1 -- 172.21.15.53:0/3065762022 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f45f801e070 con 0x7f4608074e70 2024-04-25T14:30:50.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:50.871+0000 7f45fd400700 1 --2- 172.21.15.53:0/3065762022 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f45ec0396b0 0x7f45ec03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:30:50.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:50.871+0000 7f45fd400700 1 --2- 172.21.15.53:0/3065762022 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f45ec0396b0 0x7f45ec03bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f45f0006fd0 tx=0x7f45f0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:30:50.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:50.876+0000 7f45ff400700 1 -- 172.21.15.53:0/3065762022 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f45f8013070 con 0x7f4608074e70 2024-04-25T14:30:51.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:50 smithi053 ceph-mon[27773]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:30:51.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:50 smithi053 ceph-mon[27773]: Updating smithi150:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T14:30:51.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:50 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:30:51.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:50 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:30:51.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:50 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:30:51.116 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:51.115+0000 7f460ca00700 1 -- 172.21.15.53:0/3065762022 --> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi183", "target": ["mon-mgr", ""]}) -- 0x7f460819b4d0 con 0x7f45ec0396b0 2024-04-25T14:30:52.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:51 smithi053 ceph-mon[27773]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:30:53.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:52 smithi053 ceph-mon[27773]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi183", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:30:54.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:53 smithi053 ceph-mon[27773]: Deploying cephadm binary to smithi183 2024-04-25T14:30:54.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:53 smithi053 ceph-mon[27773]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:30:56.188 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:55 smithi053 ceph-mon[27773]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:30:56.188 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:56.187+0000 7f45ff400700 1 -- 172.21.15.53:0/3065762022 <== mgr.14152 v2:172.21.15.53:6800/4154933108 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f460819b4d0 con 0x7f45ec0396b0 2024-04-25T14:30:56.189 INFO:teuthology.orchestra.run.smithi053.stdout:Added host 'smithi183' with addr '172.21.15.183' 2024-04-25T14:30:56.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:56.192+0000 7f460ca00700 1 -- 172.21.15.53:0/3065762022 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f45ec0396b0 msgr2=0x7f45ec03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:56.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:56.192+0000 7f460ca00700 1 --2- 172.21.15.53:0/3065762022 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f45ec0396b0 0x7f45ec03bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f45f0006fd0 tx=0x7f45f0006e40 comp rx=0 tx=0).stop 2024-04-25T14:30:56.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:56.192+0000 7f460ca00700 1 -- 172.21.15.53:0/3065762022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4608074e70 msgr2=0x7f460819a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:56.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:56.192+0000 7f460ca00700 1 --2- 172.21.15.53:0/3065762022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4608074e70 0x7f460819a100 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f45f8006010 tx=0x7f45f8005780 comp rx=0 tx=0).stop 2024-04-25T14:30:56.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:56.192+0000 7f460ca00700 1 -- 172.21.15.53:0/3065762022 shutdown_connections 2024-04-25T14:30:56.193 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:56.192+0000 7f460ca00700 1 --2- 172.21.15.53:0/3065762022 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f45ec0396b0 0x7f45ec03bb70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:56.193 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:56.192+0000 7f460ca00700 1 --2- 172.21.15.53:0/3065762022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4608074e70 0x7f460819a100 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:56.193 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:56.192+0000 7f460ca00700 1 -- 172.21.15.53:0/3065762022 >> 172.21.15.53:0/3065762022 conn(0x7f46080fdf80 msgr2=0x7f46081066f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:30:56.193 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:56.193+0000 7f460ca00700 1 -- 172.21.15.53:0/3065762022 shutdown_connections 2024-04-25T14:30:56.193 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:56.193+0000 7f460ca00700 1 -- 172.21.15.53:0/3065762022 wait complete. 2024-04-25T14:30:57.089 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T14:30:57.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:57 smithi053 ceph-mon[27773]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:30:57.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:57 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:30:57.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:57 smithi053 ceph-mon[27773]: Added host smithi183 2024-04-25T14:30:57.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:57 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:30:57.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:57 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:30:57.534 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:30:58.571 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:58 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:30:59.539 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:59 smithi053 ceph-mon[27773]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:30:59.539 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:30:59 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:30:59.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:59.819+0000 7f56e0600700 1 -- 172.21.15.53:0/2109616547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56d8074e10 msgr2=0x7f56d80751f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:30:59.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:59.819+0000 7f56e0600700 1 --2- 172.21.15.53:0/2109616547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56d8074e10 0x7f56d80751f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f56cc009910 tx=0x7f56cc009c20 comp rx=0 tx=0).stop 2024-04-25T14:30:59.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:59.819+0000 7f56e0600700 1 -- 172.21.15.53:0/2109616547 shutdown_connections 2024-04-25T14:30:59.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:59.819+0000 7f56e0600700 1 --2- 172.21.15.53:0/2109616547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56d8074e10 0x7f56d80751f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:30:59.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:59.819+0000 7f56e0600700 1 -- 172.21.15.53:0/2109616547 >> 172.21.15.53:0/2109616547 conn(0x7f56d80fdf80 msgr2=0x7f56d81003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:30:59.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:59.819+0000 7f56e0600700 1 -- 172.21.15.53:0/2109616547 shutdown_connections 2024-04-25T14:30:59.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:59.820+0000 7f56e0600700 1 -- 172.21.15.53:0/2109616547 wait complete. 2024-04-25T14:30:59.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:59.821+0000 7f56e0600700 1 Processor -- start 2024-04-25T14:30:59.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:59.821+0000 7f56e0600700 1 -- start start 2024-04-25T14:30:59.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:59.821+0000 7f56e0600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56d8074e10 0x7f56d819a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:59.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:59.821+0000 7f56e0600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f56d810eba0 con 0x7f56d8074e10 2024-04-25T14:30:59.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:59.822+0000 7f56ddc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56d8074e10 0x7f56d819a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:30:59.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:59.822+0000 7f56ddc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56d8074e10 0x7f56d819a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53022/0 (socket says 172.21.15.53:53022) 2024-04-25T14:30:59.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:59.822+0000 7f56ddc00700 1 -- 172.21.15.53:0/2941684581 learned_addr learned my addr 172.21.15.53:0/2941684581 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:30:59.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:59.822+0000 7f56ddc00700 1 -- 172.21.15.53:0/2941684581 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d819a650 con 0x7f56d8074e10 2024-04-25T14:30:59.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:59.823+0000 7f56ddc00700 1 --2- 172.21.15.53:0/2941684581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56d8074e10 0x7f56d819a110 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f56cc006010 tx=0x7f56cc005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:30:59.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:59.823+0000 7f56d6000700 1 -- 172.21.15.53:0/2941684581 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56cc026070 con 0x7f56d8074e10 2024-04-25T14:30:59.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:59.823+0000 7f56d6000700 1 -- 172.21.15.53:0/2941684581 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56cc00bc00 con 0x7f56d8074e10 2024-04-25T14:30:59.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:59.823+0000 7f56e0600700 1 -- 172.21.15.53:0/2941684581 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56d819a8e0 con 0x7f56d8074e10 2024-04-25T14:30:59.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:59.823+0000 7f56d6000700 1 -- 172.21.15.53:0/2941684581 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56cc003910 con 0x7f56d8074e10 2024-04-25T14:30:59.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:59.823+0000 7f56e0600700 1 -- 172.21.15.53:0/2941684581 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56d819ad40 con 0x7f56d8074e10 2024-04-25T14:30:59.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:59.824+0000 7f56e0600700 1 -- 172.21.15.53:0/2941684581 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56d80764c0 con 0x7f56d8074e10 2024-04-25T14:30:59.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:59.824+0000 7f56d6000700 1 -- 172.21.15.53:0/2941684581 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f56cc029070 con 0x7f56d8074e10 2024-04-25T14:30:59.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:59.825+0000 7f56d6000700 1 --2- 172.21.15.53:0/2941684581 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f56c0039650 0x7f56c003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:30:59.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:59.825+0000 7f56d6000700 1 -- 172.21.15.53:0/2941684581 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f56cc01e070 con 0x7f56d8074e10 2024-04-25T14:30:59.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:59.825+0000 7f56dd200700 1 --2- 172.21.15.53:0/2941684581 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f56c0039650 0x7f56c003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:30:59.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:59.826+0000 7f56dd200700 1 --2- 172.21.15.53:0/2941684581 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f56c0039650 0x7f56c003bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f56c4006fd0 tx=0x7f56c4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:30:59.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:30:59.831+0000 7f56d6000700 1 -- 172.21.15.53:0/2941684581 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f56cc013070 con 0x7f56d8074e10 2024-04-25T14:31:00.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:00.067+0000 7f56e0600700 1 -- 172.21.15.53:0/2941684581 --> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f56d819b470 con 0x7f56c0039650 2024-04-25T14:31:00.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:00.068+0000 7f56d6000700 1 -- 172.21.15.53:0/2941684581 <== mgr.14152 v2:172.21.15.53:6800/4154933108 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f56d819b470 con 0x7f56c0039650 2024-04-25T14:31:00.068 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:31:00.068 INFO:teuthology.orchestra.run.smithi053.stdout:[{"addr": "172.21.15.53", "hostname": "smithi053", "labels": [], "status": ""}, {"addr": "172.21.15.150", "hostname": "smithi150", "labels": [], "status": ""}, {"addr": "172.21.15.183", "hostname": "smithi183", "labels": [], "status": ""}] 2024-04-25T14:31:00.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:00.071+0000 7f56e0600700 1 -- 172.21.15.53:0/2941684581 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f56c0039650 msgr2=0x7f56c003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:31:00.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:00.071+0000 7f56e0600700 1 --2- 172.21.15.53:0/2941684581 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f56c0039650 0x7f56c003bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f56c4006fd0 tx=0x7f56c4006e40 comp rx=0 tx=0).stop 2024-04-25T14:31:00.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:00.072+0000 7f56e0600700 1 -- 172.21.15.53:0/2941684581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56d8074e10 msgr2=0x7f56d819a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:31:00.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:00.072+0000 7f56e0600700 1 --2- 172.21.15.53:0/2941684581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56d8074e10 0x7f56d819a110 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f56cc006010 tx=0x7f56cc005780 comp rx=0 tx=0).stop 2024-04-25T14:31:00.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:00.072+0000 7f56e0600700 1 -- 172.21.15.53:0/2941684581 shutdown_connections 2024-04-25T14:31:00.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:00.072+0000 7f56e0600700 1 --2- 172.21.15.53:0/2941684581 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f56c0039650 0x7f56c003bb10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:31:00.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:00.072+0000 7f56e0600700 1 --2- 172.21.15.53:0/2941684581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56d8074e10 0x7f56d819a110 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:31:00.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:00.072+0000 7f56e0600700 1 -- 172.21.15.53:0/2941684581 >> 172.21.15.53:0/2941684581 conn(0x7f56d80fdf80 msgr2=0x7f56d8108760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:31:00.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:00.072+0000 7f56e0600700 1 -- 172.21.15.53:0/2941684581 shutdown_connections 2024-04-25T14:31:00.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:00.072+0000 7f56e0600700 1 -- 172.21.15.53:0/2941684581 wait complete. 2024-04-25T14:31:00.786 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-25T14:31:00.786 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-25T14:31:01.230 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:31:01.700 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:01 smithi053 ceph-mon[27773]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:31:01.700 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:01 smithi053 ceph-mon[27773]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:01.700 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:01 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:01.700 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:01 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:02.603 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:02 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:02.603 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:02 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:02.603 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:02 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi183", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:31:02.603 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:02 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:02.603 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:02 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:31:02.603 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:02 smithi053 ceph-mon[27773]: Updating smithi183:/etc/ceph/ceph.conf 2024-04-25T14:31:02.603 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:02 smithi053 ceph-mon[27773]: Updating smithi183:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:31:03.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:03.391+0000 7f20c2200700 1 -- 172.21.15.53:0/2800275121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20bc10a570 msgr2=0x7f20bc10a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:31:03.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:03.391+0000 7f20c2200700 1 --2- 172.21.15.53:0/2800275121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20bc10a570 0x7f20bc10a950 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f20b0009910 tx=0x7f20b0009c20 comp rx=0 tx=0).stop 2024-04-25T14:31:03.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:03.391+0000 7f20c2200700 1 -- 172.21.15.53:0/2800275121 shutdown_connections 2024-04-25T14:31:03.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:03.391+0000 7f20c2200700 1 --2- 172.21.15.53:0/2800275121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20bc10a570 0x7f20bc10a950 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:31:03.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:03.391+0000 7f20c2200700 1 -- 172.21.15.53:0/2800275121 >> 172.21.15.53:0/2800275121 conn(0x7f20bc0fff70 msgr2=0x7f20bc102390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:31:03.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:03.391+0000 7f20c2200700 1 -- 172.21.15.53:0/2800275121 shutdown_connections 2024-04-25T14:31:03.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:03.391+0000 7f20c2200700 1 -- 172.21.15.53:0/2800275121 wait complete. 2024-04-25T14:31:03.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:03.392+0000 7f20c2200700 1 Processor -- start 2024-04-25T14:31:03.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:03.393+0000 7f20c2200700 1 -- start start 2024-04-25T14:31:03.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:03.393+0000 7f20c2200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20bc10a570 0x7f20bc199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:31:03.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:03.393+0000 7f20c2200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f20bc10e9e0 con 0x7f20bc10a570 2024-04-25T14:31:03.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:03.393+0000 7f20c0e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20bc10a570 0x7f20bc199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:31:03.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:03.393+0000 7f20c0e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20bc10a570 0x7f20bc199ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39082/0 (socket says 172.21.15.53:39082) 2024-04-25T14:31:03.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:03.393+0000 7f20c0e00700 1 -- 172.21.15.53:0/3707162616 learned_addr learned my addr 172.21.15.53:0/3707162616 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:31:03.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:03.394+0000 7f20c0e00700 1 -- 172.21.15.53:0/3707162616 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20bc19a410 con 0x7f20bc10a570 2024-04-25T14:31:03.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:03.394+0000 7f20c0e00700 1 --2- 172.21.15.53:0/3707162616 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20bc10a570 0x7f20bc199ed0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f20b0006010 tx=0x7f20b0005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:31:03.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:03.394+0000 7f20b9600700 1 -- 172.21.15.53:0/3707162616 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20b0026070 con 0x7f20bc10a570 2024-04-25T14:31:03.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:03.394+0000 7f20b9600700 1 -- 172.21.15.53:0/3707162616 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20b000bc00 con 0x7f20bc10a570 2024-04-25T14:31:03.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:03.394+0000 7f20b9600700 1 -- 172.21.15.53:0/3707162616 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20b0003910 con 0x7f20bc10a570 2024-04-25T14:31:03.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:03.395+0000 7f20c2200700 1 -- 172.21.15.53:0/3707162616 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20bc19a6a0 con 0x7f20bc10a570 2024-04-25T14:31:03.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:03.395+0000 7f20c2200700 1 -- 172.21.15.53:0/3707162616 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20bc19e7a0 con 0x7f20bc10a570 2024-04-25T14:31:03.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:03.396+0000 7f20b9600700 1 -- 172.21.15.53:0/3707162616 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f20b0029070 con 0x7f20bc10a570 2024-04-25T14:31:03.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:03.396+0000 7f20c2200700 1 -- 172.21.15.53:0/3707162616 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20bc105be0 con 0x7f20bc10a570 2024-04-25T14:31:03.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:03.396+0000 7f20b9600700 1 --2- 172.21.15.53:0/3707162616 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f20a40396b0 0x7f20a403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:31:03.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:03.396+0000 7f20b9600700 1 -- 172.21.15.53:0/3707162616 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f20b001e070 con 0x7f20bc10a570 2024-04-25T14:31:03.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:03.397+0000 7f20bbe00700 1 --2- 172.21.15.53:0/3707162616 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f20a40396b0 0x7f20a403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:31:03.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:03.397+0000 7f20bbe00700 1 --2- 172.21.15.53:0/3707162616 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f20a40396b0 0x7f20a403bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f20a8006fd0 tx=0x7f20a8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:31:03.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:03.402+0000 7f20b9600700 1 -- 172.21.15.53:0/3707162616 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f20b0013070 con 0x7f20bc10a570 2024-04-25T14:31:03.640 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:03.640+0000 7f20c2200700 1 -- 172.21.15.53:0/3707162616 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f20bc19b230 con 0x7f20bc10a570 2024-04-25T14:31:03.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:03.804+0000 7f20b9600700 1 -- 172.21.15.53:0/3707162616 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f20b000fe20 con 0x7f20bc10a570 2024-04-25T14:31:03.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:03.808+0000 7f20c2200700 1 -- 172.21.15.53:0/3707162616 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f20a40396b0 msgr2=0x7f20a403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:31:03.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:03.808+0000 7f20c2200700 1 --2- 172.21.15.53:0/3707162616 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f20a40396b0 0x7f20a403bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f20a8006fd0 tx=0x7f20a8006e40 comp rx=0 tx=0).stop 2024-04-25T14:31:03.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:03.808+0000 7f20c2200700 1 -- 172.21.15.53:0/3707162616 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20bc10a570 msgr2=0x7f20bc199ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:31:03.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:03.808+0000 7f20c2200700 1 --2- 172.21.15.53:0/3707162616 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20bc10a570 0x7f20bc199ed0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f20b0006010 tx=0x7f20b0005780 comp rx=0 tx=0).stop 2024-04-25T14:31:03.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:03.808+0000 7f20c2200700 1 -- 172.21.15.53:0/3707162616 shutdown_connections 2024-04-25T14:31:03.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:03.808+0000 7f20c2200700 1 --2- 172.21.15.53:0/3707162616 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f20a40396b0 0x7f20a403bb70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:31:03.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:03.808+0000 7f20c2200700 1 --2- 172.21.15.53:0/3707162616 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20bc10a570 0x7f20bc199ed0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:31:03.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:03.808+0000 7f20c2200700 1 -- 172.21.15.53:0/3707162616 >> 172.21.15.53:0/3707162616 conn(0x7f20bc0fff70 msgr2=0x7f20bc101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:31:03.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:03.808+0000 7f20c2200700 1 -- 172.21.15.53:0/3707162616 shutdown_connections 2024-04-25T14:31:03.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:03.809+0000 7f20c2200700 1 -- 172.21.15.53:0/3707162616 wait complete. 2024-04-25T14:31:03.810 INFO:teuthology.orchestra.run.smithi053.stderr:adjusted tunables profile to default 2024-04-25T14:31:04.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:03 smithi053 ceph-mon[27773]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:31:04.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:03 smithi053 ceph-mon[27773]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:04.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:03 smithi053 ceph-mon[27773]: Updating smithi183:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T14:31:04.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:03 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:04.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:03 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:04.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:03 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:04.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:03 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/3707162616' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T14:31:04.551 INFO:tasks.cephadm:Adding mon.a on smithi053 2024-04-25T14:31:04.551 INFO:tasks.cephadm:Adding mon.b on smithi150 2024-04-25T14:31:04.551 INFO:tasks.cephadm:Adding mon.c on smithi183 2024-04-25T14:31:04.552 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi053:172.21.15.53=a;smithi150:172.21.15.150=b;smithi183:172.21.15.183=c' 2024-04-25T14:31:04.807 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:31:05.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:04 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/3707162616' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T14:31:05.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:04 smithi053 ceph-mon[27773]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:31:06.109 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:06.108+0000 7f9e9747f700 1 -- 172.21.15.183:0/554033703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e901084e0 msgr2=0x7f9e901088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:31:06.109 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:06.108+0000 7f9e9747f700 1 --2- 172.21.15.183:0/554033703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e901084e0 0x7f9e901088c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c009910 tx=0x7f9e8c009c20 comp rx=0 tx=0).stop 2024-04-25T14:31:06.109 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:06.108+0000 7f9e9747f700 1 -- 172.21.15.183:0/554033703 shutdown_connections 2024-04-25T14:31:06.109 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:06.108+0000 7f9e9747f700 1 --2- 172.21.15.183:0/554033703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e901084e0 0x7f9e901088c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:31:06.109 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:06.108+0000 7f9e9747f700 1 -- 172.21.15.183:0/554033703 >> 172.21.15.183:0/554033703 conn(0x7f9e900fdf80 msgr2=0x7f9e901003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:31:06.109 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:06.108+0000 7f9e9747f700 1 -- 172.21.15.183:0/554033703 shutdown_connections 2024-04-25T14:31:06.109 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:06.108+0000 7f9e9747f700 1 -- 172.21.15.183:0/554033703 wait complete. 2024-04-25T14:31:06.109 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:06.109+0000 7f9e9747f700 1 Processor -- start 2024-04-25T14:31:06.109 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:06.109+0000 7f9e9747f700 1 -- start start 2024-04-25T14:31:06.109 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:06.109+0000 7f9e9747f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e901084e0 0x7f9e9019a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:31:06.110 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:06.109+0000 7f9e9747f700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9010c940 con 0x7f9e901084e0 2024-04-25T14:31:06.111 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:06.110+0000 7f9e9521b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e901084e0 0x7f9e9019a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:31:06.111 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:06.110+0000 7f9e9521b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e901084e0 0x7f9e9019a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.183:51290/0 (socket says 172.21.15.183:51290) 2024-04-25T14:31:06.111 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:06.110+0000 7f9e9521b700 1 -- 172.21.15.183:0/324295164 learned_addr learned my addr 172.21.15.183:0/324295164 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-04-25T14:31:06.111 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:06.110+0000 7f9e9521b700 1 -- 172.21.15.183:0/324295164 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e9019a620 con 0x7f9e901084e0 2024-04-25T14:31:06.111 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:06.111+0000 7f9e9521b700 1 --2- 172.21.15.183:0/324295164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e901084e0 0x7f9e9019a0e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c0056d0 tx=0x7f9e8c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:31:06.111 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:06.111+0000 7f9e867fc700 1 -- 172.21.15.183:0/324295164 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e8c026070 con 0x7f9e901084e0 2024-04-25T14:31:06.111 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:06.111+0000 7f9e9747f700 1 -- 172.21.15.183:0/324295164 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e9019a910 con 0x7f9e901084e0 2024-04-25T14:31:06.111 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:06.111+0000 7f9e9747f700 1 -- 172.21.15.183:0/324295164 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e9019e9b0 con 0x7f9e901084e0 2024-04-25T14:31:06.111 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:06.111+0000 7f9e867fc700 1 -- 172.21.15.183:0/324295164 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e8c00bba0 con 0x7f9e901084e0 2024-04-25T14:31:06.111 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:06.111+0000 7f9e867fc700 1 -- 172.21.15.183:0/324295164 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e8c004830 con 0x7f9e901084e0 2024-04-25T14:31:06.112 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:06.112+0000 7f9e9747f700 1 -- 172.21.15.183:0/324295164 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e74005330 con 0x7f9e901084e0 2024-04-25T14:31:06.112 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:06.112+0000 7f9e867fc700 1 -- 172.21.15.183:0/324295164 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f9e8c029070 con 0x7f9e901084e0 2024-04-25T14:31:06.112 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:06.113+0000 7f9e867fc700 1 --2- 172.21.15.183:0/324295164 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f9e7c0397f0 0x7f9e7c03bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:31:06.112 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:06.113+0000 7f9e867fc700 1 -- 172.21.15.183:0/324295164 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9e8c01e070 con 0x7f9e901084e0 2024-04-25T14:31:06.113 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:06.113+0000 7f9e94a1a700 1 --2- 172.21.15.183:0/324295164 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f9e7c0397f0 0x7f9e7c03bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:31:06.113 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:06.114+0000 7f9e94a1a700 1 --2- 172.21.15.183:0/324295164 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f9e7c0397f0 0x7f9e7c03bcb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9e80006fd0 tx=0x7f9e80006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:31:06.128 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:06.128+0000 7f9e867fc700 1 -- 172.21.15.183:0/324295164 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9e8c013070 con 0x7f9e901084e0 2024-04-25T14:31:06.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:05 smithi053 ceph-mon[27773]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:06.337 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:06.338+0000 7f9e9747f700 1 -- 172.21.15.183:0/324295164 --> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi053:172.21.15.53=a;smithi150:172.21.15.150=b;smithi183:172.21.15.183=c", "target": ["mon-mgr", ""]}) -- 0x7f9e74000c80 con 0x7f9e7c0397f0 2024-04-25T14:31:06.353 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:06.353+0000 7f9e867fc700 1 -- 172.21.15.183:0/324295164 <== mgr.14152 v2:172.21.15.53:6800/4154933108 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f9e74000c80 con 0x7f9e7c0397f0 2024-04-25T14:31:06.353 INFO:teuthology.orchestra.run.smithi183.stdout:Scheduled mon update... 2024-04-25T14:31:06.355 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:06.355+0000 7f9e9747f700 1 -- 172.21.15.183:0/324295164 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f9e7c0397f0 msgr2=0x7f9e7c03bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:31:06.355 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:06.355+0000 7f9e9747f700 1 --2- 172.21.15.183:0/324295164 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f9e7c0397f0 0x7f9e7c03bcb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9e80006fd0 tx=0x7f9e80006e40 comp rx=0 tx=0).stop 2024-04-25T14:31:06.355 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:06.355+0000 7f9e9747f700 1 -- 172.21.15.183:0/324295164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e901084e0 msgr2=0x7f9e9019a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:31:06.355 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:06.355+0000 7f9e9747f700 1 --2- 172.21.15.183:0/324295164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e901084e0 0x7f9e9019a0e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c0056d0 tx=0x7f9e8c0057b0 comp rx=0 tx=0).stop 2024-04-25T14:31:06.355 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:06.355+0000 7f9e9747f700 1 -- 172.21.15.183:0/324295164 shutdown_connections 2024-04-25T14:31:06.355 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:06.355+0000 7f9e9747f700 1 --2- 172.21.15.183:0/324295164 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f9e7c0397f0 0x7f9e7c03bcb0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:31:06.355 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:06.355+0000 7f9e9747f700 1 --2- 172.21.15.183:0/324295164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e901084e0 0x7f9e9019a0e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:31:06.356 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:06.355+0000 7f9e9747f700 1 -- 172.21.15.183:0/324295164 >> 172.21.15.183:0/324295164 conn(0x7f9e900fdf80 msgr2=0x7f9e90109fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:31:06.356 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:06.355+0000 7f9e9747f700 1 -- 172.21.15.183:0/324295164 shutdown_connections 2024-04-25T14:31:06.356 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:06.355+0000 7f9e9747f700 1 -- 172.21.15.183:0/324295164 wait complete. 2024-04-25T14:31:06.949 DEBUG:teuthology.orchestra.run.smithi150:mon.b> sudo journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mon.b.service 2024-04-25T14:31:06.953 DEBUG:teuthology.orchestra.run.smithi183:mon.c> sudo journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mon.c.service 2024-04-25T14:31:06.956 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T14:31:06.956 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T14:31:07.612 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:31:07.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:07 smithi053 ceph-mon[27773]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:07.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:07 smithi053 ceph-mon[27773]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi053:172.21.15.53=a;smithi150:172.21.15.150=b;smithi183:172.21.15.183=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:31:07.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:07 smithi053 ceph-mon[27773]: Saving service mon spec with placement smithi053:172.21.15.53=a;smithi150:172.21.15.150=b;smithi183:172.21.15.183=c;count:3 2024-04-25T14:31:07.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:07 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:07.784 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:07 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:31:07.784 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:07 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:07.784 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:07 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:31:07.784 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:07 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:07.784 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:07 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:31:07.784 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:07 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:07.784 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:07 smithi053 ceph-mon[27773]: Deploying daemon mon.c on smithi183 2024-04-25T14:31:09.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:09 smithi053 ceph-mon[27773]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:10.657 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:10.655+0000 7f1bfe586700 1 -- 172.21.15.183:0/3456731940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bf8101590 msgr2=0x7f1bf8101970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:31:10.657 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:10.655+0000 7f1bfe586700 1 --2- 172.21.15.183:0/3456731940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bf8101590 0x7f1bf8101970 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1be8009910 tx=0x7f1be8009c20 comp rx=0 tx=0).stop 2024-04-25T14:31:10.657 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:10.655+0000 7f1bfe586700 1 -- 172.21.15.183:0/3456731940 shutdown_connections 2024-04-25T14:31:10.657 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:10.655+0000 7f1bfe586700 1 --2- 172.21.15.183:0/3456731940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bf8101590 0x7f1bf8101970 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:31:10.657 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:10.655+0000 7f1bfe586700 1 -- 172.21.15.183:0/3456731940 >> 172.21.15.183:0/3456731940 conn(0x7f1bf80fce40 msgr2=0x7f1bf80ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:31:10.657 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:10.655+0000 7f1bfe586700 1 -- 172.21.15.183:0/3456731940 shutdown_connections 2024-04-25T14:31:10.657 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:10.655+0000 7f1bfe586700 1 -- 172.21.15.183:0/3456731940 wait complete. 2024-04-25T14:31:10.658 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:10.655+0000 7f1bfe586700 1 Processor -- start 2024-04-25T14:31:10.658 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:10.655+0000 7f1bfe586700 1 -- start start 2024-04-25T14:31:10.658 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:10.656+0000 7f1bfe586700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bf8101590 0x7f1bf8197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:31:10.658 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:10.656+0000 7f1bfe586700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf810b810 con 0x7f1bf8101590 2024-04-25T14:31:10.658 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:10.656+0000 7f1bf7fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bf8101590 0x7f1bf8197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:31:10.658 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:10.656+0000 7f1bf7fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bf8101590 0x7f1bf8197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.183:34736/0 (socket says 172.21.15.183:34736) 2024-04-25T14:31:10.658 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:10.656+0000 7f1bf7fff700 1 -- 172.21.15.183:0/3628102683 learned_addr learned my addr 172.21.15.183:0/3628102683 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-04-25T14:31:10.659 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:10.657+0000 7f1bf7fff700 1 -- 172.21.15.183:0/3628102683 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bf81983e0 con 0x7f1bf8101590 2024-04-25T14:31:10.659 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:10.657+0000 7f1bf7fff700 1 --2- 172.21.15.183:0/3628102683 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bf8101590 0x7f1bf8197ea0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1be80056d0 tx=0x7f1be80057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:31:10.659 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:10.658+0000 7f1bf5ffb700 1 -- 172.21.15.183:0/3628102683 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1be8026070 con 0x7f1bf8101590 2024-04-25T14:31:10.659 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:10.658+0000 7f1bfe586700 1 -- 172.21.15.183:0/3628102683 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bf81986d0 con 0x7f1bf8101590 2024-04-25T14:31:10.659 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:10.658+0000 7f1bfe586700 1 -- 172.21.15.183:0/3628102683 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bf819c770 con 0x7f1bf8101590 2024-04-25T14:31:10.659 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:10.658+0000 7f1bf5ffb700 1 -- 172.21.15.183:0/3628102683 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1be800bba0 con 0x7f1bf8101590 2024-04-25T14:31:10.660 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:10.658+0000 7f1bf5ffb700 1 -- 172.21.15.183:0/3628102683 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1be8030060 con 0x7f1bf8101590 2024-04-25T14:31:10.660 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:10.658+0000 7f1bfe586700 1 -- 172.21.15.183:0/3628102683 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bf8102a70 con 0x7f1bf8101590 2024-04-25T14:31:10.660 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:10.659+0000 7f1bf5ffb700 1 -- 172.21.15.183:0/3628102683 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f1be8029070 con 0x7f1bf8101590 2024-04-25T14:31:10.660 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:10.659+0000 7f1bf5ffb700 1 --2- 172.21.15.183:0/3628102683 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f1be4039650 0x7f1be403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:31:10.660 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:10.659+0000 7f1bf5ffb700 1 -- 172.21.15.183:0/3628102683 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1be801e070 con 0x7f1bf8101590 2024-04-25T14:31:10.661 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:10.660+0000 7f1beffff700 1 --2- 172.21.15.183:0/3628102683 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f1be4039650 0x7f1be403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:31:10.661 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:10.661+0000 7f1beffff700 1 --2- 172.21.15.183:0/3628102683 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f1be4039650 0x7f1be403bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1be0006fd0 tx=0x7f1be0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:31:10.676 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:10.675+0000 7f1bf5ffb700 1 -- 172.21.15.183:0/3628102683 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1be8013070 con 0x7f1bf8101590 2024-04-25T14:31:10.984 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:10.984+0000 7f1bfe586700 1 -- 172.21.15.183:0/3628102683 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f1bf8199200 con 0x7f1bf8101590 2024-04-25T14:31:10.985 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:10.985+0000 7f1bf5ffb700 1 -- 172.21.15.183:0/3628102683 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f1be801f310 con 0x7f1bf8101590 2024-04-25T14:31:10.985 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-25T14:31:10.985 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":1,"fsid":"fd30e6ce-030f-11ef-bc93-c7b262605968","modified":"2024-04-25T14:28:49.679784Z","created":"2024-04-25T14:28:49.679784Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3300","nonce":0},{"type":"v1","addr":"172.21.15.53:6789","nonce":0}]},"addr":"172.21.15.53:6789/0","public_addr":"172.21.15.53:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T14:31:10.988 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:10.987+0000 7f1bfe586700 1 -- 172.21.15.183:0/3628102683 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f1be4039650 msgr2=0x7f1be403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:31:10.988 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:10.987+0000 7f1bfe586700 1 --2- 172.21.15.183:0/3628102683 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f1be4039650 0x7f1be403bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1be0006fd0 tx=0x7f1be0006e40 comp rx=0 tx=0).stop 2024-04-25T14:31:10.988 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:10.987+0000 7f1bfe586700 1 -- 172.21.15.183:0/3628102683 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bf8101590 msgr2=0x7f1bf8197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:31:10.988 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:10.987+0000 7f1bfe586700 1 --2- 172.21.15.183:0/3628102683 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bf8101590 0x7f1bf8197ea0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1be80056d0 tx=0x7f1be80057b0 comp rx=0 tx=0).stop 2024-04-25T14:31:10.988 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:10.987+0000 7f1bfe586700 1 -- 172.21.15.183:0/3628102683 shutdown_connections 2024-04-25T14:31:10.988 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:10.987+0000 7f1bfe586700 1 --2- 172.21.15.183:0/3628102683 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f1be4039650 0x7f1be403bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:31:10.988 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:10.987+0000 7f1bfe586700 1 --2- 172.21.15.183:0/3628102683 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bf8101590 0x7f1bf8197ea0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:31:10.988 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:10.987+0000 7f1bfe586700 1 -- 172.21.15.183:0/3628102683 >> 172.21.15.183:0/3628102683 conn(0x7f1bf80fce40 msgr2=0x7f1bf806a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:31:10.988 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:10.987+0000 7f1bfe586700 1 -- 172.21.15.183:0/3628102683 shutdown_connections 2024-04-25T14:31:10.988 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:10.987+0000 7f1bfe586700 1 -- 172.21.15.183:0/3628102683 wait complete. 2024-04-25T14:31:10.989 INFO:teuthology.orchestra.run.smithi183.stderr:dumped monmap epoch 1 2024-04-25T14:31:11.121 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 podman[32553]: 2024-04-25 14:31:11.095865713 +0000 UTC m=+0.332512819 container start d39383c27b474810cd5eaf27f0f64e3f221be15ed885b44e32452b018b8e4a0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416) 2024-04-25T14:31:11.121 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T14:31:11.121 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-25T14:31:11.121 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: pidfile_write: ignore empty --pid-file 2024-04-25T14:31:11.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: load: jerasure load: lrc 2024-04-25T14:31:11.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: RocksDB version: 7.9.2 2024-04-25T14:31:11.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Git sha 0 2024-04-25T14:31:11.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-25T14:31:11.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: DB SUMMARY 2024-04-25T14:31:11.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: DB Session ID: 94EKWIJV5592KPTYVGPX 2024-04-25T14:31:11.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: CURRENT file: CURRENT 2024-04-25T14:31:11.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T14:31:11.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-25T14:31:11.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-25T14:31:11.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-25T14:31:11.456 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.error_if_exists: 0 2024-04-25T14:31:11.456 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.create_if_missing: 0 2024-04-25T14:31:11.456 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.paranoid_checks: 1 2024-04-25T14:31:11.456 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T14:31:11.456 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T14:31:11.456 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T14:31:11.456 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.env: 0x55fb87c97cc0 2024-04-25T14:31:11.456 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T14:31:11.456 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.info_log: 0x55fb89be5be0 2024-04-25T14:31:11.456 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T14:31:11.456 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.statistics: (nil) 2024-04-25T14:31:11.457 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.use_fsync: 0 2024-04-25T14:31:11.457 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.max_log_file_size: 0 2024-04-25T14:31:11.457 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T14:31:11.457 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T14:31:11.457 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T14:31:11.457 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T14:31:11.457 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.allow_fallocate: 1 2024-04-25T14:31:11.457 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T14:31:11.457 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T14:31:11.457 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.use_direct_reads: 0 2024-04-25T14:31:11.457 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T14:31:11.457 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T14:31:11.457 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.db_log_dir: 2024-04-25T14:31:11.457 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.wal_dir: 2024-04-25T14:31:11.458 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T14:31:11.458 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T14:31:11.458 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T14:31:11.458 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T14:31:11.458 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T14:31:11.458 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T14:31:11.458 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T14:31:11.458 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T14:31:11.458 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.write_buffer_manager: 0x55fb89b790e0 2024-04-25T14:31:11.458 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T14:31:11.458 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T14:31:11.458 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T14:31:11.459 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T14:31:11.459 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T14:31:11.459 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T14:31:11.459 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T14:31:11.459 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T14:31:11.459 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.unordered_write: 0 2024-04-25T14:31:11.459 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T14:31:11.459 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T14:31:11.459 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T14:31:11.459 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T14:31:11.459 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.row_cache: None 2024-04-25T14:31:11.459 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.wal_filter: None 2024-04-25T14:31:11.459 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T14:31:11.459 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T14:31:11.460 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.two_write_queues: 0 2024-04-25T14:31:11.460 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T14:31:11.460 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.wal_compression: 0 2024-04-25T14:31:11.460 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.atomic_flush: 0 2024-04-25T14:31:11.460 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T14:31:11.460 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T14:31:11.460 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T14:31:11.460 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.log_readahead_size: 0 2024-04-25T14:31:11.460 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T14:31:11.460 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T14:31:11.460 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T14:31:11.460 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T14:31:11.460 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T14:31:11.460 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T14:31:11.460 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T14:31:11.461 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.max_background_jobs: 2 2024-04-25T14:31:11.461 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.max_background_compactions: -1 2024-04-25T14:31:11.461 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.max_subcompactions: 1 2024-04-25T14:31:11.462 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T14:31:11.462 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T14:31:11.463 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T14:31:11.463 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T14:31:11.463 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T14:31:11.463 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T14:31:11.463 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T14:31:11.463 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T14:31:11.463 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.max_open_files: -1 2024-04-25T14:31:11.464 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T14:31:11.464 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T14:31:11.464 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T14:31:11.464 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T14:31:11.464 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.max_background_flushes: -1 2024-04-25T14:31:11.464 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Compression algorithms supported: 2024-04-25T14:31:11.464 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T14:31:11.464 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: kZSTD supported: 0 2024-04-25T14:31:11.464 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: kXpressCompression supported: 0 2024-04-25T14:31:11.464 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T14:31:11.465 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: kZlibCompression supported: 1 2024-04-25T14:31:11.465 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: kSnappyCompression supported: 1 2024-04-25T14:31:11.465 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: kLZ4Compression supported: 1 2024-04-25T14:31:11.465 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: kBZip2Compression supported: 0 2024-04-25T14:31:11.465 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T14:31:11.465 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T14:31:11.465 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-25T14:31:11.465 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T14:31:11.465 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T14:31:11.465 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.merge_operator: 2024-04-25T14:31:11.465 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.compaction_filter: None 2024-04-25T14:31:11.466 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T14:31:11.466 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T14:31:11.466 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T14:31:11.466 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T14:31:11.466 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55fb89be5dc0) 2024-04-25T14:31:11.466 INFO:journalctl@ceph.mon.c.smithi183.stdout: cache_index_and_filter_blocks: 1 2024-04-25T14:31:11.466 INFO:journalctl@ceph.mon.c.smithi183.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T14:31:11.466 INFO:journalctl@ceph.mon.c.smithi183.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T14:31:11.466 INFO:journalctl@ceph.mon.c.smithi183.stdout: pin_top_level_index_and_filter: 1 2024-04-25T14:31:11.466 INFO:journalctl@ceph.mon.c.smithi183.stdout: index_type: 0 2024-04-25T14:31:11.466 INFO:journalctl@ceph.mon.c.smithi183.stdout: data_block_index_type: 0 2024-04-25T14:31:11.466 INFO:journalctl@ceph.mon.c.smithi183.stdout: index_shortening: 1 2024-04-25T14:31:11.466 INFO:journalctl@ceph.mon.c.smithi183.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T14:31:11.467 INFO:journalctl@ceph.mon.c.smithi183.stdout: checksum: 4 2024-04-25T14:31:11.467 INFO:journalctl@ceph.mon.c.smithi183.stdout: no_block_cache: 0 2024-04-25T14:31:11.467 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_cache: 0x55fb89bab1f0 2024-04-25T14:31:11.467 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_cache_name: BinnedLRUCache 2024-04-25T14:31:11.467 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_cache_options: 2024-04-25T14:31:11.467 INFO:journalctl@ceph.mon.c.smithi183.stdout: capacity : 536870912 2024-04-25T14:31:11.467 INFO:journalctl@ceph.mon.c.smithi183.stdout: num_shard_bits : 4 2024-04-25T14:31:11.467 INFO:journalctl@ceph.mon.c.smithi183.stdout: strict_capacity_limit : 0 2024-04-25T14:31:11.467 INFO:journalctl@ceph.mon.c.smithi183.stdout: high_pri_pool_ratio: 0.000 2024-04-25T14:31:11.467 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_cache_compressed: (nil) 2024-04-25T14:31:11.467 INFO:journalctl@ceph.mon.c.smithi183.stdout: persistent_cache: (nil) 2024-04-25T14:31:11.467 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_size: 4096 2024-04-25T14:31:11.467 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_size_deviation: 10 2024-04-25T14:31:11.468 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_restart_interval: 16 2024-04-25T14:31:11.468 INFO:journalctl@ceph.mon.c.smithi183.stdout: index_block_restart_interval: 1 2024-04-25T14:31:11.468 INFO:journalctl@ceph.mon.c.smithi183.stdout: metadata_block_size: 4096 2024-04-25T14:31:11.468 INFO:journalctl@ceph.mon.c.smithi183.stdout: partition_filters: 0 2024-04-25T14:31:11.468 INFO:journalctl@ceph.mon.c.smithi183.stdout: use_delta_encoding: 1 2024-04-25T14:31:11.468 INFO:journalctl@ceph.mon.c.smithi183.stdout: filter_policy: bloomfilter 2024-04-25T14:31:11.468 INFO:journalctl@ceph.mon.c.smithi183.stdout: whole_key_filtering: 1 2024-04-25T14:31:11.468 INFO:journalctl@ceph.mon.c.smithi183.stdout: verify_compression: 0 2024-04-25T14:31:11.468 INFO:journalctl@ceph.mon.c.smithi183.stdout: read_amp_bytes_per_bit: 0 2024-04-25T14:31:11.468 INFO:journalctl@ceph.mon.c.smithi183.stdout: format_version: 5 2024-04-25T14:31:11.468 INFO:journalctl@ceph.mon.c.smithi183.stdout: enable_index_compression: 1 2024-04-25T14:31:11.468 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_align: 0 2024-04-25T14:31:11.468 INFO:journalctl@ceph.mon.c.smithi183.stdout: max_auto_readahead_size: 262144 2024-04-25T14:31:11.468 INFO:journalctl@ceph.mon.c.smithi183.stdout: prepopulate_block_cache: 0 2024-04-25T14:31:11.469 INFO:journalctl@ceph.mon.c.smithi183.stdout: initial_auto_readahead_size: 8192 2024-04-25T14:31:11.469 INFO:journalctl@ceph.mon.c.smithi183.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T14:31:11.469 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T14:31:11.469 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T14:31:11.469 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.compression: NoCompression 2024-04-25T14:31:11.469 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T14:31:11.469 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T14:31:11.469 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T14:31:11.469 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.num_levels: 7 2024-04-25T14:31:11.469 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T14:31:11.469 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T14:31:11.469 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T14:31:11.469 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T14:31:11.469 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T14:31:11.469 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T14:31:11.470 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T14:31:11.470 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T14:31:11.470 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T14:31:11.470 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T14:31:11.470 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T14:31:11.470 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T14:31:11.470 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T14:31:11.470 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T14:31:11.470 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T14:31:11.470 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T14:31:11.470 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T14:31:11.470 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T14:31:11.471 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T14:31:11.471 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T14:31:11.471 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T14:31:11.471 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T14:31:11.471 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T14:31:11.471 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T14:31:11.471 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T14:31:11.471 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T14:31:11.471 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T14:31:11.471 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T14:31:11.471 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T14:31:11.471 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T14:31:11.471 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T14:31:11.471 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T14:31:11.472 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T14:31:11.472 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T14:31:11.472 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T14:31:11.472 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T14:31:11.472 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T14:31:11.472 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T14:31:11.472 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T14:31:11.472 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T14:31:11.472 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T14:31:11.472 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T14:31:11.472 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T14:31:11.472 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T14:31:11.472 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T14:31:11.473 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T14:31:11.473 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T14:31:11.473 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T14:31:11.473 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T14:31:11.473 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T14:31:11.473 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T14:31:11.473 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T14:31:11.473 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T14:31:11.473 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T14:31:11.473 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.inplace_update_support: 0 2024-04-25T14:31:11.473 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T14:31:11.473 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T14:31:11.473 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T14:31:11.473 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T14:31:11.473 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.bloom_locality: 0 2024-04-25T14:31:11.474 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.max_successive_merges: 0 2024-04-25T14:31:11.474 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T14:31:11.474 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T14:31:11.474 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T14:31:11.474 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T14:31:11.474 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.ttl: 2592000 2024-04-25T14:31:11.474 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T14:31:11.474 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T14:31:11.474 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T14:31:11.474 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.enable_blob_files: false 2024-04-25T14:31:11.474 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.min_blob_size: 0 2024-04-25T14:31:11.474 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T14:31:11.474 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T14:31:11.474 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T14:31:11.475 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T14:31:11.475 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T14:31:11.475 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T14:31:11.475 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T14:31:11.475 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T14:31:11.475 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-25T14:31:11.475 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-25T14:31:11.475 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 92fe905d-1ace-4737-bb54-c7f7dcb06ad3 2024-04-25T14:31:11.475 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714055471127886, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-25T14:31:11.475 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-25T14:31:11.475 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714055471128437, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714055471, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "92fe905d-1ace-4737-bb54-c7f7dcb06ad3", "db_session_id": "94EKWIJV5592KPTYVGPX", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-25T14:31:11.476 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714055471128545, "job": 1, "event": "recovery_finished"} 2024-04-25T14:31:11.476 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-25T14:31:11.476 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T14:31:11.476 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55fb89cc0000 2024-04-25T14:31:11.476 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: DB pointer 0x55fb89caa000 2024-04-25T14:31:11.476 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T14:31:11.476 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T14:31:11.476 INFO:journalctl@ceph.mon.c.smithi183.stdout: ** DB Stats ** 2024-04-25T14:31:11.476 INFO:journalctl@ceph.mon.c.smithi183.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T14:31:11.476 INFO:journalctl@ceph.mon.c.smithi183.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T14:31:11.476 INFO:journalctl@ceph.mon.c.smithi183.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T14:31:11.476 INFO:journalctl@ceph.mon.c.smithi183.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T14:31:11.476 INFO:journalctl@ceph.mon.c.smithi183.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T14:31:11.476 INFO:journalctl@ceph.mon.c.smithi183.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T14:31:11.477 INFO:journalctl@ceph.mon.c.smithi183.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T14:31:11.477 INFO:journalctl@ceph.mon.c.smithi183.stdout: 2024-04-25T14:31:11.477 INFO:journalctl@ceph.mon.c.smithi183.stdout: ** Compaction Stats [default] ** 2024-04-25T14:31:11.477 INFO:journalctl@ceph.mon.c.smithi183.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T14:31:11.477 INFO:journalctl@ceph.mon.c.smithi183.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T14:31:11.477 INFO:journalctl@ceph.mon.c.smithi183.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T14:31:11.477 INFO:journalctl@ceph.mon.c.smithi183.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T14:31:11.477 INFO:journalctl@ceph.mon.c.smithi183.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T14:31:11.477 INFO:journalctl@ceph.mon.c.smithi183.stdout: 2024-04-25T14:31:11.477 INFO:journalctl@ceph.mon.c.smithi183.stdout: ** Compaction Stats [default] ** 2024-04-25T14:31:11.477 INFO:journalctl@ceph.mon.c.smithi183.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T14:31:11.477 INFO:journalctl@ceph.mon.c.smithi183.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T14:31:11.477 INFO:journalctl@ceph.mon.c.smithi183.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T14:31:11.477 INFO:journalctl@ceph.mon.c.smithi183.stdout: 2024-04-25T14:31:11.478 INFO:journalctl@ceph.mon.c.smithi183.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T14:31:11.478 INFO:journalctl@ceph.mon.c.smithi183.stdout: 2024-04-25T14:31:11.478 INFO:journalctl@ceph.mon.c.smithi183.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T14:31:11.478 INFO:journalctl@ceph.mon.c.smithi183.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T14:31:11.478 INFO:journalctl@ceph.mon.c.smithi183.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T14:31:11.478 INFO:journalctl@ceph.mon.c.smithi183.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T14:31:11.478 INFO:journalctl@ceph.mon.c.smithi183.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T14:31:11.478 INFO:journalctl@ceph.mon.c.smithi183.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T14:31:11.478 INFO:journalctl@ceph.mon.c.smithi183.stdout: Cumulative compaction: 0.00 GB write, 0.49 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T14:31:11.478 INFO:journalctl@ceph.mon.c.smithi183.stdout: Interval compaction: 0.00 GB write, 0.49 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T14:31:11.478 INFO:journalctl@ceph.mon.c.smithi183.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T14:31:11.478 INFO:journalctl@ceph.mon.c.smithi183.stdout: Block cache BinnedLRUCache@0x55fb89bab1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-04-25T14:31:11.478 INFO:journalctl@ceph.mon.c.smithi183.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-25T14:31:11.478 INFO:journalctl@ceph.mon.c.smithi183.stdout: 2024-04-25T14:31:11.478 INFO:journalctl@ceph.mon.c.smithi183.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T14:31:11.479 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-25T14:31:11.479 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: using public_addr v2:172.21.15.183:0/0 -> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] 2024-04-25T14:31:11.479 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: starting mon.c rank -1 at public addrs [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] at bind addrs [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid fd30e6ce-030f-11ef-bc93-c7b262605968 2024-04-25T14:31:11.479 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: mon.c@-1(???) e0 preinit fsid fd30e6ce-030f-11ef-bc93-c7b262605968 2024-04-25T14:31:11.479 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: mon.c@-1(synchronizing).mds e1 new map 2024-04-25T14:31:11.479 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-25T14:31:11.479 INFO:journalctl@ceph.mon.c.smithi183.stdout: e1 2024-04-25T14:31:11.479 INFO:journalctl@ceph.mon.c.smithi183.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T14:31:11.479 INFO:journalctl@ceph.mon.c.smithi183.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T14:31:11.479 INFO:journalctl@ceph.mon.c.smithi183.stdout: legacy client fscid: -1 2024-04-25T14:31:11.479 INFO:journalctl@ceph.mon.c.smithi183.stdout: 2024-04-25T14:31:11.479 INFO:journalctl@ceph.mon.c.smithi183.stdout: No filesystems configured 2024-04-25T14:31:11.479 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-25T14:31:11.479 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-25T14:31:11.480 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-25T14:31:11.480 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-25T14:31:11.480 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-25T14:31:11.480 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-25T14:31:11.480 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T14:31:11.480 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T14:31:11.480 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T14:31:11.480 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T14:31:11.480 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:11.480 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: Deploying cephadm binary to smithi150 2024-04-25T14:31:11.480 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:11.480 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:11.480 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:11.480 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: Added host smithi150 2024-04-25T14:31:11.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:31:11.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:11.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:11.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:11.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:11.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:11.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:31:11.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:11.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:11.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:11.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:11.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:11.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi150", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:31:11.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:11.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:31:11.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: Updating smithi150:/etc/ceph/ceph.conf 2024-04-25T14:31:11.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: Updating smithi150:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:31:11.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:31:11.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: Updating smithi150:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T14:31:11.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:11.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:11.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:11.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:11.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi183", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:31:11.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: Deploying cephadm binary to smithi183 2024-04-25T14:31:11.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:11.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:11.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:11.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:11.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: Added host smithi183 2024-04-25T14:31:11.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:31:11.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:11.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:11.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:11.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:11.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:31:11.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:11.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:11.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:11.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:11.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:11.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi183", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:31:11.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:11.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:31:11.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: Updating smithi183:/etc/ceph/ceph.conf 2024-04-25T14:31:11.484 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: Updating smithi183:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:31:11.484 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:31:11.484 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:11.484 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: Updating smithi183:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T14:31:11.484 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:11.484 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:11.484 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:11.484 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/3707162616' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T14:31:11.484 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/3707162616' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T14:31:11.484 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:31:11.484 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:11.484 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:11.484 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi053:172.21.15.53=a;smithi150:172.21.15.150=b;smithi183:172.21.15.183=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:31:11.484 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: Saving service mon spec with placement smithi053:172.21.15.53=a;smithi150:172.21.15.150=b;smithi183:172.21.15.183=c;count:3 2024-04-25T14:31:11.485 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:11.485 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:31:11.485 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:11.485 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:31:11.485 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:11.485 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:31:11.485 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:11.485 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: Deploying daemon mon.c on smithi183 2024-04-25T14:31:11.485 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:11.485 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 ceph-mon[32584]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-25T14:31:11.485 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 bash[32553]: d39383c27b474810cd5eaf27f0f64e3f221be15ed885b44e32452b018b8e4a0c 2024-04-25T14:31:11.485 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:11 smithi183 systemd[1]: Started Ceph mon.c for fd30e6ce-030f-11ef-bc93-c7b262605968. 2024-04-25T14:31:12.813 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T14:31:12.814 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T14:31:13.059 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:31:16.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:16 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:31:16.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:16 smithi053 ceph-mon[27773]: mon.a calling monitor election 2024-04-25T14:31:16.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:16 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:31:16.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:16 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:31:16.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:16 smithi053 ceph-mon[27773]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:16.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:16 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:31:16.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:16 smithi053 ceph-mon[27773]: mon.c calling monitor election 2024-04-25T14:31:16.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:16 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:31:16.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:16 smithi053 ceph-mon[27773]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:16.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:16 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:31:16.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:16 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:31:16.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:16 smithi053 ceph-mon[27773]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T14:31:16.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:16 smithi053 ceph-mon[27773]: monmap epoch 2 2024-04-25T14:31:16.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:16 smithi053 ceph-mon[27773]: fsid fd30e6ce-030f-11ef-bc93-c7b262605968 2024-04-25T14:31:16.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:16 smithi053 ceph-mon[27773]: last_changed 2024-04-25T14:31:11.164419+0000 2024-04-25T14:31:16.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:16 smithi053 ceph-mon[27773]: created 2024-04-25T14:28:49.679784+0000 2024-04-25T14:31:16.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:16 smithi053 ceph-mon[27773]: min_mon_release 19 (squid) 2024-04-25T14:31:16.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:16 smithi053 ceph-mon[27773]: election_strategy: 1 2024-04-25T14:31:16.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:16 smithi053 ceph-mon[27773]: 0: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.a 2024-04-25T14:31:16.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:16 smithi053 ceph-mon[27773]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-04-25T14:31:16.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:16 smithi053 ceph-mon[27773]: fsmap 2024-04-25T14:31:16.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:16 smithi053 ceph-mon[27773]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:31:16.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:16 smithi053 ceph-mon[27773]: mgrmap e13: x(active, since 62s) 2024-04-25T14:31:16.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:16 smithi053 ceph-mon[27773]: overall HEALTH_OK 2024-04-25T14:31:16.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:16 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:16.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:16 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:16.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:16 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:31:16.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:16 smithi183 ceph-mon[32584]: mon.a calling monitor election 2024-04-25T14:31:16.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:16 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:31:16.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:16 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:31:16.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:16 smithi183 ceph-mon[32584]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:16.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:16 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:31:16.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:16 smithi183 ceph-mon[32584]: mon.c calling monitor election 2024-04-25T14:31:16.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:16 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:31:16.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:16 smithi183 ceph-mon[32584]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:16.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:16 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:31:16.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:16 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:31:16.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:16 smithi183 ceph-mon[32584]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T14:31:16.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:16 smithi183 ceph-mon[32584]: monmap epoch 2 2024-04-25T14:31:16.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:16 smithi183 ceph-mon[32584]: fsid fd30e6ce-030f-11ef-bc93-c7b262605968 2024-04-25T14:31:16.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:16 smithi183 ceph-mon[32584]: last_changed 2024-04-25T14:31:11.164419+0000 2024-04-25T14:31:16.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:16 smithi183 ceph-mon[32584]: created 2024-04-25T14:28:49.679784+0000 2024-04-25T14:31:16.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:16 smithi183 ceph-mon[32584]: min_mon_release 19 (squid) 2024-04-25T14:31:16.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:16 smithi183 ceph-mon[32584]: election_strategy: 1 2024-04-25T14:31:16.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:16 smithi183 ceph-mon[32584]: 0: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.a 2024-04-25T14:31:16.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:16 smithi183 ceph-mon[32584]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-04-25T14:31:16.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:16 smithi183 ceph-mon[32584]: fsmap 2024-04-25T14:31:16.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:16 smithi183 ceph-mon[32584]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:31:16.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:16 smithi183 ceph-mon[32584]: mgrmap e13: x(active, since 62s) 2024-04-25T14:31:16.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:16 smithi183 ceph-mon[32584]: overall HEALTH_OK 2024-04-25T14:31:16.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:16 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:16.706 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:16 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:17.368 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:17 smithi183 ceph-mon[32584]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:17.368 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:17 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:17.368 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:17 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:31:17.368 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:17 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:17.368 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:17 smithi183 ceph-mon[32584]: Deploying daemon mon.b on smithi150 2024-04-25T14:31:17.368 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:17 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:31:17.370 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.367+0000 7f1b5edf8700 1 -- 172.21.15.183:0/226353354 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b400035c0 msgr2=0x7f1b40005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:31:17.370 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.367+0000 7f1b5edf8700 1 --2- 172.21.15.183:0/226353354 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b400035c0 0x7f1b40005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1b58079c80 tx=0x7f1b48010ec0 comp rx=0 tx=0).stop 2024-04-25T14:31:17.370 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.367+0000 7f1b5edf8700 1 -- 172.21.15.183:0/226353354 shutdown_connections 2024-04-25T14:31:17.370 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.367+0000 7f1b5edf8700 1 --2- 172.21.15.183:0/226353354 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b400035c0 0x7f1b40005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:31:17.370 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.367+0000 7f1b5edf8700 1 -- 172.21.15.183:0/226353354 >> 172.21.15.183:0/226353354 conn(0x7f1b58077af0 msgr2=0x7f1b58077f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:31:17.371 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.367+0000 7f1b5edf8700 1 -- 172.21.15.183:0/226353354 shutdown_connections 2024-04-25T14:31:17.371 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.367+0000 7f1b5edf8700 1 -- 172.21.15.183:0/226353354 wait complete. 2024-04-25T14:31:17.371 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.368+0000 7f1b5edf8700 1 Processor -- start 2024-04-25T14:31:17.371 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.368+0000 7f1b5edf8700 1 -- start start 2024-04-25T14:31:17.371 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.368+0000 7f1b5edf8700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b400035c0 0x7f1b58198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:31:17.371 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.368+0000 7f1b5edf8700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b581985b0 0x7f1b5819c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:31:17.371 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.368+0000 7f1b5edf8700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1b58108290 con 0x7f1b400035c0 2024-04-25T14:31:17.371 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.368+0000 7f1b5edf8700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1b58108110 con 0x7f1b581985b0 2024-04-25T14:31:17.371 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.368+0000 7f1b57fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b581985b0 0x7f1b5819c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:31:17.371 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.368+0000 7f1b57fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b581985b0 0x7f1b5819c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:59874/0 (socket says 172.21.15.183:59874) 2024-04-25T14:31:17.371 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.368+0000 7f1b57fff700 1 -- 172.21.15.183:0/860328997 learned_addr learned my addr 172.21.15.183:0/860328997 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-04-25T14:31:17.371 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.369+0000 7f1b57fff700 1 -- 172.21.15.183:0/860328997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b581985b0 msgr2=0x7f1b5819c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-25T14:31:17.371 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.369+0000 7f1b57fff700 1 -- 172.21.15.183:0/860328997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b581985b0 msgr2=0x7f1b5819c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T14:31:17.371 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.369+0000 7f1b57fff700 1 --2- 172.21.15.183:0/860328997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b581985b0 0x7f1b5819c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T14:31:17.372 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.369+0000 7f1b57fff700 1 --2- 172.21.15.183:0/860328997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b581985b0 0x7f1b5819c8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:31:17.372 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.369+0000 7f1b5cb94700 1 --2- 172.21.15.183:0/860328997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b400035c0 0x7f1b58198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:31:17.372 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.370+0000 7f1b5cb94700 1 -- 172.21.15.183:0/860328997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b581985b0 msgr2=0x7f1b5819c8f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:31:17.372 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.370+0000 7f1b5cb94700 1 --2- 172.21.15.183:0/860328997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b581985b0 0x7f1b5819c8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:31:17.372 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.370+0000 7f1b5cb94700 1 -- 172.21.15.183:0/860328997 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b58198bf0 con 0x7f1b400035c0 2024-04-25T14:31:17.372 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.370+0000 7f1b5cb94700 1 --2- 172.21.15.183:0/860328997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b400035c0 0x7f1b58198070 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1b4801c000 tx=0x7f1b480054f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:31:17.373 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.370+0000 7f1b55ffb700 1 -- 172.21.15.183:0/860328997 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f1b48012c20 con 0x7f1b400035c0 2024-04-25T14:31:17.373 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.370+0000 7f1b5edf8700 1 -- 172.21.15.183:0/860328997 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b5819ce30 con 0x7f1b400035c0 2024-04-25T14:31:17.373 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.370+0000 7f1b55ffb700 1 -- 172.21.15.183:0/860328997 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b480044e0 con 0x7f1b400035c0 2024-04-25T14:31:17.373 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.370+0000 7f1b55ffb700 1 -- 172.21.15.183:0/860328997 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f1b4801b410 con 0x7f1b400035c0 2024-04-25T14:31:17.373 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.371+0000 7f1b5edf8700 1 -- 172.21.15.183:0/860328997 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b5819d390 con 0x7f1b400035c0 2024-04-25T14:31:17.373 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.372+0000 7f1b5edf8700 1 -- 172.21.15.183:0/860328997 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b58102320 con 0x7f1b400035c0 2024-04-25T14:31:17.373 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.372+0000 7f1b55ffb700 1 -- 172.21.15.183:0/860328997 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f1b4800c040 con 0x7f1b400035c0 2024-04-25T14:31:17.373 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.372+0000 7f1b55ffb700 1 --2- 172.21.15.183:0/860328997 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f1b440393f0 0x7f1b4403b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:31:17.374 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.372+0000 7f1b55ffb700 1 -- 172.21.15.183:0/860328997 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1b48021040 con 0x7f1b400035c0 2024-04-25T14:31:17.374 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.373+0000 7f1b57fff700 1 --2- 172.21.15.183:0/860328997 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f1b440393f0 0x7f1b4403b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:31:17.374 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.374+0000 7f1b57fff700 1 --2- 172.21.15.183:0/860328997 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f1b440393f0 0x7f1b4403b8b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009180 tx=0x7f1b4c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:31:17.388 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.388+0000 7f1b55ffb700 1 -- 172.21.15.183:0/860328997 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1b4800e960 con 0x7f1b400035c0 2024-04-25T14:31:17.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:17 smithi053 ceph-mon[27773]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:17.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:17 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:17.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:17 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:31:17.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:17 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:17.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:17 smithi053 ceph-mon[27773]: Deploying daemon mon.b on smithi150 2024-04-25T14:31:17.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:17 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:31:17.668 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.668+0000 7f1b5edf8700 1 -- 172.21.15.183:0/860328997 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f1b5819d640 con 0x7f1b400035c0 2024-04-25T14:31:17.669 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.668+0000 7f1b55ffb700 1 -- 172.21.15.183:0/860328997 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f1b48012dc0 con 0x7f1b400035c0 2024-04-25T14:31:17.669 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-25T14:31:17.669 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":2,"fsid":"fd30e6ce-030f-11ef-bc93-c7b262605968","modified":"2024-04-25T14:31:11.164419Z","created":"2024-04-25T14:28:49.679784Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3300","nonce":0},{"type":"v1","addr":"172.21.15.53:6789","nonce":0}]},"addr":"172.21.15.53:6789/0","public_addr":"172.21.15.53:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T14:31:17.672 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.672+0000 7f1b5edf8700 1 -- 172.21.15.183:0/860328997 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f1b440393f0 msgr2=0x7f1b4403b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:31:17.672 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.672+0000 7f1b5edf8700 1 --2- 172.21.15.183:0/860328997 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f1b440393f0 0x7f1b4403b8b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009180 tx=0x7f1b4c008040 comp rx=0 tx=0).stop 2024-04-25T14:31:17.672 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.672+0000 7f1b5edf8700 1 -- 172.21.15.183:0/860328997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b400035c0 msgr2=0x7f1b58198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:31:17.672 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.672+0000 7f1b5edf8700 1 --2- 172.21.15.183:0/860328997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b400035c0 0x7f1b58198070 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1b4801c000 tx=0x7f1b480054f0 comp rx=0 tx=0).stop 2024-04-25T14:31:17.673 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.672+0000 7f1b5edf8700 1 -- 172.21.15.183:0/860328997 shutdown_connections 2024-04-25T14:31:17.673 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.672+0000 7f1b5edf8700 1 --2- 172.21.15.183:0/860328997 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f1b440393f0 0x7f1b4403b8b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:31:17.673 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.672+0000 7f1b5edf8700 1 --2- 172.21.15.183:0/860328997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b400035c0 0x7f1b58198070 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:31:17.673 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.672+0000 7f1b5edf8700 1 --2- 172.21.15.183:0/860328997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b581985b0 0x7f1b5819c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:31:17.673 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.672+0000 7f1b5edf8700 1 -- 172.21.15.183:0/860328997 >> 172.21.15.183:0/860328997 conn(0x7f1b58077af0 msgr2=0x7f1b580fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:31:17.673 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.672+0000 7f1b5edf8700 1 -- 172.21.15.183:0/860328997 shutdown_connections 2024-04-25T14:31:17.673 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:17.672+0000 7f1b5edf8700 1 -- 172.21.15.183:0/860328997 wait complete. 2024-04-25T14:31:17.673 INFO:teuthology.orchestra.run.smithi183.stderr:dumped monmap epoch 2 2024-04-25T14:31:18.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:18 smithi053 ceph-mon[27773]: from='client.? 172.21.15.183:0/860328997' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T14:31:18.533 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:31:18 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:31:18.164+0000 7fd830600700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T14:31:18.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:18 smithi183 ceph-mon[32584]: from='client.? 172.21.15.183:0/860328997' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T14:31:19.296 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T14:31:19.297 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T14:31:19.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:19 smithi053 ceph-mon[27773]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:19.537 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:31:19.659 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:19 smithi183 ceph-mon[32584]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:20.193 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: mon.b@-1(synchronizing).mds e1 new map 2024-04-25T14:31:20.193 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: mon.b@-1(synchronizing).mds e1 print_map 2024-04-25T14:31:20.193 INFO:journalctl@ceph.mon.b.smithi150.stdout: e1 2024-04-25T14:31:20.193 INFO:journalctl@ceph.mon.b.smithi150.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T14:31:20.194 INFO:journalctl@ceph.mon.b.smithi150.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T14:31:20.194 INFO:journalctl@ceph.mon.b.smithi150.stdout: legacy client fscid: -1 2024-04-25T14:31:20.194 INFO:journalctl@ceph.mon.b.smithi150.stdout: 2024-04-25T14:31:20.194 INFO:journalctl@ceph.mon.b.smithi150.stdout: No filesystems configured 2024-04-25T14:31:20.194 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-25T14:31:20.194 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-25T14:31:20.194 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-25T14:31:20.194 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-25T14:31:20.194 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-25T14:31:20.195 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-25T14:31:20.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T14:31:20.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T14:31:20.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T14:31:20.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T14:31:20.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:20.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: Deploying cephadm binary to smithi150 2024-04-25T14:31:20.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:20.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:20.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:20.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: Added host smithi150 2024-04-25T14:31:20.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:31:20.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:20.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:20.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:20.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:20.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:20.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:31:20.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:20.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:20.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:20.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:20.463 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:20.463 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi150", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:31:20.463 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:20.463 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:31:20.463 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: Updating smithi150:/etc/ceph/ceph.conf 2024-04-25T14:31:20.463 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: Updating smithi150:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:31:20.463 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:31:20.463 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: Updating smithi150:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T14:31:20.463 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:20.463 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:20.463 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:20.463 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:20.463 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi183", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:31:20.463 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: Deploying cephadm binary to smithi183 2024-04-25T14:31:20.463 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:20.464 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:20.464 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:20.464 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:20.464 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: Added host smithi183 2024-04-25T14:31:20.464 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:31:20.464 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:20.464 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:20.464 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:20.464 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:20.464 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:31:20.464 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:20.464 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:20.464 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:20.464 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:20.464 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:20.465 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi183", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:31:20.465 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:20.465 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:31:20.465 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: Updating smithi183:/etc/ceph/ceph.conf 2024-04-25T14:31:20.465 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: Updating smithi183:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:31:20.465 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:31:20.465 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:20.465 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: Updating smithi183:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T14:31:20.465 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:20.465 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:20.465 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:20.465 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/3707162616' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T14:31:20.465 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/3707162616' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T14:31:20.465 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:31:20.465 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:20.466 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:20.466 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi053:172.21.15.53=a;smithi150:172.21.15.150=b;smithi183:172.21.15.183=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:31:20.466 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: Saving service mon spec with placement smithi053:172.21.15.53=a;smithi150:172.21.15.150=b;smithi183:172.21.15.183=c;count:3 2024-04-25T14:31:20.466 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:20.466 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:31:20.467 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:20.467 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:31:20.467 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:20.467 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:31:20.467 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:20.467 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: Deploying daemon mon.c on smithi183 2024-04-25T14:31:20.467 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:20.467 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:31:20.467 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: mon.a calling monitor election 2024-04-25T14:31:20.467 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:31:20.467 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:31:20.467 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:20.467 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:31:20.467 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: mon.c calling monitor election 2024-04-25T14:31:20.467 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:31:20.468 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:20.468 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:31:20.468 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:31:20.468 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T14:31:20.468 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: monmap epoch 2 2024-04-25T14:31:20.468 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: fsid fd30e6ce-030f-11ef-bc93-c7b262605968 2024-04-25T14:31:20.468 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: last_changed 2024-04-25T14:31:11.164419+0000 2024-04-25T14:31:20.468 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: created 2024-04-25T14:28:49.679784+0000 2024-04-25T14:31:20.468 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: min_mon_release 19 (squid) 2024-04-25T14:31:20.468 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: election_strategy: 1 2024-04-25T14:31:20.468 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: 0: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.a 2024-04-25T14:31:20.468 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-04-25T14:31:20.468 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: fsmap 2024-04-25T14:31:20.468 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:31:20.469 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: mgrmap e13: x(active, since 62s) 2024-04-25T14:31:20.469 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: overall HEALTH_OK 2024-04-25T14:31:20.469 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:20.469 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:20.469 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:20.469 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:20.469 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:31:20.469 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:20.469 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: Deploying daemon mon.b on smithi150 2024-04-25T14:31:20.469 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:31:20.469 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: from='client.? 172.21.15.183:0/860328997' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T14:31:20.469 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:20.469 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 ceph-mon[32504]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-25T14:31:20.469 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 bash[32485]: 807bc911e52dd35645dcff82f412ead739e9c66f97bcbcd0d7b1850ae8baa005 2024-04-25T14:31:20.469 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:20 smithi150 systemd[1]: Started Ceph mon.b for fd30e6ce-030f-11ef-bc93-c7b262605968. 2024-04-25T14:31:25.460 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.458+0000 7f740b830700 1 -- 172.21.15.183:0/378010754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74041074e0 msgr2=0x7f73ec005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:31:25.460 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.458+0000 7f740b830700 1 --2- 172.21.15.183:0/378010754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74041074e0 0x7f73ec005610 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f73f4009c10 tx=0x7f73f4009f20 comp rx=0 tx=0).stop 2024-04-25T14:31:25.460 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.458+0000 7f740b830700 1 -- 172.21.15.183:0/378010754 shutdown_connections 2024-04-25T14:31:25.461 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.458+0000 7f740b830700 1 --2- 172.21.15.183:0/378010754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74041074e0 0x7f73ec005610 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:31:25.461 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.459+0000 7f740b830700 1 -- 172.21.15.183:0/378010754 >> 172.21.15.183:0/378010754 conn(0x7f74040fcf40 msgr2=0x7f74040ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:31:25.461 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.459+0000 7f740b830700 1 -- 172.21.15.183:0/378010754 shutdown_connections 2024-04-25T14:31:25.461 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.459+0000 7f740b830700 1 -- 172.21.15.183:0/378010754 wait complete. 2024-04-25T14:31:25.461 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.460+0000 7f740b830700 1 Processor -- start 2024-04-25T14:31:25.461 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.460+0000 7f740b830700 1 -- start start 2024-04-25T14:31:25.463 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.460+0000 7f740b830700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74041074e0 0x7f74041980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:31:25.463 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.460+0000 7f740b830700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7404198600 0x7f740419c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:31:25.463 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.460+0000 7f740b830700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f740419ced0 0x7f740419d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:31:25.463 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.460+0000 7f740b830700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7404109360 con 0x7f7404198600 2024-04-25T14:31:25.463 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.460+0000 7f740b830700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f74041091e0 con 0x7f740419ced0 2024-04-25T14:31:25.463 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.460+0000 7f740b830700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f74041094e0 con 0x7f74041074e0 2024-04-25T14:31:25.463 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.460+0000 7f74095cc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74041074e0 0x7f74041980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:31:25.463 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.460+0000 7f74095cc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74041074e0 0x7f74041980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:40196/0 (socket says 172.21.15.183:40196) 2024-04-25T14:31:25.463 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.460+0000 7f74095cc700 1 -- 172.21.15.183:0/3846853680 learned_addr learned my addr 172.21.15.183:0/3846853680 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-04-25T14:31:25.464 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.460+0000 7f74095cc700 1 -- 172.21.15.183:0/3846853680 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74041074e0 msgr2=0x7f74041980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-25T14:31:25.464 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.460+0000 7f7409dcd700 1 --2- 172.21.15.183:0/3846853680 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f740419ced0 0x7f740419d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:31:25.464 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.460+0000 7f74095cc700 1 -- 172.21.15.183:0/3846853680 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74041074e0 msgr2=0x7f74041980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T14:31:25.464 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.460+0000 7f74095cc700 1 --2- 172.21.15.183:0/3846853680 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74041074e0 0x7f74041980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T14:31:25.464 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.460+0000 7f74095cc700 1 --2- 172.21.15.183:0/3846853680 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74041074e0 0x7f74041980c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:31:25.464 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.460+0000 7f7409dcd700 1 -- 172.21.15.183:0/3846853680 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f740419ced0 msgr2=0x7f740419d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-25T14:31:25.464 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.460+0000 7f7409dcd700 1 -- 172.21.15.183:0/3846853680 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f740419ced0 msgr2=0x7f740419d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T14:31:25.464 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.460+0000 7f7409dcd700 1 --2- 172.21.15.183:0/3846853680 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f740419ced0 0x7f740419d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T14:31:25.464 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.460+0000 7f7409dcd700 1 --2- 172.21.15.183:0/3846853680 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f740419ced0 0x7f740419d380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:31:25.464 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.461+0000 7f7408dcb700 1 --2- 172.21.15.183:0/3846853680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7404198600 0x7f740419c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:31:25.464 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.461+0000 7f7408dcb700 1 -- 172.21.15.183:0/3846853680 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74041074e0 msgr2=0x7f74041980c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:31:25.464 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.461+0000 7f7408dcb700 1 --2- 172.21.15.183:0/3846853680 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74041074e0 0x7f74041980c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:31:25.464 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.461+0000 7f7408dcb700 1 -- 172.21.15.183:0/3846853680 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f740419ced0 msgr2=0x7f740419d380 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:31:25.464 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.461+0000 7f7408dcb700 1 --2- 172.21.15.183:0/3846853680 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f740419ced0 0x7f740419d380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:31:25.464 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.461+0000 7f7408dcb700 1 -- 172.21.15.183:0/3846853680 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74041a4c40 con 0x7f7404198600 2024-04-25T14:31:25.465 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.462+0000 7f7408dcb700 1 --2- 172.21.15.183:0/3846853680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7404198600 0x7f740419c990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f73f800b550 tx=0x7f73f800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:31:25.465 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.462+0000 7f74027fc700 1 -- 172.21.15.183:0/3846853680 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f8004230 con 0x7f7404198600 2024-04-25T14:31:25.465 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.462+0000 7f740b830700 1 -- 172.21.15.183:0/3846853680 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74041a4e70 con 0x7f7404198600 2024-04-25T14:31:25.465 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.462+0000 7f74027fc700 1 -- 172.21.15.183:0/3846853680 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73f80043d0 con 0x7f7404198600 2024-04-25T14:31:25.465 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.462+0000 7f74027fc700 1 -- 172.21.15.183:0/3846853680 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f8011b90 con 0x7f7404198600 2024-04-25T14:31:25.465 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.462+0000 7f740b830700 1 -- 172.21.15.183:0/3846853680 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74041a53a0 con 0x7f7404198600 2024-04-25T14:31:25.466 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.463+0000 7f740b830700 1 -- 172.21.15.183:0/3846853680 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74040640a0 con 0x7f7404198600 2024-04-25T14:31:25.466 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.464+0000 7f74027fc700 1 -- 172.21.15.183:0/3846853680 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f73f80036a0 con 0x7f7404198600 2024-04-25T14:31:25.466 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.464+0000 7f74027fc700 1 --2- 172.21.15.183:0/3846853680 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f73f0039ae0 0x7f73f003bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:31:25.466 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.464+0000 7f74027fc700 1 -- 172.21.15.183:0/3846853680 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f73f804c340 con 0x7f7404198600 2024-04-25T14:31:25.466 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.465+0000 7f74095cc700 1 --2- 172.21.15.183:0/3846853680 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f73f0039ae0 0x7f73f003bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:31:25.466 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.466+0000 7f74095cc700 1 --2- 172.21.15.183:0/3846853680 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f73f0039ae0 0x7f73f003bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f73f4009a50 tx=0x7f73f400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:31:25.482 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.481+0000 7f74027fc700 1 -- 172.21.15.183:0/3846853680 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f73f8010380 con 0x7f7404198600 2024-04-25T14:31:25.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:25 smithi053 ceph-mon[27773]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:25.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:25 smithi053 ceph-mon[27773]: mon.c calling monitor election 2024-04-25T14:31:25.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:25 smithi053 ceph-mon[27773]: mon.a calling monitor election 2024-04-25T14:31:25.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:25 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:31:25.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:25 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:31:25.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:25 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:31:25.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:25 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:31:25.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:25 smithi053 ceph-mon[27773]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:25.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:25 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:31:25.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:25 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:31:25.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:25 smithi053 ceph-mon[27773]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:25.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:25 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:31:25.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:25 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:31:25.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:25 smithi053 ceph-mon[27773]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T14:31:25.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:25 smithi053 ceph-mon[27773]: monmap epoch 3 2024-04-25T14:31:25.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:25 smithi053 ceph-mon[27773]: fsid fd30e6ce-030f-11ef-bc93-c7b262605968 2024-04-25T14:31:25.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:25 smithi053 ceph-mon[27773]: last_changed 2024-04-25T14:31:20.199475+0000 2024-04-25T14:31:25.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:25 smithi053 ceph-mon[27773]: created 2024-04-25T14:28:49.679784+0000 2024-04-25T14:31:25.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:25 smithi053 ceph-mon[27773]: min_mon_release 19 (squid) 2024-04-25T14:31:25.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:25 smithi053 ceph-mon[27773]: election_strategy: 1 2024-04-25T14:31:25.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:25 smithi053 ceph-mon[27773]: 0: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.a 2024-04-25T14:31:25.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:25 smithi053 ceph-mon[27773]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-04-25T14:31:25.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:25 smithi053 ceph-mon[27773]: 2: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2024-04-25T14:31:25.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:25 smithi053 ceph-mon[27773]: fsmap 2024-04-25T14:31:25.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:25 smithi053 ceph-mon[27773]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:31:25.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:25 smithi053 ceph-mon[27773]: mgrmap e13: x(active, since 71s) 2024-04-25T14:31:25.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:25 smithi053 ceph-mon[27773]: overall HEALTH_OK 2024-04-25T14:31:25.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:25 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:25.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:25 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:25.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:25 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:25.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:25 smithi183 ceph-mon[32584]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:25.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:25 smithi183 ceph-mon[32584]: mon.c calling monitor election 2024-04-25T14:31:25.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:25 smithi183 ceph-mon[32584]: mon.a calling monitor election 2024-04-25T14:31:25.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:25 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:31:25.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:25 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:31:25.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:25 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:31:25.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:25 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:31:25.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:25 smithi183 ceph-mon[32584]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:25.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:25 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:31:25.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:25 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:31:25.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:25 smithi183 ceph-mon[32584]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:25.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:25 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:31:25.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:25 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:31:25.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:25 smithi183 ceph-mon[32584]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T14:31:25.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:25 smithi183 ceph-mon[32584]: monmap epoch 3 2024-04-25T14:31:25.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:25 smithi183 ceph-mon[32584]: fsid fd30e6ce-030f-11ef-bc93-c7b262605968 2024-04-25T14:31:25.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:25 smithi183 ceph-mon[32584]: last_changed 2024-04-25T14:31:20.199475+0000 2024-04-25T14:31:25.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:25 smithi183 ceph-mon[32584]: created 2024-04-25T14:28:49.679784+0000 2024-04-25T14:31:25.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:25 smithi183 ceph-mon[32584]: min_mon_release 19 (squid) 2024-04-25T14:31:25.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:25 smithi183 ceph-mon[32584]: election_strategy: 1 2024-04-25T14:31:25.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:25 smithi183 ceph-mon[32584]: 0: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.a 2024-04-25T14:31:25.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:25 smithi183 ceph-mon[32584]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-04-25T14:31:25.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:25 smithi183 ceph-mon[32584]: 2: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2024-04-25T14:31:25.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:25 smithi183 ceph-mon[32584]: fsmap 2024-04-25T14:31:25.706 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:25 smithi183 ceph-mon[32584]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:31:25.706 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:25 smithi183 ceph-mon[32584]: mgrmap e13: x(active, since 71s) 2024-04-25T14:31:25.706 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:25 smithi183 ceph-mon[32584]: overall HEALTH_OK 2024-04-25T14:31:25.706 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:25 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:25.706 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:25 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:25.706 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:25 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:25.784 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.784+0000 7f740b830700 1 -- 172.21.15.183:0/3846853680 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f74041a56f0 con 0x7f7404198600 2024-04-25T14:31:25.785 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.785+0000 7f74027fc700 1 -- 172.21.15.183:0/3846853680 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7f73f800d280 con 0x7f7404198600 2024-04-25T14:31:25.785 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-25T14:31:25.785 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":3,"fsid":"fd30e6ce-030f-11ef-bc93-c7b262605968","modified":"2024-04-25T14:31:20.199475Z","created":"2024-04-25T14:28:49.679784Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3300","nonce":0},{"type":"v1","addr":"172.21.15.53:6789","nonce":0}]},"addr":"172.21.15.53:6789/0","public_addr":"172.21.15.53:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:3300","nonce":0},{"type":"v1","addr":"172.21.15.150:6789","nonce":0}]},"addr":"172.21.15.150:6789/0","public_addr":"172.21.15.150:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T14:31:25.788 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.787+0000 7f740b830700 1 -- 172.21.15.183:0/3846853680 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f73f0039ae0 msgr2=0x7f73f003bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:31:25.788 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.787+0000 7f740b830700 1 --2- 172.21.15.183:0/3846853680 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f73f0039ae0 0x7f73f003bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f73f4009a50 tx=0x7f73f400b040 comp rx=0 tx=0).stop 2024-04-25T14:31:25.788 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.787+0000 7f740b830700 1 -- 172.21.15.183:0/3846853680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7404198600 msgr2=0x7f740419c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:31:25.788 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.787+0000 7f740b830700 1 --2- 172.21.15.183:0/3846853680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7404198600 0x7f740419c990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f73f800b550 tx=0x7f73f800b910 comp rx=0 tx=0).stop 2024-04-25T14:31:25.788 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.787+0000 7f740b830700 1 -- 172.21.15.183:0/3846853680 shutdown_connections 2024-04-25T14:31:25.788 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.787+0000 7f740b830700 1 --2- 172.21.15.183:0/3846853680 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f73f0039ae0 0x7f73f003bfa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:31:25.788 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.787+0000 7f740b830700 1 --2- 172.21.15.183:0/3846853680 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f740419ced0 0x7f740419d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:31:25.788 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.787+0000 7f740b830700 1 --2- 172.21.15.183:0/3846853680 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74041074e0 0x7f74041980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:31:25.788 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.787+0000 7f740b830700 1 --2- 172.21.15.183:0/3846853680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7404198600 0x7f740419c990 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:31:25.789 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.787+0000 7f740b830700 1 -- 172.21.15.183:0/3846853680 >> 172.21.15.183:0/3846853680 conn(0x7f74040fcf40 msgr2=0x7f74041068e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:31:25.789 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.787+0000 7f740b830700 1 -- 172.21.15.183:0/3846853680 shutdown_connections 2024-04-25T14:31:25.789 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:25.787+0000 7f740b830700 1 -- 172.21.15.183:0/3846853680 wait complete. 2024-04-25T14:31:25.789 INFO:teuthology.orchestra.run.smithi183.stderr:dumped monmap epoch 3 2024-04-25T14:31:26.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:26 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:26.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:26 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:31:26.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:26 smithi183 ceph-mon[32584]: from='client.? 172.21.15.183:0/3846853680' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T14:31:26.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:26 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:31:26.533 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-25T14:31:26.533 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-25T14:31:26.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:26 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:26.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:26 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:31:26.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:26 smithi053 ceph-mon[27773]: from='client.? 172.21.15.183:0/3846853680' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T14:31:26.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:26 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:31:26.968 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:31:27.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:27 smithi150 ceph-mon[32504]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:27.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:27 smithi150 ceph-mon[32504]: mon.c calling monitor election 2024-04-25T14:31:27.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:27 smithi150 ceph-mon[32504]: mon.a calling monitor election 2024-04-25T14:31:27.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:27 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:31:27.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:27 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:31:27.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:27 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:31:27.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:27 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:31:27.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:27 smithi150 ceph-mon[32504]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:27.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:27 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:31:27.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:27 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:31:27.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:27 smithi150 ceph-mon[32504]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:27.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:27 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:31:27.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:27 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:31:27.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:27 smithi150 ceph-mon[32504]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T14:31:27.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:27 smithi150 ceph-mon[32504]: monmap epoch 3 2024-04-25T14:31:27.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:27 smithi150 ceph-mon[32504]: fsid fd30e6ce-030f-11ef-bc93-c7b262605968 2024-04-25T14:31:27.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:27 smithi150 ceph-mon[32504]: last_changed 2024-04-25T14:31:20.199475+0000 2024-04-25T14:31:27.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:27 smithi150 ceph-mon[32504]: created 2024-04-25T14:28:49.679784+0000 2024-04-25T14:31:27.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:27 smithi150 ceph-mon[32504]: min_mon_release 19 (squid) 2024-04-25T14:31:27.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:27 smithi150 ceph-mon[32504]: election_strategy: 1 2024-04-25T14:31:27.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:27 smithi150 ceph-mon[32504]: 0: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.a 2024-04-25T14:31:27.463 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:27 smithi150 ceph-mon[32504]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-04-25T14:31:27.463 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:27 smithi150 ceph-mon[32504]: 2: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2024-04-25T14:31:27.463 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:27 smithi150 ceph-mon[32504]: fsmap 2024-04-25T14:31:27.463 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:27 smithi150 ceph-mon[32504]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:31:27.463 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:27 smithi150 ceph-mon[32504]: mgrmap e13: x(active, since 71s) 2024-04-25T14:31:27.463 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:27 smithi150 ceph-mon[32504]: overall HEALTH_OK 2024-04-25T14:31:27.463 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:27 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:27.463 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:27 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:27.463 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:27 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:27.463 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:27 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:27.463 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:27 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:31:27.464 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:27 smithi150 ceph-mon[32504]: from='client.? 172.21.15.183:0/3846853680' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T14:31:27.464 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:27 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:31:28.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:28 smithi053 ceph-mon[27773]: mon.b calling monitor election 2024-04-25T14:31:28.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:28 smithi053 ceph-mon[27773]: mon.b calling monitor election 2024-04-25T14:31:28.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:28 smithi053 ceph-mon[27773]: mon.c calling monitor election 2024-04-25T14:31:28.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:28 smithi053 ceph-mon[27773]: mon.a calling monitor election 2024-04-25T14:31:28.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:28 smithi053 ceph-mon[27773]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T14:31:28.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:28 smithi053 ceph-mon[27773]: monmap epoch 3 2024-04-25T14:31:28.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:28 smithi053 ceph-mon[27773]: fsid fd30e6ce-030f-11ef-bc93-c7b262605968 2024-04-25T14:31:28.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:28 smithi053 ceph-mon[27773]: last_changed 2024-04-25T14:31:20.199475+0000 2024-04-25T14:31:28.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:28 smithi053 ceph-mon[27773]: created 2024-04-25T14:28:49.679784+0000 2024-04-25T14:31:28.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:28 smithi053 ceph-mon[27773]: min_mon_release 19 (squid) 2024-04-25T14:31:28.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:28 smithi053 ceph-mon[27773]: election_strategy: 1 2024-04-25T14:31:28.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:28 smithi053 ceph-mon[27773]: 0: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.a 2024-04-25T14:31:28.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:28 smithi053 ceph-mon[27773]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-04-25T14:31:28.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:28 smithi053 ceph-mon[27773]: 2: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2024-04-25T14:31:28.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:28 smithi053 ceph-mon[27773]: fsmap 2024-04-25T14:31:28.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:28 smithi053 ceph-mon[27773]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:31:28.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:28 smithi053 ceph-mon[27773]: mgrmap e13: x(active, since 73s) 2024-04-25T14:31:28.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:28 smithi053 ceph-mon[27773]: overall HEALTH_OK 2024-04-25T14:31:28.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:28 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:28.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:28 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:28.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:28 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:28.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:28 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:31:28.285 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:28 smithi053 ceph-mon[27773]: Updating smithi053:/etc/ceph/ceph.conf 2024-04-25T14:31:28.285 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:28 smithi053 ceph-mon[27773]: Updating smithi150:/etc/ceph/ceph.conf 2024-04-25T14:31:28.285 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:28 smithi053 ceph-mon[27773]: Updating smithi183:/etc/ceph/ceph.conf 2024-04-25T14:31:28.285 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:28 smithi053 ceph-mon[27773]: Updating smithi150:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:31:28.285 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:28 smithi053 ceph-mon[27773]: Updating smithi183:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:31:28.285 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:28 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:28.285 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:28 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:28.285 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:28 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:28.285 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:28 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:28.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:28 smithi183 ceph-mon[32584]: mon.b calling monitor election 2024-04-25T14:31:28.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:28 smithi183 ceph-mon[32584]: mon.b calling monitor election 2024-04-25T14:31:28.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:28 smithi183 ceph-mon[32584]: mon.c calling monitor election 2024-04-25T14:31:28.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:28 smithi183 ceph-mon[32584]: mon.a calling monitor election 2024-04-25T14:31:28.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:28 smithi183 ceph-mon[32584]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T14:31:28.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:28 smithi183 ceph-mon[32584]: monmap epoch 3 2024-04-25T14:31:28.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:28 smithi183 ceph-mon[32584]: fsid fd30e6ce-030f-11ef-bc93-c7b262605968 2024-04-25T14:31:28.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:28 smithi183 ceph-mon[32584]: last_changed 2024-04-25T14:31:20.199475+0000 2024-04-25T14:31:28.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:28 smithi183 ceph-mon[32584]: created 2024-04-25T14:28:49.679784+0000 2024-04-25T14:31:28.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:28 smithi183 ceph-mon[32584]: min_mon_release 19 (squid) 2024-04-25T14:31:28.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:28 smithi183 ceph-mon[32584]: election_strategy: 1 2024-04-25T14:31:28.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:28 smithi183 ceph-mon[32584]: 0: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.a 2024-04-25T14:31:28.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:28 smithi183 ceph-mon[32584]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-04-25T14:31:28.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:28 smithi183 ceph-mon[32584]: 2: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2024-04-25T14:31:28.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:28 smithi183 ceph-mon[32584]: fsmap 2024-04-25T14:31:28.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:28 smithi183 ceph-mon[32584]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:31:28.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:28 smithi183 ceph-mon[32584]: mgrmap e13: x(active, since 73s) 2024-04-25T14:31:28.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:28 smithi183 ceph-mon[32584]: overall HEALTH_OK 2024-04-25T14:31:28.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:28 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:28.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:28 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:28.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:28 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:28.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:28 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:31:28.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:28 smithi183 ceph-mon[32584]: Updating smithi053:/etc/ceph/ceph.conf 2024-04-25T14:31:28.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:28 smithi183 ceph-mon[32584]: Updating smithi150:/etc/ceph/ceph.conf 2024-04-25T14:31:28.456 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:28 smithi183 ceph-mon[32584]: Updating smithi183:/etc/ceph/ceph.conf 2024-04-25T14:31:28.456 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:28 smithi183 ceph-mon[32584]: Updating smithi150:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:31:28.456 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:28 smithi183 ceph-mon[32584]: Updating smithi183:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:31:28.456 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:28 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:28.456 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:28 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:28.456 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:28 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:28.456 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:28 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:28.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:28 smithi150 ceph-mon[32504]: mon.b calling monitor election 2024-04-25T14:31:28.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:28 smithi150 ceph-mon[32504]: mon.b calling monitor election 2024-04-25T14:31:28.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:28 smithi150 ceph-mon[32504]: mon.c calling monitor election 2024-04-25T14:31:28.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:28 smithi150 ceph-mon[32504]: mon.a calling monitor election 2024-04-25T14:31:28.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:28 smithi150 ceph-mon[32504]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T14:31:28.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:28 smithi150 ceph-mon[32504]: monmap epoch 3 2024-04-25T14:31:28.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:28 smithi150 ceph-mon[32504]: fsid fd30e6ce-030f-11ef-bc93-c7b262605968 2024-04-25T14:31:28.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:28 smithi150 ceph-mon[32504]: last_changed 2024-04-25T14:31:20.199475+0000 2024-04-25T14:31:28.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:28 smithi150 ceph-mon[32504]: created 2024-04-25T14:28:49.679784+0000 2024-04-25T14:31:28.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:28 smithi150 ceph-mon[32504]: min_mon_release 19 (squid) 2024-04-25T14:31:28.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:28 smithi150 ceph-mon[32504]: election_strategy: 1 2024-04-25T14:31:28.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:28 smithi150 ceph-mon[32504]: 0: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.a 2024-04-25T14:31:28.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:28 smithi150 ceph-mon[32504]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-04-25T14:31:28.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:28 smithi150 ceph-mon[32504]: 2: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2024-04-25T14:31:28.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:28 smithi150 ceph-mon[32504]: fsmap 2024-04-25T14:31:28.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:28 smithi150 ceph-mon[32504]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:31:28.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:28 smithi150 ceph-mon[32504]: mgrmap e13: x(active, since 73s) 2024-04-25T14:31:28.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:28 smithi150 ceph-mon[32504]: overall HEALTH_OK 2024-04-25T14:31:28.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:28 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:28.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:28 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:28.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:28 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:28.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:28 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:31:28.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:28 smithi150 ceph-mon[32504]: Updating smithi053:/etc/ceph/ceph.conf 2024-04-25T14:31:28.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:28 smithi150 ceph-mon[32504]: Updating smithi150:/etc/ceph/ceph.conf 2024-04-25T14:31:28.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:28 smithi150 ceph-mon[32504]: Updating smithi183:/etc/ceph/ceph.conf 2024-04-25T14:31:28.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:28 smithi150 ceph-mon[32504]: Updating smithi150:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:31:28.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:28 smithi150 ceph-mon[32504]: Updating smithi183:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:31:28.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:28 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:28.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:28 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:28.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:28 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:28.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:28 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:28.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:28.997+0000 7fb50bc00700 1 -- 172.21.15.53:0/3991325786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5041035f0 msgr2=0x7fb5041039d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:31:28.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:28.997+0000 7fb50bc00700 1 --2- 172.21.15.53:0/3991325786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5041035f0 0x7fb5041039d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4009910 tx=0x7fb4f4009c20 comp rx=0 tx=0).stop 2024-04-25T14:31:28.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:28.997+0000 7fb50bc00700 1 -- 172.21.15.53:0/3991325786 shutdown_connections 2024-04-25T14:31:28.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:28.997+0000 7fb50bc00700 1 --2- 172.21.15.53:0/3991325786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5041035f0 0x7fb5041039d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:31:28.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:28.997+0000 7fb50bc00700 1 -- 172.21.15.53:0/3991325786 >> 172.21.15.53:0/3991325786 conn(0x7fb5040fce40 msgr2=0x7fb5040ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:31:28.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:28.997+0000 7fb50bc00700 1 -- 172.21.15.53:0/3991325786 shutdown_connections 2024-04-25T14:31:28.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:28.997+0000 7fb50bc00700 1 -- 172.21.15.53:0/3991325786 wait complete. 2024-04-25T14:31:28.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:28.998+0000 7fb50bc00700 1 Processor -- start 2024-04-25T14:31:28.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:28.998+0000 7fb50bc00700 1 -- start start 2024-04-25T14:31:28.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:28.999+0000 7fb50bc00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5041035f0 0x7fb50419a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:31:29.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:28.999+0000 7fb50bc00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb50419a8e0 0x7fb50419ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:31:29.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:28.999+0000 7fb50bc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb50419f1b0 0x7fb50419f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:31:29.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:28.999+0000 7fb50bc00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb50410c990 con 0x7fb50419f1b0 2024-04-25T14:31:29.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:28.999+0000 7fb50bc00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb50410c810 con 0x7fb5041035f0 2024-04-25T14:31:29.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:28.999+0000 7fb50bc00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb50410cb10 con 0x7fb50419a8e0 2024-04-25T14:31:29.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:28.999+0000 7fb509c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb50419f1b0 0x7fb50419f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:31:29.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:28.999+0000 7fb4fbe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb50419a8e0 0x7fb50419ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:31:29.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:28.999+0000 7fb4fbe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb50419a8e0 0x7fb50419ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:56198/0 (socket says 172.21.15.53:56198) 2024-04-25T14:31:29.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:28.999+0000 7fb4fbe00700 1 -- 172.21.15.53:0/963383449 learned_addr learned my addr 172.21.15.53:0/963383449 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:31:29.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:28.999+0000 7fb509200700 1 --2- 172.21.15.53:0/963383449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5041035f0 0x7fb50419a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:31:29.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:29.000+0000 7fb4fbe00700 1 -- 172.21.15.53:0/963383449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5041035f0 msgr2=0x7fb50419a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:31:29.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:29.000+0000 7fb4fbe00700 1 --2- 172.21.15.53:0/963383449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5041035f0 0x7fb50419a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:31:29.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:29.000+0000 7fb4fbe00700 1 -- 172.21.15.53:0/963383449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb50419f1b0 msgr2=0x7fb50419f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:31:29.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:29.000+0000 7fb4fbe00700 1 --2- 172.21.15.53:0/963383449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb50419f1b0 0x7fb50419f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:31:29.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:29.000+0000 7fb4fbe00700 1 -- 172.21.15.53:0/963383449 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5041a6f20 con 0x7fb50419a8e0 2024-04-25T14:31:29.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:29.000+0000 7fb509c00700 1 --2- 172.21.15.53:0/963383449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb50419f1b0 0x7fb50419f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:31:29.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:29.000+0000 7fb4fbe00700 1 --2- 172.21.15.53:0/963383449 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb50419a8e0 0x7fb50419ec70 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec00b780 tx=0x7fb4ec00ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:31:29.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:29.000+0000 7fb502000700 1 -- 172.21.15.53:0/963383449 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec016070 con 0x7fb50419a8e0 2024-04-25T14:31:29.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:29.000+0000 7fb502000700 1 -- 172.21.15.53:0/963383449 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4ec0036a0 con 0x7fb50419a8e0 2024-04-25T14:31:29.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:29.000+0000 7fb502000700 1 -- 172.21.15.53:0/963383449 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec0119c0 con 0x7fb50419a8e0 2024-04-25T14:31:29.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:29.001+0000 7fb50bc00700 1 -- 172.21.15.53:0/963383449 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5041a7150 con 0x7fb50419a8e0 2024-04-25T14:31:29.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:29.001+0000 7fb50bc00700 1 -- 172.21.15.53:0/963383449 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5041a76e0 con 0x7fb50419a8e0 2024-04-25T14:31:29.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:29.002+0000 7fb502000700 1 -- 172.21.15.53:0/963383449 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fb4ec019070 con 0x7fb50419a8e0 2024-04-25T14:31:29.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:29.002+0000 7fb50bc00700 1 -- 172.21.15.53:0/963383449 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5040640a0 con 0x7fb50419a8e0 2024-04-25T14:31:29.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:29.002+0000 7fb502000700 1 --2- 172.21.15.53:0/963383449 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fb4e8039ae0 0x7fb4e803bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:31:29.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:29.002+0000 7fb502000700 1 -- 172.21.15.53:0/963383449 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb4ec04b450 con 0x7fb50419a8e0 2024-04-25T14:31:29.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:29.002+0000 7fb509200700 1 --2- 172.21.15.53:0/963383449 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fb4e8039ae0 0x7fb4e803bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:31:29.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:29.003+0000 7fb509200700 1 --2- 172.21.15.53:0/963383449 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fb4e8039ae0 0x7fb4e803bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4006010 tx=0x7fb4f401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:31:29.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:29.008+0000 7fb502000700 1 -- 172.21.15.53:0/963383449 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb4ec01ad70 con 0x7fb50419a8e0 2024-04-25T14:31:29.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:29.270+0000 7fb50bc00700 1 -- 172.21.15.53:0/963383449 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fb50402df80 con 0x7fb50419a8e0 2024-04-25T14:31:29.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:29.270+0000 7fb502000700 1 -- 172.21.15.53:0/963383449 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7fb4ec014120 con 0x7fb50419a8e0 2024-04-25T14:31:29.272 INFO:teuthology.orchestra.run.smithi053.stdout:# minimal ceph.conf for fd30e6ce-030f-11ef-bc93-c7b262605968 2024-04-25T14:31:29.272 INFO:teuthology.orchestra.run.smithi053.stdout:[global] 2024-04-25T14:31:29.272 INFO:teuthology.orchestra.run.smithi053.stdout: fsid = fd30e6ce-030f-11ef-bc93-c7b262605968 2024-04-25T14:31:29.272 INFO:teuthology.orchestra.run.smithi053.stdout: mon_host = [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] 2024-04-25T14:31:29.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:29.273+0000 7fb50bc00700 1 -- 172.21.15.53:0/963383449 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fb4e8039ae0 msgr2=0x7fb4e803bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:31:29.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:29.273+0000 7fb50bc00700 1 --2- 172.21.15.53:0/963383449 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fb4e8039ae0 0x7fb4e803bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4006010 tx=0x7fb4f401a040 comp rx=0 tx=0).stop 2024-04-25T14:31:29.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:29.273+0000 7fb50bc00700 1 -- 172.21.15.53:0/963383449 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb50419a8e0 msgr2=0x7fb50419ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:31:29.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:29.273+0000 7fb50bc00700 1 --2- 172.21.15.53:0/963383449 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb50419a8e0 0x7fb50419ec70 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec00b780 tx=0x7fb4ec00ba90 comp rx=0 tx=0).stop 2024-04-25T14:31:29.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:29.273+0000 7fb50bc00700 1 -- 172.21.15.53:0/963383449 shutdown_connections 2024-04-25T14:31:29.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:29.273+0000 7fb50bc00700 1 --2- 172.21.15.53:0/963383449 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fb4e8039ae0 0x7fb4e803bfa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:31:29.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:29.273+0000 7fb50bc00700 1 --2- 172.21.15.53:0/963383449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb50419f1b0 0x7fb50419f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:31:29.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:29.273+0000 7fb50bc00700 1 --2- 172.21.15.53:0/963383449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5041035f0 0x7fb50419a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:31:29.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:29.274+0000 7fb50bc00700 1 --2- 172.21.15.53:0/963383449 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb50419a8e0 0x7fb50419ec70 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:31:29.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:29.274+0000 7fb50bc00700 1 -- 172.21.15.53:0/963383449 >> 172.21.15.53:0/963383449 conn(0x7fb5040fce40 msgr2=0x7fb5041095e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:31:29.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:29.274+0000 7fb50bc00700 1 -- 172.21.15.53:0/963383449 shutdown_connections 2024-04-25T14:31:29.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:29.274+0000 7fb50bc00700 1 -- 172.21.15.53:0/963383449 wait complete. 2024-04-25T14:31:29.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:29 smithi183 ceph-mon[32584]: Updating smithi053:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:31:29.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:29 smithi183 ceph-mon[32584]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:29.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:29 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:31:29.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:29 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:29.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:29 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:29.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:29 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:29.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:29 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:29.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:29 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:29.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:29 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:29.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:29 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:29.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:29 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:31:29.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:29 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T14:31:29.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:29 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:29.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:29 smithi150 ceph-mon[32504]: Updating smithi053:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:31:29.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:29 smithi150 ceph-mon[32504]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:29.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:29 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:31:29.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:29 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:29.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:29 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:29.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:29 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:29.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:29 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:29.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:29 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:29.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:29 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:29.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:29 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:29.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:29 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:31:29.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:29 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T14:31:29.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:29 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:29.462 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:29 smithi053 ceph-mon[27773]: Updating smithi053:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:31:29.462 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:29 smithi053 ceph-mon[27773]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:29.462 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:29 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:31:29.462 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:29 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:29.462 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:29 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:29.463 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:29 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:29.463 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:29 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:29.463 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:29 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:29.463 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:29 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:29.463 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:29 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:29.463 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:29 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:31:29.463 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:29 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T14:31:29.463 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:29 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:30.022 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-25T14:31:30.023 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-25T14:31:30.023 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T14:31:30.100 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-25T14:31:30.100 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:31:30.207 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T14:31:30.207 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T14:31:30.233 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T14:31:30.234 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:31:30.297 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-25T14:31:30.297 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T14:31:30.331 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-25T14:31:30.332 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:31:30.399 INFO:tasks.cephadm:Adding mgr.x on smithi053 2024-04-25T14:31:30.400 INFO:tasks.cephadm:Adding mgr.y on smithi150 2024-04-25T14:31:30.400 INFO:tasks.cephadm:Adding mgr.z on smithi183 2024-04-25T14:31:30.400 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi053=x;smithi150=y;smithi183=z' 2024-04-25T14:31:30.432 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:30 smithi053 ceph-mon[27773]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T14:31:30.432 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:30 smithi053 ceph-mon[27773]: Reconfiguring daemon mon.a on smithi053 2024-04-25T14:31:30.432 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:30 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:30.433 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:30 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/963383449' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:30.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:30 smithi150 ceph-mon[32504]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T14:31:30.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:30 smithi150 ceph-mon[32504]: Reconfiguring daemon mon.a on smithi053 2024-04-25T14:31:30.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:30 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:30.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:30 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/963383449' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:30.672 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:31:30.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:30 smithi183 ceph-mon[32584]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T14:31:30.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:30 smithi183 ceph-mon[32584]: Reconfiguring daemon mon.a on smithi053 2024-04-25T14:31:30.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:30 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:30.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:30 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/963383449' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:31.326 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:31 smithi053 ceph-mon[27773]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:31.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:31 smithi183 ceph-mon[32584]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:31.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:31 smithi150 ceph-mon[32504]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:32.172 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.170+0000 7f0f459aa700 1 -- 172.21.15.183:0/680435088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f40103bf0 msgr2=0x7f0f40104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:31:32.172 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.170+0000 7f0f459aa700 1 --2- 172.21.15.183:0/680435088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f40103bf0 0x7f0f40104010 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0f30009910 tx=0x7f0f30009c20 comp rx=0 tx=0).stop 2024-04-25T14:31:32.173 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.170+0000 7f0f459aa700 1 -- 172.21.15.183:0/680435088 shutdown_connections 2024-04-25T14:31:32.173 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.170+0000 7f0f459aa700 1 --2- 172.21.15.183:0/680435088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f40103bf0 0x7f0f40104010 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:31:32.173 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.170+0000 7f0f459aa700 1 -- 172.21.15.183:0/680435088 >> 172.21.15.183:0/680435088 conn(0x7f0f400fc1f0 msgr2=0x7f0f400fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:31:32.173 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.171+0000 7f0f459aa700 1 -- 172.21.15.183:0/680435088 shutdown_connections 2024-04-25T14:31:32.173 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.171+0000 7f0f459aa700 1 -- 172.21.15.183:0/680435088 wait complete. 2024-04-25T14:31:32.173 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.171+0000 7f0f459aa700 1 Processor -- start 2024-04-25T14:31:32.173 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.171+0000 7f0f459aa700 1 -- start start 2024-04-25T14:31:32.173 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.171+0000 7f0f459aa700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f40103bf0 0x7f0f401032f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:31:32.173 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.171+0000 7f0f459aa700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f40101940 0x7f0f40101dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:31:32.173 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.172+0000 7f0f459aa700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f40102300 0x7f0f401a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:31:32.173 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.172+0000 7f0f459aa700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0f401084d0 con 0x7f0f40103bf0 2024-04-25T14:31:32.173 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.172+0000 7f0f459aa700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0f40108350 con 0x7f0f40101940 2024-04-25T14:31:32.173 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.172+0000 7f0f459aa700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0f40108650 con 0x7f0f40102300 2024-04-25T14:31:32.174 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.172+0000 7f0f3f7fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f40102300 0x7f0f401a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:31:32.174 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.172+0000 7f0f3f7fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f40102300 0x7f0f401a1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:35246/0 (socket says 172.21.15.183:35246) 2024-04-25T14:31:32.174 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.172+0000 7f0f3f7fe700 1 -- 172.21.15.183:0/312331027 learned_addr learned my addr 172.21.15.183:0/312331027 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-04-25T14:31:32.174 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.172+0000 7f0f3e7fc700 1 --2- 172.21.15.183:0/312331027 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f40101940 0x7f0f40101dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:31:32.174 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.172+0000 7f0f3effd700 1 --2- 172.21.15.183:0/312331027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f40103bf0 0x7f0f401032f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:31:32.174 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.172+0000 7f0f3f7fe700 1 -- 172.21.15.183:0/312331027 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f40101940 msgr2=0x7f0f40101dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:31:32.174 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.172+0000 7f0f3f7fe700 1 --2- 172.21.15.183:0/312331027 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f40101940 0x7f0f40101dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:31:32.174 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.172+0000 7f0f3f7fe700 1 -- 172.21.15.183:0/312331027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f40103bf0 msgr2=0x7f0f401032f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:31:32.174 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.172+0000 7f0f3f7fe700 1 --2- 172.21.15.183:0/312331027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f40103bf0 0x7f0f401032f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:31:32.174 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.172+0000 7f0f3f7fe700 1 -- 172.21.15.183:0/312331027 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f401a2370 con 0x7f0f40102300 2024-04-25T14:31:32.174 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.172+0000 7f0f3effd700 1 --2- 172.21.15.183:0/312331027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f40103bf0 0x7f0f401032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:31:32.174 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.172+0000 7f0f3f7fe700 1 --2- 172.21.15.183:0/312331027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f40102300 0x7f0f401a1db0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0f3800ba50 tx=0x7f0f3800bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:31:32.174 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.172+0000 7f0f449a8700 1 -- 172.21.15.183:0/312331027 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f38019070 con 0x7f0f40102300 2024-04-25T14:31:32.174 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.172+0000 7f0f459aa700 1 -- 172.21.15.183:0/312331027 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f401a2660 con 0x7f0f40102300 2024-04-25T14:31:32.175 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.172+0000 7f0f449a8700 1 -- 172.21.15.183:0/312331027 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f38014440 con 0x7f0f40102300 2024-04-25T14:31:32.175 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.172+0000 7f0f449a8700 1 -- 172.21.15.183:0/312331027 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3800ce60 con 0x7f0f40102300 2024-04-25T14:31:32.175 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.172+0000 7f0f459aa700 1 -- 172.21.15.183:0/312331027 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f401a2c20 con 0x7f0f40102300 2024-04-25T14:31:32.176 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.173+0000 7f0f449a8700 1 -- 172.21.15.183:0/312331027 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f0f3801c070 con 0x7f0f40102300 2024-04-25T14:31:32.176 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.173+0000 7f0f459aa700 1 -- 172.21.15.183:0/312331027 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f400640a0 con 0x7f0f40102300 2024-04-25T14:31:32.176 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.173+0000 7f0f449a8700 1 --2- 172.21.15.183:0/312331027 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f0f28039ae0 0x7f0f2803bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:31:32.176 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.174+0000 7f0f449a8700 1 -- 172.21.15.183:0/312331027 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0f3804be60 con 0x7f0f40102300 2024-04-25T14:31:32.176 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.174+0000 7f0f3effd700 1 --2- 172.21.15.183:0/312331027 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f0f28039ae0 0x7f0f2803bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:31:32.176 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.175+0000 7f0f3effd700 1 --2- 172.21.15.183:0/312331027 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f0f28039ae0 0x7f0f2803bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0f3000b5c0 tx=0x7f0f30005260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:31:32.178 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.179+0000 7f0f449a8700 1 -- 172.21.15.183:0/312331027 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0f380145f0 con 0x7f0f40102300 2024-04-25T14:31:32.388 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.389+0000 7f0f459aa700 1 -- 172.21.15.183:0/312331027 --> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi053=x;smithi150=y;smithi183=z", "target": ["mon-mgr", ""]}) -- 0x7f0f400fe180 con 0x7f0f28039ae0 2024-04-25T14:31:32.402 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.403+0000 7f0f449a8700 1 -- 172.21.15.183:0/312331027 <== mgr.14152 v2:172.21.15.53:6800/4154933108 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0f400fe180 con 0x7f0f28039ae0 2024-04-25T14:31:32.403 INFO:teuthology.orchestra.run.smithi183.stdout:Scheduled mgr update... 2024-04-25T14:31:32.404 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.405+0000 7f0f459aa700 1 -- 172.21.15.183:0/312331027 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f0f28039ae0 msgr2=0x7f0f2803bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:31:32.405 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.405+0000 7f0f459aa700 1 --2- 172.21.15.183:0/312331027 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f0f28039ae0 0x7f0f2803bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0f3000b5c0 tx=0x7f0f30005260 comp rx=0 tx=0).stop 2024-04-25T14:31:32.405 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.405+0000 7f0f459aa700 1 -- 172.21.15.183:0/312331027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f40102300 msgr2=0x7f0f401a1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:31:32.405 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.405+0000 7f0f459aa700 1 --2- 172.21.15.183:0/312331027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f40102300 0x7f0f401a1db0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0f3800ba50 tx=0x7f0f3800bd60 comp rx=0 tx=0).stop 2024-04-25T14:31:32.405 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.405+0000 7f0f459aa700 1 -- 172.21.15.183:0/312331027 shutdown_connections 2024-04-25T14:31:32.405 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.405+0000 7f0f459aa700 1 --2- 172.21.15.183:0/312331027 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f0f28039ae0 0x7f0f2803bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:31:32.405 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.405+0000 7f0f459aa700 1 --2- 172.21.15.183:0/312331027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f40102300 0x7f0f401a1db0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:31:32.405 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.405+0000 7f0f459aa700 1 --2- 172.21.15.183:0/312331027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f40103bf0 0x7f0f401032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:31:32.405 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.405+0000 7f0f459aa700 1 --2- 172.21.15.183:0/312331027 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f40101940 0x7f0f40101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:31:32.405 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.405+0000 7f0f459aa700 1 -- 172.21.15.183:0/312331027 >> 172.21.15.183:0/312331027 conn(0x7f0f400fc1f0 msgr2=0x7f0f400fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:31:32.405 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.405+0000 7f0f459aa700 1 -- 172.21.15.183:0/312331027 shutdown_connections 2024-04-25T14:31:32.405 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:31:32.405+0000 7f0f459aa700 1 -- 172.21.15.183:0/312331027 wait complete. 2024-04-25T14:31:32.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:32 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:32.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:32 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:32.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:32 smithi183 ceph-mon[32584]: Reconfiguring mon.b (monmap changed)... 2024-04-25T14:31:32.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:32 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:31:32.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:32 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T14:31:32.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:32 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:32.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:32 smithi183 ceph-mon[32584]: Reconfiguring daemon mon.b on smithi150 2024-04-25T14:31:32.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:32 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:32.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:32 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:32.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:32 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:32.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:32 smithi150 ceph-mon[32504]: Reconfiguring mon.b (monmap changed)... 2024-04-25T14:31:32.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:32 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:31:32.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:32 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T14:31:32.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:32 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:32.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:32 smithi150 ceph-mon[32504]: Reconfiguring daemon mon.b on smithi150 2024-04-25T14:31:32.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:32 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:32.997 DEBUG:teuthology.orchestra.run.smithi150:mgr.y> sudo journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mgr.y.service 2024-04-25T14:31:33.000 DEBUG:teuthology.orchestra.run.smithi183:mgr.z> sudo journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mgr.z.service 2024-04-25T14:31:33.002 INFO:tasks.cephadm:Deploying OSDs... 2024-04-25T14:31:33.003 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-25T14:31:33.003 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T14:31:33.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:32 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:33.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:32 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:33.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:32 smithi053 ceph-mon[27773]: Reconfiguring mon.b (monmap changed)... 2024-04-25T14:31:33.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:32 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:31:33.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:32 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T14:31:33.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:32 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:33.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:32 smithi053 ceph-mon[27773]: Reconfiguring daemon mon.b on smithi150 2024-04-25T14:31:33.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:32 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:33.064 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T14:31:33.065 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_1 2024-04-25T14:31:33.168 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T14:31:33.168 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:31:33.168 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 6h/6d Inode: 632 Links: 1 2024-04-25T14:31:33.168 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:31:33.168 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:31:33.168 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-04-25 14:30:30.700987254 +0000 2024-04-25T14:31:33.168 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-04-25 14:25:17.419275804 +0000 2024-04-25T14:31:33.168 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-04-25 14:25:17.419275804 +0000 2024-04-25T14:31:33.168 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-04-25 14:25:17.419275804 +0000 2024-04-25T14:31:33.169 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T14:31:33.253 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-04-25T14:31:33.253 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-04-25T14:31:33.253 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000276981 s, 1.8 MB/s 2024-04-25T14:31:33.258 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T14:31:33.314 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_2 2024-04-25T14:31:33.368 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T14:31:33.368 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:31:33.368 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-04-25T14:31:33.368 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:31:33.368 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:31:33.368 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-04-25 14:30:30.786985535 +0000 2024-04-25T14:31:33.369 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-04-25 14:25:18.008264134 +0000 2024-04-25T14:31:33.369 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-04-25 14:25:18.008264134 +0000 2024-04-25T14:31:33.369 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-04-25 14:25:18.008264134 +0000 2024-04-25T14:31:33.369 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T14:31:33.450 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-04-25T14:31:33.451 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-04-25T14:31:33.451 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000264675 s, 1.9 MB/s 2024-04-25T14:31:33.454 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T14:31:33.511 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_3 2024-04-25T14:31:33.566 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T14:31:33.566 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:31:33.566 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-04-25T14:31:33.566 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:31:33.566 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:31:33.567 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-04-25 14:30:30.861984036 +0000 2024-04-25T14:31:33.567 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-04-25 14:25:18.609252227 +0000 2024-04-25T14:31:33.567 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-04-25 14:25:18.609252227 +0000 2024-04-25T14:31:33.567 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-04-25 14:25:18.609252227 +0000 2024-04-25T14:31:33.567 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T14:31:33.651 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-04-25T14:31:33.651 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-04-25T14:31:33.651 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000299427 s, 1.7 MB/s 2024-04-25T14:31:33.655 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T14:31:33.709 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_4 2024-04-25T14:31:33.760 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T14:31:33.760 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:31:33.760 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-04-25T14:31:33.760 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:31:33.761 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:31:33.761 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-04-25 14:30:30.925982757 +0000 2024-04-25T14:31:33.761 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-04-25 14:25:19.221240101 +0000 2024-04-25T14:31:33.761 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-04-25 14:25:19.221240101 +0000 2024-04-25T14:31:33.761 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-04-25 14:25:19.221240101 +0000 2024-04-25T14:31:33.761 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T14:31:33.832 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:33 smithi183 ceph-mon[32584]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:33.833 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:33 smithi183 ceph-mon[32584]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi053=x;smithi150=y;smithi183=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:31:33.833 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:33 smithi183 ceph-mon[32584]: Saving service mgr spec with placement smithi053=x;smithi150=y;smithi183=z;count:3 2024-04-25T14:31:33.833 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:33 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:33.833 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:33 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:33.833 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:33 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:31:33.833 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:33 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T14:31:33.833 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:33 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:33.834 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:33 smithi053 ceph-mon[27773]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:33.835 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:33 smithi053 ceph-mon[27773]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi053=x;smithi150=y;smithi183=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:31:33.835 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:33 smithi053 ceph-mon[27773]: Saving service mgr spec with placement smithi053=x;smithi150=y;smithi183=z;count:3 2024-04-25T14:31:33.835 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:33 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:33.835 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:33 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:33.835 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:33 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:31:33.835 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:33 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T14:31:33.835 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:33 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:33.844 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-04-25T14:31:33.844 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-04-25T14:31:33.844 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000273497 s, 1.9 MB/s 2024-04-25T14:31:33.848 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T14:31:33.907 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T14:31:33.907 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T14:31:33.924 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T14:31:33.924 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_1 2024-04-25T14:31:33.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:33 smithi150 ceph-mon[32504]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:33.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:33 smithi150 ceph-mon[32504]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi053=x;smithi150=y;smithi183=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:31:33.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:33 smithi150 ceph-mon[32504]: Saving service mgr spec with placement smithi053=x;smithi150=y;smithi183=z;count:3 2024-04-25T14:31:33.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:33 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:33.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:33 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:33.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:33 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:31:33.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:33 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T14:31:33.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:33 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:33.976 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T14:31:33.976 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:31:33.976 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-25T14:31:33.976 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:31:33.976 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:31:33.976 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-04-25 14:31:19.636877516 +0000 2024-04-25T14:31:33.976 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-04-25 14:24:01.436186991 +0000 2024-04-25T14:31:33.977 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-04-25 14:24:01.436186991 +0000 2024-04-25T14:31:33.977 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-04-25 14:24:01.436186991 +0000 2024-04-25T14:31:33.977 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T14:31:34.040 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-04-25T14:31:34.040 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-04-25T14:31:34.041 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000238443 s, 2.1 MB/s 2024-04-25T14:31:34.042 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T14:31:34.098 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_2 2024-04-25T14:31:34.154 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T14:31:34.154 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:31:34.154 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-25T14:31:34.154 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:31:34.154 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:31:34.154 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-04-25 14:31:19.636877516 +0000 2024-04-25T14:31:34.155 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-04-25 14:24:01.758180340 +0000 2024-04-25T14:31:34.155 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-04-25 14:24:01.758180340 +0000 2024-04-25T14:31:34.155 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-04-25 14:24:01.758180340 +0000 2024-04-25T14:31:34.155 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T14:31:34.217 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-04-25T14:31:34.217 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-04-25T14:31:34.217 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000232912 s, 2.2 MB/s 2024-04-25T14:31:34.218 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T14:31:34.275 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_3 2024-04-25T14:31:34.331 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T14:31:34.331 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:31:34.331 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-25T14:31:34.331 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:31:34.331 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:31:34.331 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-04-25 14:31:19.636877516 +0000 2024-04-25T14:31:34.331 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-04-25 14:24:02.082173647 +0000 2024-04-25T14:31:34.331 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-04-25 14:24:02.082173647 +0000 2024-04-25T14:31:34.331 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-04-25 14:24:02.082173647 +0000 2024-04-25T14:31:34.332 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T14:31:34.393 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-04-25T14:31:34.393 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-04-25T14:31:34.393 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000216221 s, 2.4 MB/s 2024-04-25T14:31:34.394 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T14:31:34.450 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_4 2024-04-25T14:31:34.505 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T14:31:34.505 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:31:34.506 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-25T14:31:34.506 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:31:34.506 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:31:34.506 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-04-25 14:31:19.637877496 +0000 2024-04-25T14:31:34.506 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-04-25 14:24:02.411166851 +0000 2024-04-25T14:31:34.506 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-04-25 14:24:02.411166851 +0000 2024-04-25T14:31:34.506 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-04-25 14:24:02.411166851 +0000 2024-04-25T14:31:34.506 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T14:31:34.569 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-04-25T14:31:34.569 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-04-25T14:31:34.569 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.00060187 s, 851 kB/s 2024-04-25T14:31:34.570 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T14:31:34.626 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-25T14:31:34.627 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T14:31:34.643 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T14:31:34.643 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_1 2024-04-25T14:31:34.651 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:34 smithi183 ceph-mon[32584]: Reconfiguring mon.c (monmap changed)... 2024-04-25T14:31:34.651 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:34 smithi183 ceph-mon[32584]: Reconfiguring daemon mon.c on smithi183 2024-04-25T14:31:34.667 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T14:31:34.667 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:31:34.667 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-25T14:31:34.667 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:31:34.667 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:31:34.667 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-04-25 14:31:10.395956138 +0000 2024-04-25T14:31:34.667 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-04-25 14:24:01.448826856 +0000 2024-04-25T14:31:34.667 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-04-25 14:24:01.448826856 +0000 2024-04-25T14:31:34.667 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-04-25 14:24:01.448826856 +0000 2024-04-25T14:31:34.668 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T14:31:34.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:34 smithi150 ceph-mon[32504]: Reconfiguring mon.c (monmap changed)... 2024-04-25T14:31:34.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:34 smithi150 ceph-mon[32504]: Reconfiguring daemon mon.c on smithi183 2024-04-25T14:31:34.732 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-04-25T14:31:34.732 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-04-25T14:31:34.732 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000212124 s, 2.4 MB/s 2024-04-25T14:31:34.733 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T14:31:34.789 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_2 2024-04-25T14:31:34.843 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T14:31:34.843 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:31:34.843 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-25T14:31:34.844 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:31:34.844 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:31:34.844 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-04-25 14:31:10.395956138 +0000 2024-04-25T14:31:34.844 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-04-25 14:24:01.782817347 +0000 2024-04-25T14:31:34.844 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-04-25 14:24:01.782817347 +0000 2024-04-25T14:31:34.844 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-04-25 14:24:01.782817347 +0000 2024-04-25T14:31:34.844 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T14:31:34.906 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-04-25T14:31:34.906 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-04-25T14:31:34.906 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000214894 s, 2.4 MB/s 2024-04-25T14:31:34.907 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T14:31:34.962 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_3 2024-04-25T14:31:35.017 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T14:31:35.018 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:31:35.018 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-25T14:31:35.018 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:31:35.018 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:31:35.018 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-04-25 14:31:10.396956110 +0000 2024-04-25T14:31:35.018 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-04-25 14:24:02.104808180 +0000 2024-04-25T14:31:35.018 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-04-25 14:24:02.104808180 +0000 2024-04-25T14:31:35.018 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-04-25 14:24:02.104808180 +0000 2024-04-25T14:31:35.018 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T14:31:35.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:34 smithi053 ceph-mon[27773]: Reconfiguring mon.c (monmap changed)... 2024-04-25T14:31:35.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:34 smithi053 ceph-mon[27773]: Reconfiguring daemon mon.c on smithi183 2024-04-25T14:31:35.080 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-04-25T14:31:35.081 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-04-25T14:31:35.081 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000206926 s, 2.5 MB/s 2024-04-25T14:31:35.082 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T14:31:35.139 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_4 2024-04-25T14:31:35.194 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T14:31:35.194 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:31:35.194 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-25T14:31:35.194 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:31:35.194 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:31:35.194 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-04-25 14:31:10.396956110 +0000 2024-04-25T14:31:35.194 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-04-25 14:24:02.443798529 +0000 2024-04-25T14:31:35.195 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-04-25 14:24:02.443798529 +0000 2024-04-25T14:31:35.195 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-04-25 14:24:02.443798529 +0000 2024-04-25T14:31:35.195 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T14:31:35.267 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-04-25T14:31:35.267 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-04-25T14:31:35.267 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000200314 s, 2.6 MB/s 2024-04-25T14:31:35.268 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T14:31:35.296 INFO:tasks.cephadm:Deploying osd.0 on smithi053 with /dev/vg_nvme/lv_4... 2024-04-25T14:31:35.296 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T14:31:35.752 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:31:35.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:35 smithi183 ceph-mon[32584]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:35.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:35 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:35.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:35 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:35.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:35 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:31:35.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:35 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:35.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:35 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:31:35.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:35 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:35.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:35 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T14:31:35.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:35 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T14:31:35.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:35 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T14:31:35.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:35 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:35.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:35 smithi183 ceph-mon[32584]: Deploying daemon mgr.z on smithi183 2024-04-25T14:31:35.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:35 smithi150 ceph-mon[32504]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:35.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:35 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:35.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:35 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:35.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:35 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:31:35.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:35 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:35.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:35 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:31:35.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:35 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:35.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:35 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T14:31:35.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:35 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T14:31:35.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:35 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T14:31:35.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:35 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:35.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:35 smithi150 ceph-mon[32504]: Deploying daemon mgr.z on smithi183 2024-04-25T14:31:36.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:35 smithi053 ceph-mon[27773]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:36.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:35 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:36.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:35 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:36.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:35 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:31:36.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:35 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:36.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:35 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:31:36.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:35 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:36.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:35 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T14:31:36.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:35 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T14:31:36.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:35 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T14:31:36.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:35 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:36.034 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:35 smithi053 ceph-mon[27773]: Deploying daemon mgr.z on smithi183 2024-04-25T14:31:36.954 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:31:36 smithi183 systemd[1]: Starting Ceph mgr.z for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:31:37.296 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:31:37 smithi183 podman[34439]: 2024-04-25 14:31:37.002301374 +0000 UTC m=+0.109215721 container create 71b0b1274392928cdc0a289efbf34084202f5a30ff113243781c471e4d48d73e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:31:37.296 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:31:37 smithi183 podman[34439]: 2024-04-25 14:31:36.910681966 +0000 UTC m=+0.017596326 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T14:31:37.296 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:31:37 smithi183 podman[34439]: 2024-04-25 14:31:37.196168271 +0000 UTC m=+0.303082624 container init 71b0b1274392928cdc0a289efbf34084202f5a30ff113243781c471e4d48d73e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True) 2024-04-25T14:31:37.297 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:31:37 smithi183 podman[34439]: 2024-04-25 14:31:37.198960024 +0000 UTC m=+0.305874373 container start 71b0b1274392928cdc0a289efbf34084202f5a30ff113243781c471e4d48d73e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-25T14:31:37.297 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:31:37 smithi183 ceph-mgr[34458]: -- 172.21.15.183:0/2674943370 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e61d7025a0 con 0x55e61d6d7000 2024-04-25T14:31:37.297 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:31:37 smithi183 bash[34439]: 71b0b1274392928cdc0a289efbf34084202f5a30ff113243781c471e4d48d73e 2024-04-25T14:31:37.635 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:31:37 smithi183 systemd[1]: Started Ceph mgr.z for fd30e6ce-030f-11ef-bc93-c7b262605968. 2024-04-25T14:31:37.635 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:31:37 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:31:37.349+0000 7f9727598200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:31:37.635 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:31:37 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:31:37.534+0000 7f9727598200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:31:37.635 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:31:37 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:31:37.635+0000 7f9727598200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:31:37.941 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:37 smithi183 ceph-mon[32584]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:37.941 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:37 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:37.941 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:37 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:37.942 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:37 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:37.942 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:37 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T14:31:37.942 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:37 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T14:31:37.942 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:37 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T14:31:37.942 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:37 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:38.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:37 smithi053 ceph-mon[27773]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:38.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:37 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:38.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:37 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:38.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:37 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:38.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:37 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T14:31:38.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:37 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T14:31:38.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:37 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T14:31:38.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:37 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:38.073 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:37 smithi150 ceph-mon[32504]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:38.073 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:37 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:38.074 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:37 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:38.074 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:37 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:38.074 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:37 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T14:31:38.074 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:37 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T14:31:38.074 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:37 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T14:31:38.074 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:37 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:38.204 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:31:37 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:31:37.941+0000 7f9727598200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:31:38.204 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:31:38 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:31:38.026+0000 7f9727598200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:31:38.595 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T14:31:38.595 INFO:teuthology.orchestra.run.smithi053.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T14:31:38.595 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10+0 records in 2024-04-25T14:31:38.595 INFO:teuthology.orchestra.run.smithi053.stderr:10+0 records out 2024-04-25T14:31:38.595 INFO:teuthology.orchestra.run.smithi053.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0344707 s, 304 MB/s 2024-04-25T14:31:38.595 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping successful for: 2024-04-25T14:31:38.932 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:38 smithi053 ceph-mon[27773]: Deploying daemon mgr.y on smithi150 2024-04-25T14:31:39.096 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:38 smithi150 ceph-mon[32504]: Deploying daemon mgr.y on smithi150 2024-04-25T14:31:39.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:38 smithi183 ceph-mon[32584]: Deploying daemon mgr.y on smithi150 2024-04-25T14:31:39.369 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi053:vg_nvme/lv_4 2024-04-25T14:31:39.460 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:31:39 smithi150 systemd[1]: Starting Ceph mgr.y for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:31:39.596 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:31:39 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:31:39.256+0000 7f9727598200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:31:39.596 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:31:39 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:31:39.425+0000 7f9727598200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:31:39.596 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:31:39 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:31:39.596+0000 7f9727598200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:31:39.801 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:39 smithi150 ceph-mon[32504]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:39.801 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:31:39 smithi150 podman[33864]: 2024-04-25 14:31:39.421489755 +0000 UTC m=+0.018116921 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T14:31:39.801 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:31:39 smithi150 podman[33864]: 2024-04-25 14:31:39.55799395 +0000 UTC m=+0.154621147 container create 637d815f7da0dd9a02b553b1676c9e15a6046e303f368cbadecfa5628dc2bf63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416) 2024-04-25T14:31:39.801 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:31:39 smithi150 podman[33864]: 2024-04-25 14:31:39.801908701 +0000 UTC m=+0.398535852 container init 637d815f7da0dd9a02b553b1676c9e15a6046e303f368cbadecfa5628dc2bf63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T14:31:39.802 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:31:39.881 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:39 smithi183 ceph-mon[32584]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:40.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:39 smithi053 ceph-mon[27773]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:40.144 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:31:39 smithi150 podman[33864]: 2024-04-25 14:31:39.804764138 +0000 UTC m=+0.401391291 container start 637d815f7da0dd9a02b553b1676c9e15a6046e303f368cbadecfa5628dc2bf63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-25T14:31:40.145 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:31:39 smithi150 ceph-mgr[33883]: -- 172.21.15.150:0/980935470 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5600549d45a0 con 0x5600549a8c00 2024-04-25T14:31:40.145 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:31:39 smithi150 bash[33864]: 637d815f7da0dd9a02b553b1676c9e15a6046e303f368cbadecfa5628dc2bf63 2024-04-25T14:31:40.145 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:31:39 smithi150 systemd[1]: Started Ceph mgr.y for fd30e6ce-030f-11ef-bc93-c7b262605968. 2024-04-25T14:31:40.145 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:31:39 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:31:39.957+0000 7efe76f3b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:31:40.145 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:31:40 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:31:40.144+0000 7efe76f3b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:31:40.177 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:31:39 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:31:39.881+0000 7f9727598200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:31:40.177 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:31:39 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:31:39.974+0000 7f9727598200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:31:40.177 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:31:40 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:31:40.177+0000 7f9727598200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:31:40.453 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:31:40 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:31:40.373+0000 7f9727598200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:31:40.460 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:31:40 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:31:40.250+0000 7efe76f3b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:31:40.951 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:31:40 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:31:40.577+0000 7efe76f3b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:31:40.951 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:31:40 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:31:40.665+0000 7efe76f3b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:31:41.204 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:31:41 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:31:41.014+0000 7f9727598200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T14:31:41.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:40 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:41.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:40 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:41.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:40 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:41.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:40 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:41.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:40 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:31:41.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:40 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:41.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:40 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:41.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:40 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:41.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:40 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:41.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:40 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:31:41.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:40 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:41.258 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:40 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:41.258 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:40 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:41.258 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:40 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:41.258 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:40 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:31:41.538 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:31:41 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:31:41.443+0000 7f9727598200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:31:41.538 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:31:41 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:31:41.538+0000 7f9727598200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:31:41.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:41.764+0000 7f9859600700 1 -- 172.21.15.53:0/2387274728 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f985410a9f0 msgr2=0x7f985410ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:31:41.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:41.764+0000 7f9859600700 1 --2- 172.21.15.53:0/2387274728 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f985410a9f0 0x7f985410ce60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f984c00b0a0 tx=0x7f984c00b3b0 comp rx=0 tx=0).stop 2024-04-25T14:31:41.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:41.764+0000 7f9859600700 1 -- 172.21.15.53:0/2387274728 shutdown_connections 2024-04-25T14:31:41.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:41.764+0000 7f9859600700 1 --2- 172.21.15.53:0/2387274728 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f985410a9f0 0x7f985410ce60 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:31:41.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:41.764+0000 7f9859600700 1 --2- 172.21.15.53:0/2387274728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98540769f0 0x7f9854074e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:31:41.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:41.764+0000 7f9859600700 1 --2- 172.21.15.53:0/2387274728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9854075420 0x7f98540758a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:31:41.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:41.764+0000 7f9859600700 1 -- 172.21.15.53:0/2387274728 >> 172.21.15.53:0/2387274728 conn(0x7f98540fe090 msgr2=0x7f98541004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:31:41.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:41.764+0000 7f9859600700 1 -- 172.21.15.53:0/2387274728 shutdown_connections 2024-04-25T14:31:41.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:41.766+0000 7f9859600700 1 -- 172.21.15.53:0/2387274728 wait complete. 2024-04-25T14:31:41.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:41.766+0000 7f9859600700 1 Processor -- start 2024-04-25T14:31:41.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:41.766+0000 7f9859600700 1 -- start start 2024-04-25T14:31:41.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:41.767+0000 7f9859600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9854075420 0x7f985419ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:31:41.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:41.767+0000 7f9859600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98540769f0 0x7f985419b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:31:41.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:41.767+0000 7f9859600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f985410a9f0 0x7f98541a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:31:41.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:41.767+0000 7f9859600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9854110790 con 0x7f9854075420 2024-04-25T14:31:41.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:41.767+0000 7f9859600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9854110610 con 0x7f985410a9f0 2024-04-25T14:31:41.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:41.767+0000 7f9859600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9854110910 con 0x7f98540769f0 2024-04-25T14:31:41.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:41.767+0000 7f9852000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98540769f0 0x7f985419b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:31:41.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:41.767+0000 7f9852a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9854075420 0x7f985419ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:31:41.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:41.767+0000 7f9852000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98540769f0 0x7f985419b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:44364/0 (socket says 172.21.15.53:44364) 2024-04-25T14:31:41.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:41.767+0000 7f9852000700 1 -- 172.21.15.53:0/993097562 learned_addr learned my addr 172.21.15.53:0/993097562 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:31:41.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:41.767+0000 7f9853400700 1 --2- 172.21.15.53:0/993097562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f985410a9f0 0x7f98541a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:31:41.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:41.767+0000 7f9852000700 1 -- 172.21.15.53:0/993097562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f985410a9f0 msgr2=0x7f98541a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:31:41.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:41.767+0000 7f9852000700 1 --2- 172.21.15.53:0/993097562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f985410a9f0 0x7f98541a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:31:41.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:41.767+0000 7f9852000700 1 -- 172.21.15.53:0/993097562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9854075420 msgr2=0x7f985419ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:31:41.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:41.767+0000 7f9852000700 1 --2- 172.21.15.53:0/993097562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9854075420 0x7f985419ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:31:41.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:41.767+0000 7f9852000700 1 -- 172.21.15.53:0/993097562 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98541a29b0 con 0x7f98540769f0 2024-04-25T14:31:41.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:41.768+0000 7f9852000700 1 --2- 172.21.15.53:0/993097562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98540769f0 0x7f985419b290 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f984000e9d0 tx=0x7f984000ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:31:41.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:41.768+0000 7f984b400700 1 -- 172.21.15.53:0/993097562 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984000cbd0 con 0x7f98540769f0 2024-04-25T14:31:41.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:41.768+0000 7f984b400700 1 -- 172.21.15.53:0/993097562 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f984000ee70 con 0x7f98540769f0 2024-04-25T14:31:41.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:41.768+0000 7f9859600700 1 -- 172.21.15.53:0/993097562 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98541a2ca0 con 0x7f98540769f0 2024-04-25T14:31:41.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:41.768+0000 7f984b400700 1 -- 172.21.15.53:0/993097562 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9840018680 con 0x7f98540769f0 2024-04-25T14:31:41.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:41.768+0000 7f9859600700 1 -- 172.21.15.53:0/993097562 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98541a31d0 con 0x7f98540769f0 2024-04-25T14:31:41.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:41.769+0000 7f9859600700 1 -- 172.21.15.53:0/993097562 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9854069730 con 0x7f98540769f0 2024-04-25T14:31:41.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:41.770+0000 7f984b400700 1 -- 172.21.15.53:0/993097562 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f9840018820 con 0x7f98540769f0 2024-04-25T14:31:41.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:41.770+0000 7f984b400700 1 --2- 172.21.15.53:0/993097562 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f9838039890 0x7f983803bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:31:41.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:41.770+0000 7f984b400700 1 -- 172.21.15.53:0/993097562 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f984004ce10 con 0x7f98540769f0 2024-04-25T14:31:41.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:41.770+0000 7f9852a00700 1 --2- 172.21.15.53:0/993097562 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f9838039890 0x7f983803bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:31:41.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:41.771+0000 7f9852a00700 1 --2- 172.21.15.53:0/993097562 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f9838039890 0x7f983803bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9844009c60 tx=0x7f9844009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:31:41.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:41.780+0000 7f984b400700 1 -- 172.21.15.53:0/993097562 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9840016330 con 0x7f98540769f0 2024-04-25T14:31:41.953 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:31:41 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:31:41.765+0000 7f9727598200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T14:31:42.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:42.029+0000 7f9859600700 1 -- 172.21.15.53:0/993097562 --> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f9854062e80 con 0x7f9838039890 2024-04-25T14:31:42.210 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:31:41 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:31:41.944+0000 7efe76f3b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:31:42.210 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:31:42 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:31:42.116+0000 7efe76f3b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:31:42.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:41 smithi150 ceph-mon[32504]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:42.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:41 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:42.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:41 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:42.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:41 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:42.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:41 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:42.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:41 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:42.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:41 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:31:42.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:41 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:42.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:41 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T14:31:42.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:41 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T14:31:42.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:41 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:42.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:41 smithi053 ceph-mon[27773]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:42.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:41 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:42.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:41 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:42.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:41 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:42.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:41 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:42.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:41 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:42.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:41 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:31:42.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:41 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:42.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:41 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T14:31:42.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:41 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T14:31:42.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:41 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:42.294 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:41 smithi183 ceph-mon[32584]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:42.294 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:41 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:42.294 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:41 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:42.294 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:41 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:42.294 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:41 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:42.294 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:41 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:42.294 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:41 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:31:42.294 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:41 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:42.295 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:41 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T14:31:42.295 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:41 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T14:31:42.295 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:41 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:42.578 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:31:42 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:31:42.288+0000 7efe76f3b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:31:42.578 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:31:42 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:31:42.578+0000 7efe76f3b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:31:42.623 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:31:42 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:31:42.294+0000 7f9727598200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:31:42.624 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:31:42 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:31:42.385+0000 7f9727598200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:31:42.874 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:31:42 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:31:42.670+0000 7efe76f3b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:31:42.874 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:31:42 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:31:42.874+0000 7efe76f3b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:31:42.953 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:31:42 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:31:42.623+0000 7f9727598200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:31:42.954 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:31:42 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:31:42.756+0000 7f9727598200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:31:42.954 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:31:42 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:31:42.852+0000 7f9727598200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:31:43.115 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:42 smithi053 ceph-mon[27773]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T14:31:43.115 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:42 smithi053 ceph-mon[27773]: Reconfiguring daemon mgr.x on smithi053 2024-04-25T14:31:43.115 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:42 smithi053 ceph-mon[27773]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:31:43.116 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:42 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:31:43.116 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:42 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:31:43.116 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:42 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:43.210 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:31:43 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:31:43.067+0000 7efe76f3b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:31:43.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:42 smithi150 ceph-mon[32504]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T14:31:43.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:42 smithi150 ceph-mon[32504]: Reconfiguring daemon mgr.x on smithi053 2024-04-25T14:31:43.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:42 smithi150 ceph-mon[32504]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:31:43.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:42 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:31:43.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:42 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:31:43.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:42 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:43.286 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:31:42 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:31:42.968+0000 7f9727598200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:31:43.286 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:31:43 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:31:43.286+0000 7f9727598200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:31:43.287 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:42 smithi183 ceph-mon[32584]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T14:31:43.287 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:42 smithi183 ceph-mon[32584]: Reconfiguring daemon mgr.x on smithi053 2024-04-25T14:31:43.287 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:42 smithi183 ceph-mon[32584]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:31:43.287 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:42 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:31:43.287 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:42 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:31:43.287 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:42 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:43.960 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:31:43 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:31:43.667+0000 7efe76f3b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T14:31:44.187 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:43 smithi053 ceph-mon[27773]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:44.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:43 smithi183 ceph-mon[32584]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:44.204 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:31:43 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:31:43.880+0000 7f9727598200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:31:44.205 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:31:44 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:31:44.101+0000 7f9727598200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:31:44.396 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:43 smithi150 ceph-mon[32504]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:44.397 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:31:44 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:31:44.094+0000 7efe76f3b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:31:44.397 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:31:44 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:31:44.190+0000 7efe76f3b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:31:44.397 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:31:44 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:31:44.396+0000 7efe76f3b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T14:31:45.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:45.052+0000 7f984b400700 1 -- 172.21.15.53:0/993097562 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7f98400178f0 con 0x7f98540769f0 2024-04-25T14:31:45.210 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:31:44 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:31:44.918+0000 7efe76f3b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:31:45.210 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:31:45 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:31:45.011+0000 7efe76f3b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:31:45.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:45 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:45.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:45 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:45.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:45 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:31:45.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:45 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:45.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:45 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:31:45.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:45 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:45.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:45 smithi150 ceph-mon[32504]: Standby manager daemon z started 2024-04-25T14:31:45.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:45 smithi150 ceph-mon[32504]: from='mgr.? 172.21.15.183:0/2732231839' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T14:31:45.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:45 smithi150 ceph-mon[32504]: from='mgr.? 172.21.15.183:0/2732231839' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T14:31:45.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:45 smithi150 ceph-mon[32504]: from='mgr.? 172.21.15.183:0/2732231839' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T14:31:45.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:45 smithi150 ceph-mon[32504]: from='mgr.? 172.21.15.183:0/2732231839' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T14:31:45.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:45 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:45.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:45 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:45.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:45 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:45.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:45 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:31:45.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:45 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:45.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:45 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:31:45.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:45 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:45.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:45 smithi053 ceph-mon[27773]: Standby manager daemon z started 2024-04-25T14:31:45.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:45 smithi053 ceph-mon[27773]: from='mgr.? 172.21.15.183:0/2732231839' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T14:31:45.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:45 smithi053 ceph-mon[27773]: from='mgr.? 172.21.15.183:0/2732231839' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T14:31:45.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:45 smithi053 ceph-mon[27773]: from='mgr.? 172.21.15.183:0/2732231839' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T14:31:45.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:45 smithi053 ceph-mon[27773]: from='mgr.? 172.21.15.183:0/2732231839' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T14:31:45.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:45 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:45.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:45 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:45.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:45 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:45.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:45 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:31:45.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:45 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:45.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:45 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:31:45.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:45 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:45.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:45 smithi183 ceph-mon[32584]: Standby manager daemon z started 2024-04-25T14:31:45.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:45 smithi183 ceph-mon[32584]: from='mgr.? 172.21.15.183:0/2732231839' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T14:31:45.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:45 smithi183 ceph-mon[32584]: from='mgr.? 172.21.15.183:0/2732231839' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T14:31:45.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:45 smithi183 ceph-mon[32584]: from='mgr.? 172.21.15.183:0/2732231839' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T14:31:45.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:45 smithi183 ceph-mon[32584]: from='mgr.? 172.21.15.183:0/2732231839' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T14:31:45.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:45 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:31:45.603 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:31:45 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:31:45.260+0000 7efe76f3b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:31:45.604 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:31:45 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:31:45.393+0000 7efe76f3b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:31:45.604 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:31:45 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:31:45.487+0000 7efe76f3b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:31:45.604 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:31:45 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:31:45.603+0000 7efe76f3b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:31:46.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:46 smithi150 ceph-mon[32504]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:46.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:46 smithi150 ceph-mon[32504]: mgrmap e14: x(active, since 90s), standbys: z 2024-04-25T14:31:46.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:46 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T14:31:46.211 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:31:45 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:31:45.905+0000 7efe76f3b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:31:46.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:46 smithi053 ceph-mon[27773]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:46.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:46 smithi053 ceph-mon[27773]: mgrmap e14: x(active, since 90s), standbys: z 2024-04-25T14:31:46.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:46 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T14:31:46.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:46 smithi183 ceph-mon[32584]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:46.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:46 smithi183 ceph-mon[32584]: mgrmap e14: x(active, since 90s), standbys: z 2024-04-25T14:31:46.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:46 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T14:31:46.960 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:31:46 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:31:46.510+0000 7efe76f3b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:31:46.960 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:31:46 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:31:46.725+0000 7efe76f3b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:31:47.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:31:47.022+0000 7f984b400700 1 -- 172.21.15.53:0/993097562 <== mon.1 v2:172.21.15.183:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f98400173d0 con 0x7f98540769f0 2024-04-25T14:31:47.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:47 smithi053 ceph-mon[27773]: Standby manager daemon y started 2024-04-25T14:31:47.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:47 smithi053 ceph-mon[27773]: from='mgr.? 172.21.15.150:0/180087421' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T14:31:47.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:47 smithi053 ceph-mon[27773]: from='mgr.? 172.21.15.150:0/180087421' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T14:31:47.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:47 smithi053 ceph-mon[27773]: from='mgr.? 172.21.15.150:0/180087421' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T14:31:47.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:47 smithi053 ceph-mon[27773]: from='mgr.? 172.21.15.150:0/180087421' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T14:31:47.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:47 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/3100305736' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aecfbaf0-39da-4749-a49b-c7205dabd4d8"}]: dispatch 2024-04-25T14:31:47.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:47 smithi053 ceph-mon[27773]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aecfbaf0-39da-4749-a49b-c7205dabd4d8"}]: dispatch 2024-04-25T14:31:47.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:47 smithi183 ceph-mon[32584]: Standby manager daemon y started 2024-04-25T14:31:47.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:47 smithi183 ceph-mon[32584]: from='mgr.? 172.21.15.150:0/180087421' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T14:31:47.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:47 smithi183 ceph-mon[32584]: from='mgr.? 172.21.15.150:0/180087421' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T14:31:47.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:47 smithi183 ceph-mon[32584]: from='mgr.? 172.21.15.150:0/180087421' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T14:31:47.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:47 smithi183 ceph-mon[32584]: from='mgr.? 172.21.15.150:0/180087421' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T14:31:47.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:47 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/3100305736' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aecfbaf0-39da-4749-a49b-c7205dabd4d8"}]: dispatch 2024-04-25T14:31:47.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:47 smithi183 ceph-mon[32584]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aecfbaf0-39da-4749-a49b-c7205dabd4d8"}]: dispatch 2024-04-25T14:31:47.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:47 smithi150 ceph-mon[32504]: Standby manager daemon y started 2024-04-25T14:31:47.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:47 smithi150 ceph-mon[32504]: from='mgr.? 172.21.15.150:0/180087421' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T14:31:47.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:47 smithi150 ceph-mon[32504]: from='mgr.? 172.21.15.150:0/180087421' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T14:31:47.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:47 smithi150 ceph-mon[32504]: from='mgr.? 172.21.15.150:0/180087421' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T14:31:47.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:47 smithi150 ceph-mon[32504]: from='mgr.? 172.21.15.150:0/180087421' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T14:31:47.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:47 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/3100305736' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aecfbaf0-39da-4749-a49b-c7205dabd4d8"}]: dispatch 2024-04-25T14:31:47.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:47 smithi150 ceph-mon[32504]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aecfbaf0-39da-4749-a49b-c7205dabd4d8"}]: dispatch 2024-04-25T14:31:48.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:48 smithi053 ceph-mon[27773]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:48.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:48 smithi053 ceph-mon[27773]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aecfbaf0-39da-4749-a49b-c7205dabd4d8"}]': finished 2024-04-25T14:31:48.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:48 smithi053 ceph-mon[27773]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T14:31:48.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:48 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:31:48.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:48 smithi053 ceph-mon[27773]: mgrmap e15: x(active, since 92s), standbys: z, y 2024-04-25T14:31:48.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:48 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T14:31:48.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:48 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1940390774' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:31:48.359 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:48 smithi183 ceph-mon[32584]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:48.359 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:48 smithi183 ceph-mon[32584]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aecfbaf0-39da-4749-a49b-c7205dabd4d8"}]': finished 2024-04-25T14:31:48.359 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:48 smithi183 ceph-mon[32584]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T14:31:48.359 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:48 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:31:48.359 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:48 smithi183 ceph-mon[32584]: mgrmap e15: x(active, since 92s), standbys: z, y 2024-04-25T14:31:48.359 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:48 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T14:31:48.359 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:48 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/1940390774' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:31:48.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:48 smithi150 ceph-mon[32504]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:48.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:48 smithi150 ceph-mon[32504]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aecfbaf0-39da-4749-a49b-c7205dabd4d8"}]': finished 2024-04-25T14:31:48.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:48 smithi150 ceph-mon[32504]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T14:31:48.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:48 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:31:48.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:48 smithi150 ceph-mon[32504]: mgrmap e15: x(active, since 92s), standbys: z, y 2024-04-25T14:31:48.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:48 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T14:31:48.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:48 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/1940390774' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:31:50.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:50 smithi053 ceph-mon[27773]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:50.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:50 smithi183 ceph-mon[32584]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:50.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:50 smithi150 ceph-mon[32504]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:52.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:52 smithi053 ceph-mon[27773]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:52.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:52 smithi183 ceph-mon[32584]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:52.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:52 smithi150 ceph-mon[32504]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:54.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:54 smithi053 ceph-mon[27773]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:54.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:54 smithi183 ceph-mon[32584]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:54.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:54 smithi150 ceph-mon[32504]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:56.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:56 smithi183 ceph-mon[32584]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:56.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:56 smithi150 ceph-mon[32504]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:56.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:56 smithi053 ceph-mon[27773]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:57.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:57 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T14:31:57.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:57 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:57.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:57 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T14:31:57.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:57 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:57.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:57 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T14:31:57.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:57 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:31:58.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:58 smithi053 ceph-mon[27773]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:58.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:58 smithi053 ceph-mon[27773]: Deploying daemon osd.0 on smithi053 2024-04-25T14:31:58.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:58 smithi183 ceph-mon[32584]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:58.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:58 smithi183 ceph-mon[32584]: Deploying daemon osd.0 on smithi053 2024-04-25T14:31:58.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:58 smithi150 ceph-mon[32504]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:58.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:58 smithi150 ceph-mon[32504]: Deploying daemon osd.0 on smithi053 2024-04-25T14:31:59.371 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:31:59 smithi053 ceph-mon[27773]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:59.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:31:59 smithi183 ceph-mon[32584]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:31:59.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:31:59 smithi150 ceph-mon[32504]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:32:01.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:01 smithi053 ceph-mon[27773]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:32:01.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:01 smithi183 ceph-mon[32584]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:32:01.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:01 smithi150 ceph-mon[32504]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:32:03.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:03 smithi053 ceph-mon[27773]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:32:03.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:03 smithi183 ceph-mon[32584]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:32:03.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:03 smithi150 ceph-mon[32504]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:32:06.142 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:05 smithi053 ceph-mon[27773]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:32:06.142 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:05 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:06.142 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:05 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:32:06.142 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:05 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:06.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:05 smithi183 ceph-mon[32584]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:32:06.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:05 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:06.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:05 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:32:06.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:05 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:06.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:05 smithi150 ceph-mon[32504]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:32:06.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:05 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:06.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:05 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:32:06.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:05 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:07.970 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:07 smithi053 ceph-mon[27773]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:32:07.971 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:07 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:07.971 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:07 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:07.971 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:07 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:32:07.971 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:07 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:32:07.971 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:07 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:08.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:07 smithi183 ceph-mon[32584]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:32:08.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:07 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:08.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:07 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:08.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:07 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:32:08.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:07 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:32:08.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:07 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:08.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:07 smithi150 ceph-mon[32504]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:32:08.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:07 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:08.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:07 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:08.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:07 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:32:08.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:07 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:32:08.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:07 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:09.123 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:08 smithi053 ceph-mon[27773]: from='osd.0 [v2:172.21.15.53:6802/464176015,v1:172.21.15.53:6803/464176015]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T14:32:09.123 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:08 smithi053 ceph-mon[27773]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T14:32:09.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:08 smithi183 ceph-mon[32584]: from='osd.0 [v2:172.21.15.53:6802/464176015,v1:172.21.15.53:6803/464176015]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T14:32:09.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:08 smithi183 ceph-mon[32584]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T14:32:09.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:08 smithi150 ceph-mon[32504]: from='osd.0 [v2:172.21.15.53:6802/464176015,v1:172.21.15.53:6803/464176015]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T14:32:09.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:08 smithi150 ceph-mon[32504]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T14:32:09.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:32:09.579+0000 7f984b400700 1 -- 172.21.15.53:0/993097562 <== mgr.14152 v2:172.21.15.53:6800/4154933108 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9854062e80 con 0x7f9838039890 2024-04-25T14:32:09.580 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 0 on host 'smithi053' 2024-04-25T14:32:09.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:32:09.583+0000 7f9859600700 1 -- 172.21.15.53:0/993097562 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f9838039890 msgr2=0x7f983803bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:32:09.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:32:09.583+0000 7f9859600700 1 --2- 172.21.15.53:0/993097562 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f9838039890 0x7f983803bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9844009c60 tx=0x7f9844009260 comp rx=0 tx=0).stop 2024-04-25T14:32:09.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:32:09.583+0000 7f9859600700 1 -- 172.21.15.53:0/993097562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98540769f0 msgr2=0x7f985419b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:32:09.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:32:09.583+0000 7f9859600700 1 --2- 172.21.15.53:0/993097562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98540769f0 0x7f985419b290 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f984000e9d0 tx=0x7f984000ece0 comp rx=0 tx=0).stop 2024-04-25T14:32:09.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:32:09.583+0000 7f9859600700 1 -- 172.21.15.53:0/993097562 shutdown_connections 2024-04-25T14:32:09.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:32:09.583+0000 7f9859600700 1 --2- 172.21.15.53:0/993097562 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f9838039890 0x7f983803bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:32:09.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:32:09.583+0000 7f9859600700 1 --2- 172.21.15.53:0/993097562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f985410a9f0 0x7f98541a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:32:09.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:32:09.583+0000 7f9859600700 1 --2- 172.21.15.53:0/993097562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9854075420 0x7f985419ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:32:09.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:32:09.583+0000 7f9859600700 1 --2- 172.21.15.53:0/993097562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98540769f0 0x7f985419b290 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:32:09.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:32:09.583+0000 7f9859600700 1 -- 172.21.15.53:0/993097562 >> 172.21.15.53:0/993097562 conn(0x7f98540fe090 msgr2=0x7f9854106940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:32:09.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:32:09.583+0000 7f9859600700 1 -- 172.21.15.53:0/993097562 shutdown_connections 2024-04-25T14:32:09.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:32:09.584+0000 7f9859600700 1 -- 172.21.15.53:0/993097562 wait complete. 2024-04-25T14:32:10.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:09 smithi183 ceph-mon[32584]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:32:10.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:09 smithi183 ceph-mon[32584]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T14:32:10.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:09 smithi183 ceph-mon[32584]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T14:32:10.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:09 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:32:10.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:09 smithi183 ceph-mon[32584]: from='osd.0 [v2:172.21.15.53:6802/464176015,v1:172.21.15.53:6803/464176015]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-04-25T14:32:10.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:09 smithi183 ceph-mon[32584]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-04-25T14:32:10.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:09 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:10.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:09 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:10.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:09 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:32:10.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:09 smithi150 ceph-mon[32504]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:32:10.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:09 smithi150 ceph-mon[32504]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T14:32:10.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:09 smithi150 ceph-mon[32504]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T14:32:10.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:09 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:32:10.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:09 smithi150 ceph-mon[32504]: from='osd.0 [v2:172.21.15.53:6802/464176015,v1:172.21.15.53:6803/464176015]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-04-25T14:32:10.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:09 smithi150 ceph-mon[32504]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-04-25T14:32:10.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:09 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:10.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:09 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:10.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:09 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:32:10.281 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:09 smithi053 ceph-mon[27773]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:32:10.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:09 smithi053 ceph-mon[27773]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T14:32:10.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:09 smithi053 ceph-mon[27773]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T14:32:10.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:09 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:32:10.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:09 smithi053 ceph-mon[27773]: from='osd.0 [v2:172.21.15.53:6802/464176015,v1:172.21.15.53:6803/464176015]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-04-25T14:32:10.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:09 smithi053 ceph-mon[27773]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-04-25T14:32:10.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:09 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:10.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:09 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:10.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:09 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:32:10.340 DEBUG:teuthology.orchestra.run.smithi053:osd.0> sudo journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@osd.0.service 2024-04-25T14:32:10.345 INFO:tasks.cephadm:Deploying osd.1 on smithi150 with /dev/vg_nvme/lv_4... 2024-04-25T14:32:10.345 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T14:32:10.586 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:32:10.951 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:10 smithi150 ceph-mon[32504]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-04-25T14:32:10.952 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:10 smithi150 ceph-mon[32504]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T14:32:10.952 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:10 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:32:10.952 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:10 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:32:11.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:10 smithi183 ceph-mon[32584]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-04-25T14:32:11.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:10 smithi183 ceph-mon[32584]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T14:32:11.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:10 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:32:11.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:10 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:32:11.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:10 smithi053 ceph-mon[27773]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-04-25T14:32:11.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:10 smithi053 ceph-mon[27773]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T14:32:11.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:10 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:32:11.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:10 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:32:11.284 INFO:journalctl@ceph.osd.0.smithi053.stdout:Apr 25 14:32:10 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-0[42448]: 2024-04-25T14:32:10.916+0000 7fa1a7400700 -1 osd.0 0 waiting for initial osdmap 2024-04-25T14:32:11.284 INFO:journalctl@ceph.osd.0.smithi053.stdout:Apr 25 14:32:10 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-0[42448]: 2024-04-25T14:32:10.934+0000 7fa19d200700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:32:11.962 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:11 smithi053 ceph-mon[27773]: purged_snaps scrub starts 2024-04-25T14:32:11.963 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:11 smithi053 ceph-mon[27773]: purged_snaps scrub ok 2024-04-25T14:32:11.963 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:11 smithi053 ceph-mon[27773]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:32:11.963 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:11 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:32:11.963 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:11 smithi053 ceph-mon[27773]: from='osd.0 ' entity='osd.0' 2024-04-25T14:32:12.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:11 smithi183 ceph-mon[32584]: purged_snaps scrub starts 2024-04-25T14:32:12.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:11 smithi183 ceph-mon[32584]: purged_snaps scrub ok 2024-04-25T14:32:12.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:11 smithi183 ceph-mon[32584]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:32:12.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:32:12.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:11 smithi183 ceph-mon[32584]: from='osd.0 ' entity='osd.0' 2024-04-25T14:32:12.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:11 smithi150 ceph-mon[32504]: purged_snaps scrub starts 2024-04-25T14:32:12.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:11 smithi150 ceph-mon[32504]: purged_snaps scrub ok 2024-04-25T14:32:12.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:11 smithi150 ceph-mon[32504]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:32:12.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:11 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:32:12.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:11 smithi150 ceph-mon[32504]: from='osd.0 ' entity='osd.0' 2024-04-25T14:32:12.397 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T14:32:12.397 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T14:32:12.397 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-04-25T14:32:12.397 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-04-25T14:32:12.397 INFO:teuthology.orchestra.run.smithi150.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0129941 s, 807 MB/s 2024-04-25T14:32:12.397 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-04-25T14:32:12.947 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_4 2024-04-25T14:32:13.186 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:32:13.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:12 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:32:13.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:12 smithi183 ceph-mon[32584]: osd.0 [v2:172.21.15.53:6802/464176015,v1:172.21.15.53:6803/464176015] boot 2024-04-25T14:32:13.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:12 smithi183 ceph-mon[32584]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T14:32:13.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:12 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:32:13.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:12 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:32:13.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:12 smithi150 ceph-mon[32504]: osd.0 [v2:172.21.15.53:6802/464176015,v1:172.21.15.53:6803/464176015] boot 2024-04-25T14:32:13.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:12 smithi150 ceph-mon[32504]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T14:32:13.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:12 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:32:13.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:12 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:32:13.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:12 smithi053 ceph-mon[27773]: osd.0 [v2:172.21.15.53:6802/464176015,v1:172.21.15.53:6803/464176015] boot 2024-04-25T14:32:13.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:12 smithi053 ceph-mon[27773]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T14:32:13.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:12 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:32:14.191 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:13 smithi150 ceph-mon[32504]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:32:14.191 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:13 smithi150 ceph-mon[32504]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T14:32:14.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:13 smithi183 ceph-mon[32584]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:32:14.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:13 smithi183 ceph-mon[32584]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T14:32:14.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:13 smithi053 ceph-mon[27773]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:32:14.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:13 smithi053 ceph-mon[27773]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T14:32:14.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:14.700+0000 7f34fd5b1700 1 -- 172.21.15.150:0/3309410318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34f8103550 msgr2=0x7f34f8109e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:32:14.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:14.700+0000 7f34fd5b1700 1 --2- 172.21.15.150:0/3309410318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34f8103550 0x7f34f8109e00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f34f000b0d0 tx=0x7f34f000b3e0 comp rx=0 tx=0).stop 2024-04-25T14:32:14.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:14.700+0000 7f34fd5b1700 1 -- 172.21.15.150:0/3309410318 shutdown_connections 2024-04-25T14:32:14.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:14.700+0000 7f34fd5b1700 1 --2- 172.21.15.150:0/3309410318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34f8103550 0x7f34f8109e00 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:32:14.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:14.700+0000 7f34fd5b1700 1 --2- 172.21.15.150:0/3309410318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34f81019b0 0x7f34f8101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:32:14.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:14.700+0000 7f34fd5b1700 1 --2- 172.21.15.150:0/3309410318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34f8102bb0 0x7f34f8103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:32:14.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:14.700+0000 7f34fd5b1700 1 -- 172.21.15.150:0/3309410318 >> 172.21.15.150:0/3309410318 conn(0x7f34f80fcf70 msgr2=0x7f34f80ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:32:14.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:14.700+0000 7f34fd5b1700 1 -- 172.21.15.150:0/3309410318 shutdown_connections 2024-04-25T14:32:14.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:14.700+0000 7f34fd5b1700 1 -- 172.21.15.150:0/3309410318 wait complete. 2024-04-25T14:32:14.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:14.701+0000 7f34fd5b1700 1 Processor -- start 2024-04-25T14:32:14.704 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:14.701+0000 7f34fd5b1700 1 -- start start 2024-04-25T14:32:14.704 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:14.701+0000 7f34fd5b1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34f81019b0 0x7f34f8197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:32:14.704 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:14.701+0000 7f34fd5b1700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34f8102bb0 0x7f34f81980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:32:14.704 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:14.701+0000 7f34fd5b1700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34f8103550 0x7f34f819f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:32:14.704 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:14.701+0000 7f34fd5b1700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f34f810d510 con 0x7f34f8102bb0 2024-04-25T14:32:14.704 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:14.701+0000 7f34fd5b1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f34f810d390 con 0x7f34f81019b0 2024-04-25T14:32:14.704 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:14.701+0000 7f34fd5b1700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f34f810d690 con 0x7f34f8103550 2024-04-25T14:32:14.704 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:14.701+0000 7f34f6ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34f81019b0 0x7f34f8197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:32:14.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:14.701+0000 7f34f6ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34f81019b0 0x7f34f8197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50574/0 (socket says 172.21.15.150:50574) 2024-04-25T14:32:14.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:14.701+0000 7f34f6ffd700 1 -- 172.21.15.150:0/1272770348 learned_addr learned my addr 172.21.15.150:0/1272770348 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-25T14:32:14.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:14.702+0000 7f34f77fe700 1 --2- 172.21.15.150:0/1272770348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34f8103550 0x7f34f819f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:32:14.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:14.702+0000 7f34f6ffd700 1 -- 172.21.15.150:0/1272770348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34f8103550 msgr2=0x7f34f819f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:32:14.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:14.702+0000 7f34f6ffd700 1 --2- 172.21.15.150:0/1272770348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34f8103550 0x7f34f819f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:32:14.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:14.702+0000 7f34f6ffd700 1 -- 172.21.15.150:0/1272770348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34f8102bb0 msgr2=0x7f34f81980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:32:14.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:14.702+0000 7f34f6ffd700 1 --2- 172.21.15.150:0/1272770348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34f8102bb0 0x7f34f81980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:32:14.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:14.702+0000 7f34f6ffd700 1 -- 172.21.15.150:0/1272770348 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f819f800 con 0x7f34f81019b0 2024-04-25T14:32:14.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:14.702+0000 7f34f6ffd700 1 --2- 172.21.15.150:0/1272770348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34f81019b0 0x7f34f8197ba0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f34e800b500 tx=0x7f34e800b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:32:14.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:14.702+0000 7f34dffff700 1 -- 172.21.15.150:0/1272770348 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e8015020 con 0x7f34f81019b0 2024-04-25T14:32:14.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:14.702+0000 7f34fd5b1700 1 -- 172.21.15.150:0/1272770348 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34f819faf0 con 0x7f34f81019b0 2024-04-25T14:32:14.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:14.702+0000 7f34dffff700 1 -- 172.21.15.150:0/1272770348 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34e8011ce0 con 0x7f34f81019b0 2024-04-25T14:32:14.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:14.702+0000 7f34dffff700 1 -- 172.21.15.150:0/1272770348 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e8010d50 con 0x7f34f81019b0 2024-04-25T14:32:14.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:14.702+0000 7f34fd5b1700 1 -- 172.21.15.150:0/1272770348 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34f81a0030 con 0x7f34f81019b0 2024-04-25T14:32:14.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:14.703+0000 7f34fd5b1700 1 -- 172.21.15.150:0/1272770348 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34f8069730 con 0x7f34f81019b0 2024-04-25T14:32:14.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:14.704+0000 7f34dffff700 1 -- 172.21.15.150:0/1272770348 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f34e8018020 con 0x7f34f81019b0 2024-04-25T14:32:14.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:14.705+0000 7f34dffff700 1 --2- 172.21.15.150:0/1272770348 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f34e00a3bb0 0x7f34e00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:32:14.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:14.705+0000 7f34dffff700 1 -- 172.21.15.150:0/1272770348 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f34e80ce100 con 0x7f34f81019b0 2024-04-25T14:32:14.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:14.705+0000 7f34f67fc700 1 --2- 172.21.15.150:0/1272770348 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f34e00a3bb0 0x7f34e00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:32:14.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:14.706+0000 7f34f67fc700 1 --2- 172.21.15.150:0/1272770348 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f34e00a3bb0 0x7f34e00a6070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f34ec005d90 tx=0x7f34ec005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:32:14.710 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:14.710+0000 7f34dffff700 1 -- 172.21.15.150:0/1272770348 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f34e801f020 con 0x7f34f81019b0 2024-04-25T14:32:14.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:14.919+0000 7f34fd5b1700 1 -- 172.21.15.150:0/1272770348 --> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f34f8062e80 con 0x7f34e00a3bb0 2024-04-25T14:32:15.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:14 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:32:15.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:14 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:32:15.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:14 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:32:16.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:15 smithi183 ceph-mon[32584]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:32:16.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:15 smithi183 ceph-mon[32584]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:32:16.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:15 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:32:16.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:15 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:32:16.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:15 smithi183 ceph-mon[32584]: Detected new or changed devices on smithi053 2024-04-25T14:32:16.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:15 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:16.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:15 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:16.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:15 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:32:16.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:15 smithi183 ceph-mon[32584]: Adjusting osd_memory_target on smithi053 to 17094M 2024-04-25T14:32:16.206 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:15 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:16.206 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:15 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:32:16.206 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:15 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:32:16.206 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:15 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:16.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:15 smithi150 ceph-mon[32504]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:32:16.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:15 smithi150 ceph-mon[32504]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:32:16.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:15 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:32:16.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:15 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:32:16.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:15 smithi150 ceph-mon[32504]: Detected new or changed devices on smithi053 2024-04-25T14:32:16.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:15 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:16.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:15 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:16.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:15 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:32:16.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:15 smithi150 ceph-mon[32504]: Adjusting osd_memory_target on smithi053 to 17094M 2024-04-25T14:32:16.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:15 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:16.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:15 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:32:16.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:15 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:32:16.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:15 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:16.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:15 smithi053 ceph-mon[27773]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:32:16.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:15 smithi053 ceph-mon[27773]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:32:16.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:15 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:32:16.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:15 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:32:16.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:15 smithi053 ceph-mon[27773]: Detected new or changed devices on smithi053 2024-04-25T14:32:16.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:15 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:16.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:15 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:16.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:15 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:32:16.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:15 smithi053 ceph-mon[27773]: Adjusting osd_memory_target on smithi053 to 17094M 2024-04-25T14:32:16.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:15 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:16.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:15 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:32:16.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:15 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:32:16.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:15 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:18.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:17 smithi183 ceph-mon[32584]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:32:18.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:17 smithi183 ceph-mon[32584]: from='client.? 172.21.15.150:0/2959886344' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c4a21ae-c43e-4839-8a56-c3fc63aeb7e5"}]: dispatch 2024-04-25T14:32:18.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:17 smithi183 ceph-mon[32584]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c4a21ae-c43e-4839-8a56-c3fc63aeb7e5"}]: dispatch 2024-04-25T14:32:18.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:17 smithi183 ceph-mon[32584]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c4a21ae-c43e-4839-8a56-c3fc63aeb7e5"}]': finished 2024-04-25T14:32:18.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:17 smithi183 ceph-mon[32584]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T14:32:18.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:17 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:32:18.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:17 smithi150 ceph-mon[32504]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:32:18.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:17 smithi150 ceph-mon[32504]: from='client.? 172.21.15.150:0/2959886344' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c4a21ae-c43e-4839-8a56-c3fc63aeb7e5"}]: dispatch 2024-04-25T14:32:18.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:17 smithi150 ceph-mon[32504]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c4a21ae-c43e-4839-8a56-c3fc63aeb7e5"}]: dispatch 2024-04-25T14:32:18.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:17 smithi150 ceph-mon[32504]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c4a21ae-c43e-4839-8a56-c3fc63aeb7e5"}]': finished 2024-04-25T14:32:18.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:17 smithi150 ceph-mon[32504]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T14:32:18.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:17 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:32:18.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:17 smithi053 ceph-mon[27773]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:32:18.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:17 smithi053 ceph-mon[27773]: from='client.? 172.21.15.150:0/2959886344' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c4a21ae-c43e-4839-8a56-c3fc63aeb7e5"}]: dispatch 2024-04-25T14:32:18.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:17 smithi053 ceph-mon[27773]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c4a21ae-c43e-4839-8a56-c3fc63aeb7e5"}]: dispatch 2024-04-25T14:32:18.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:17 smithi053 ceph-mon[27773]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c4a21ae-c43e-4839-8a56-c3fc63aeb7e5"}]': finished 2024-04-25T14:32:18.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:17 smithi053 ceph-mon[27773]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T14:32:18.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:17 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:32:19.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:18 smithi053 ceph-mon[27773]: from='client.? 172.21.15.150:0/988928597' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:32:19.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:18 smithi183 ceph-mon[32584]: from='client.? 172.21.15.150:0/988928597' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:32:19.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:18 smithi150 ceph-mon[32504]: from='client.? 172.21.15.150:0/988928597' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:32:20.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:19 smithi053 ceph-mon[27773]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:32:20.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:19 smithi183 ceph-mon[32584]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:32:20.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:19 smithi150 ceph-mon[32504]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:32:22.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:21 smithi150 ceph-mon[32504]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:32:22.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:21 smithi053 ceph-mon[27773]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:32:22.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:21 smithi183 ceph-mon[32584]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:32:24.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:23 smithi053 ceph-mon[27773]: pgmap v64: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:32:24.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:23 smithi183 ceph-mon[32584]: pgmap v64: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:32:24.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:23 smithi150 ceph-mon[32504]: pgmap v64: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:32:25.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:24 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T14:32:25.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:24 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:32:25.386 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:24 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T14:32:25.387 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:24 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:32:25.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:24 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T14:32:25.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:24 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:32:26.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:25 smithi053 ceph-mon[27773]: pgmap v65: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:32:26.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:25 smithi053 ceph-mon[27773]: Deploying daemon osd.1 on smithi150 2024-04-25T14:32:26.355 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:25 smithi183 ceph-mon[32584]: pgmap v65: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:32:26.355 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:25 smithi183 ceph-mon[32584]: Deploying daemon osd.1 on smithi150 2024-04-25T14:32:26.364 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:25 smithi150 ceph-mon[32504]: pgmap v65: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:32:26.364 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:25 smithi150 ceph-mon[32504]: Deploying daemon osd.1 on smithi150 2024-04-25T14:32:28.151 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:27 smithi150 ceph-mon[32504]: pgmap v66: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:32:28.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:27 smithi053 ceph-mon[27773]: pgmap v66: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:32:28.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:27 smithi183 ceph-mon[32584]: pgmap v66: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:32:30.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:29 smithi150 ceph-mon[32504]: pgmap v67: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:32:30.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:29 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:30.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:29 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:30.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:29 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:32:30.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:29 smithi053 ceph-mon[27773]: pgmap v67: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:32:30.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:29 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:30.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:29 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:30.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:29 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:32:30.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:29 smithi183 ceph-mon[32584]: pgmap v67: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:32:30.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:29 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:30.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:29 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:30.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:29 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:32:32.272 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:31 smithi150 ceph-mon[32504]: pgmap v68: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:32:32.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:31 smithi053 ceph-mon[27773]: pgmap v68: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:32:32.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:31 smithi183 ceph-mon[32584]: pgmap v68: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:32:32.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:32.958+0000 7f34dffff700 1 -- 172.21.15.150:0/1272770348 <== mgr.14152 v2:172.21.15.53:6800/4154933108 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f34f8062e80 con 0x7f34e00a3bb0 2024-04-25T14:32:32.959 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 1 on host 'smithi150' 2024-04-25T14:32:32.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:32.960+0000 7f34fd5b1700 1 -- 172.21.15.150:0/1272770348 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f34e00a3bb0 msgr2=0x7f34e00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:32:32.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:32.960+0000 7f34fd5b1700 1 --2- 172.21.15.150:0/1272770348 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f34e00a3bb0 0x7f34e00a6070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f34ec005d90 tx=0x7f34ec005d00 comp rx=0 tx=0).stop 2024-04-25T14:32:32.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:32.960+0000 7f34fd5b1700 1 -- 172.21.15.150:0/1272770348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34f81019b0 msgr2=0x7f34f8197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:32:32.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:32.960+0000 7f34fd5b1700 1 --2- 172.21.15.150:0/1272770348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34f81019b0 0x7f34f8197ba0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f34e800b500 tx=0x7f34e800b8c0 comp rx=0 tx=0).stop 2024-04-25T14:32:32.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:32.960+0000 7f34fd5b1700 1 -- 172.21.15.150:0/1272770348 shutdown_connections 2024-04-25T14:32:32.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:32.960+0000 7f34fd5b1700 1 --2- 172.21.15.150:0/1272770348 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f34e00a3bb0 0x7f34e00a6070 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:32:32.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:32.960+0000 7f34fd5b1700 1 --2- 172.21.15.150:0/1272770348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34f8103550 0x7f34f819f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:32:32.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:32.960+0000 7f34fd5b1700 1 --2- 172.21.15.150:0/1272770348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34f81019b0 0x7f34f8197ba0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:32:32.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:32.960+0000 7f34fd5b1700 1 --2- 172.21.15.150:0/1272770348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34f8102bb0 0x7f34f81980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:32:32.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:32.960+0000 7f34fd5b1700 1 -- 172.21.15.150:0/1272770348 >> 172.21.15.150:0/1272770348 conn(0x7f34f80fcf70 msgr2=0x7f34f8105f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:32:32.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:32.960+0000 7f34fd5b1700 1 -- 172.21.15.150:0/1272770348 shutdown_connections 2024-04-25T14:32:32.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:32:32.960+0000 7f34fd5b1700 1 -- 172.21.15.150:0/1272770348 wait complete. 2024-04-25T14:32:33.512 DEBUG:teuthology.orchestra.run.smithi150:osd.1> sudo journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@osd.1.service 2024-04-25T14:32:33.514 INFO:tasks.cephadm:Deploying osd.2 on smithi183 with /dev/vg_nvme/lv_4... 2024-04-25T14:32:33.515 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T14:32:33.758 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:32:34.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:33 smithi183 ceph-mon[32584]: pgmap v69: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:32:34.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:33 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:34.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:33 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:34.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:33 smithi183 ceph-mon[32584]: from='osd.1 [v2:172.21.15.150:6800/1304621982,v1:172.21.15.150:6801/1304621982]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T14:32:34.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:33 smithi183 ceph-mon[32584]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T14:32:34.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:33 smithi150 ceph-mon[32504]: pgmap v69: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:32:34.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:33 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:34.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:33 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:34.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:33 smithi150 ceph-mon[32504]: from='osd.1 [v2:172.21.15.150:6800/1304621982,v1:172.21.15.150:6801/1304621982]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T14:32:34.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:33 smithi150 ceph-mon[32504]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T14:32:34.211 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 14:32:33 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-1[37094]: 2024-04-25T14:32:33.810+0000 7f31783c77c0 -1 osd.1 0 log_to_monitors true 2024-04-25T14:32:34.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:33 smithi053 ceph-mon[27773]: pgmap v69: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:32:34.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:33 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:34.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:33 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:34.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:33 smithi053 ceph-mon[27773]: from='osd.1 [v2:172.21.15.150:6800/1304621982,v1:172.21.15.150:6801/1304621982]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T14:32:34.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:33 smithi053 ceph-mon[27773]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T14:32:35.187 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:34 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:35.187 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:34 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:35.187 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:34 smithi183 ceph-mon[32584]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T14:32:35.187 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:34 smithi183 ceph-mon[32584]: from='osd.1 [v2:172.21.15.150:6800/1304621982,v1:172.21.15.150:6801/1304621982]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T14:32:35.188 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:34 smithi183 ceph-mon[32584]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T14:32:35.188 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:34 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:32:35.188 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:34 smithi183 ceph-mon[32584]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T14:32:35.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:34 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:35.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:34 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:35.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:34 smithi150 ceph-mon[32504]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T14:32:35.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:34 smithi150 ceph-mon[32504]: from='osd.1 [v2:172.21.15.150:6800/1304621982,v1:172.21.15.150:6801/1304621982]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T14:32:35.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:34 smithi150 ceph-mon[32504]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T14:32:35.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:34 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:32:35.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:34 smithi150 ceph-mon[32504]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T14:32:35.211 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 14:32:35 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-1[37094]: 2024-04-25T14:32:35.017+0000 7f316da2c700 -1 osd.1 0 waiting for initial osdmap 2024-04-25T14:32:35.211 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 14:32:35 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-1[37094]: 2024-04-25T14:32:35.020+0000 7f316981f700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:32:35.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:34 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:35.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:34 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:35.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:34 smithi053 ceph-mon[27773]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T14:32:35.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:34 smithi053 ceph-mon[27773]: from='osd.1 [v2:172.21.15.150:6800/1304621982,v1:172.21.15.150:6801/1304621982]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T14:32:35.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:34 smithi053 ceph-mon[27773]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T14:32:35.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:34 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:32:35.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:34 smithi053 ceph-mon[27773]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T14:32:35.736 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T14:32:35.736 INFO:teuthology.orchestra.run.smithi183.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T14:32:35.736 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10+0 records in 2024-04-25T14:32:35.736 INFO:teuthology.orchestra.run.smithi183.stderr:10+0 records out 2024-04-25T14:32:35.736 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113298 s, 925 MB/s 2024-04-25T14:32:35.736 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping successful for: 2024-04-25T14:32:36.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:36 smithi053 ceph-mon[27773]: pgmap v71: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:32:36.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:36 smithi053 ceph-mon[27773]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-25T14:32:36.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:36 smithi053 ceph-mon[27773]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T14:32:36.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:36 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:32:36.314 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:36 smithi183 ceph-mon[32584]: pgmap v71: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:32:36.314 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:36 smithi183 ceph-mon[32584]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-25T14:32:36.314 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:36 smithi183 ceph-mon[32584]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T14:32:36.314 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:36 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:32:36.340 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_4 2024-04-25T14:32:36.447 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:36 smithi150 ceph-mon[32504]: pgmap v71: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:32:36.448 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:36 smithi150 ceph-mon[32504]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-25T14:32:36.448 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:36 smithi150 ceph-mon[32504]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T14:32:36.448 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:36 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:32:36.568 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:32:37.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:37 smithi150 ceph-mon[32504]: purged_snaps scrub starts 2024-04-25T14:32:37.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:37 smithi150 ceph-mon[32504]: purged_snaps scrub ok 2024-04-25T14:32:37.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:37 smithi150 ceph-mon[32504]: osd.1 [v2:172.21.15.150:6800/1304621982,v1:172.21.15.150:6801/1304621982] boot 2024-04-25T14:32:37.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:37 smithi150 ceph-mon[32504]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T14:32:37.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:37 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:32:37.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:37 smithi183 ceph-mon[32584]: purged_snaps scrub starts 2024-04-25T14:32:37.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:37 smithi183 ceph-mon[32584]: purged_snaps scrub ok 2024-04-25T14:32:37.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:37 smithi183 ceph-mon[32584]: osd.1 [v2:172.21.15.150:6800/1304621982,v1:172.21.15.150:6801/1304621982] boot 2024-04-25T14:32:37.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:37 smithi183 ceph-mon[32584]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T14:32:37.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:37 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:32:37.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:37 smithi053 ceph-mon[27773]: purged_snaps scrub starts 2024-04-25T14:32:37.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:37 smithi053 ceph-mon[27773]: purged_snaps scrub ok 2024-04-25T14:32:37.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:37 smithi053 ceph-mon[27773]: osd.1 [v2:172.21.15.150:6800/1304621982,v1:172.21.15.150:6801/1304621982] boot 2024-04-25T14:32:37.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:37 smithi053 ceph-mon[27773]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T14:32:37.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:37 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:32:38.026 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:38.025+0000 7f0689211700 1 -- 172.21.15.183:0/2534876921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0684102aa0 msgr2=0x7f0684102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:32:38.026 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:38.025+0000 7f0689211700 1 --2- 172.21.15.183:0/2534876921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0684102aa0 0x7f0684102ec0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0674009990 tx=0x7f0674009ca0 comp rx=0 tx=0).stop 2024-04-25T14:32:38.026 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:38.025+0000 7f0689211700 1 -- 172.21.15.183:0/2534876921 shutdown_connections 2024-04-25T14:32:38.027 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:38.025+0000 7f0689211700 1 --2- 172.21.15.183:0/2534876921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0684104640 0x7f068410aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:32:38.027 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:38.025+0000 7f0689211700 1 --2- 172.21.15.183:0/2534876921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0684102aa0 0x7f0684102ec0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:32:38.027 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:38.025+0000 7f0689211700 1 --2- 172.21.15.183:0/2534876921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0684103ca0 0x7f0684104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:32:38.027 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:38.025+0000 7f0689211700 1 -- 172.21.15.183:0/2534876921 >> 172.21.15.183:0/2534876921 conn(0x7f06840fe040 msgr2=0x7f0684100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:32:38.027 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:38.025+0000 7f0689211700 1 -- 172.21.15.183:0/2534876921 shutdown_connections 2024-04-25T14:32:38.027 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:38.025+0000 7f0689211700 1 -- 172.21.15.183:0/2534876921 wait complete. 2024-04-25T14:32:38.028 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:38.026+0000 7f0689211700 1 Processor -- start 2024-04-25T14:32:38.028 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:38.026+0000 7f0689211700 1 -- start start 2024-04-25T14:32:38.029 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:38.027+0000 7f0689211700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0684102aa0 0x7f0684198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:32:38.029 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:38.027+0000 7f0689211700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0684103ca0 0x7f0684199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:32:38.029 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:38.027+0000 7f0689211700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0684104640 0x7f06841a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:32:38.029 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:38.027+0000 7f0689211700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f068410e600 con 0x7f0684103ca0 2024-04-25T14:32:38.029 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:38.027+0000 7f0689211700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f068410e480 con 0x7f0684104640 2024-04-25T14:32:38.029 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:38.027+0000 7f0689211700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f068410e780 con 0x7f0684102aa0 2024-04-25T14:32:38.029 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:38.027+0000 7f0682d9d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0684102aa0 0x7f0684198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:32:38.029 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:38.027+0000 7f0682d9d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0684102aa0 0x7f0684198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:46422/0 (socket says 172.21.15.183:46422) 2024-04-25T14:32:38.029 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:38.027+0000 7f0682d9d700 1 -- 172.21.15.183:0/2983986428 learned_addr learned my addr 172.21.15.183:0/2983986428 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-04-25T14:32:38.029 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:38.027+0000 7f0682d9d700 1 -- 172.21.15.183:0/2983986428 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0684104640 msgr2=0x7f06841a02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:32:38.029 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:38.027+0000 7f0682d9d700 1 --2- 172.21.15.183:0/2983986428 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0684104640 0x7f06841a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:32:38.029 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:38.027+0000 7f0682d9d700 1 -- 172.21.15.183:0/2983986428 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0684103ca0 msgr2=0x7f0684199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:32:38.030 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:38.027+0000 7f0682d9d700 1 --2- 172.21.15.183:0/2983986428 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0684103ca0 0x7f0684199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:32:38.030 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:38.027+0000 7f0682d9d700 1 -- 172.21.15.183:0/2983986428 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06841a0970 con 0x7f0684102aa0 2024-04-25T14:32:38.030 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:38.027+0000 7f0682d9d700 1 --2- 172.21.15.183:0/2983986428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0684102aa0 0x7f0684198d20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0674000c00 tx=0x7f0674005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:32:38.030 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:38.027+0000 7f066bfff700 1 -- 172.21.15.183:0/2983986428 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0674027070 con 0x7f0684102aa0 2024-04-25T14:32:38.030 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:38.027+0000 7f066bfff700 1 -- 172.21.15.183:0/2983986428 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0674022470 con 0x7f0684102aa0 2024-04-25T14:32:38.030 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:38.027+0000 7f0689211700 1 -- 172.21.15.183:0/2983986428 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06841a0c00 con 0x7f0684102aa0 2024-04-25T14:32:38.030 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:38.027+0000 7f066bfff700 1 -- 172.21.15.183:0/2983986428 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067400f6c0 con 0x7f0684102aa0 2024-04-25T14:32:38.030 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:38.027+0000 7f0689211700 1 -- 172.21.15.183:0/2983986428 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06841a1050 con 0x7f0684102aa0 2024-04-25T14:32:38.031 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:38.028+0000 7f0689211700 1 -- 172.21.15.183:0/2983986428 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f068402df80 con 0x7f0684102aa0 2024-04-25T14:32:38.031 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:38.029+0000 7f066bfff700 1 -- 172.21.15.183:0/2983986428 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f067402a070 con 0x7f0684102aa0 2024-04-25T14:32:38.031 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:38.030+0000 7f066bfff700 1 --2- 172.21.15.183:0/2983986428 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f066c0a3bb0 0x7f066c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:32:38.031 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:38.030+0000 7f066bfff700 1 -- 172.21.15.183:0/2983986428 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f06740cfa30 con 0x7f0684102aa0 2024-04-25T14:32:38.031 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:38.031+0000 7f068259c700 1 --2- 172.21.15.183:0/2983986428 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f066c0a3bb0 0x7f066c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:32:38.032 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:38.032+0000 7f068259c700 1 --2- 172.21.15.183:0/2983986428 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f066c0a3bb0 0x7f066c0a6070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f06780097b0 tx=0x7f0678006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:32:38.035 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:38.035+0000 7f066bfff700 1 -- 172.21.15.183:0/2983986428 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0674013070 con 0x7f0684102aa0 2024-04-25T14:32:38.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:38 smithi183 ceph-mon[32584]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:32:38.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:38 smithi183 ceph-mon[32584]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T14:32:38.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:38 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:38.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:38 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:38.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:38 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:32:38.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:38 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:38.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:38 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:32:38.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:38 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:32:38.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:38 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:38.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:38 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:32:38.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:38 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:32:38.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:38 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:32:38.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:38 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:38.247 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:38.247+0000 7f0689211700 1 -- 172.21.15.183:0/2983986428 --> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f0684062e80 con 0x7f066c0a3bb0 2024-04-25T14:32:38.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:38 smithi150 ceph-mon[32504]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:32:38.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:38 smithi150 ceph-mon[32504]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T14:32:38.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:38 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:38.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:38 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:38.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:38 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:32:38.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:38 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:38.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:38 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:32:38.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:38 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:32:38.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:38 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:38.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:38 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:32:38.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:38 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:32:38.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:38 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:32:38.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:38 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:38.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:38 smithi053 ceph-mon[27773]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:32:38.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:38 smithi053 ceph-mon[27773]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T14:32:38.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:38 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:38.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:38 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:38.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:38 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:32:38.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:38 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:38.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:38 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:32:38.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:38 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:32:38.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:38 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:38.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:38 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:32:38.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:38 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:32:38.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:38 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:32:38.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:38 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:39.175 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:39 smithi183 ceph-mon[32584]: Detected new or changed devices on smithi150 2024-04-25T14:32:39.175 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:39 smithi183 ceph-mon[32584]: Adjusting osd_memory_target on smithi150 to 17088M 2024-04-25T14:32:39.175 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:39 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:32:39.176 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:39 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:32:39.176 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:39 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:32:39.434 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:39 smithi053 ceph-mon[27773]: Detected new or changed devices on smithi150 2024-04-25T14:32:39.435 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:39 smithi053 ceph-mon[27773]: Adjusting osd_memory_target on smithi150 to 17088M 2024-04-25T14:32:39.435 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:39 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:32:39.435 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:39 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:32:39.435 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:39 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:32:39.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:39 smithi150 ceph-mon[32504]: Detected new or changed devices on smithi150 2024-04-25T14:32:39.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:39 smithi150 ceph-mon[32504]: Adjusting osd_memory_target on smithi150 to 17088M 2024-04-25T14:32:39.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:39 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:32:39.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:39 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:32:39.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:39 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:32:40.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:40 smithi183 ceph-mon[32584]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:32:40.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:40 smithi183 ceph-mon[32584]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:32:40.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:40 smithi150 ceph-mon[32504]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:32:40.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:40 smithi150 ceph-mon[32504]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:32:40.461 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:40 smithi053 ceph-mon[27773]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:32:40.461 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:40 smithi053 ceph-mon[27773]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:32:41.168 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:41 smithi053 ceph-mon[27773]: from='client.? 172.21.15.183:0/4254676059' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e8effd43-adc0-472a-9eed-a9273b06dc0e"}]: dispatch 2024-04-25T14:32:41.168 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:41 smithi053 ceph-mon[27773]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e8effd43-adc0-472a-9eed-a9273b06dc0e"}]: dispatch 2024-04-25T14:32:41.168 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:41 smithi053 ceph-mon[27773]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e8effd43-adc0-472a-9eed-a9273b06dc0e"}]': finished 2024-04-25T14:32:41.168 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:41 smithi053 ceph-mon[27773]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T14:32:41.169 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:41 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:32:41.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:41 smithi183 ceph-mon[32584]: from='client.? 172.21.15.183:0/4254676059' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e8effd43-adc0-472a-9eed-a9273b06dc0e"}]: dispatch 2024-04-25T14:32:41.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:41 smithi183 ceph-mon[32584]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e8effd43-adc0-472a-9eed-a9273b06dc0e"}]: dispatch 2024-04-25T14:32:41.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:41 smithi183 ceph-mon[32584]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e8effd43-adc0-472a-9eed-a9273b06dc0e"}]': finished 2024-04-25T14:32:41.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:41 smithi183 ceph-mon[32584]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T14:32:41.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:41 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:32:41.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:41 smithi150 ceph-mon[32504]: from='client.? 172.21.15.183:0/4254676059' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e8effd43-adc0-472a-9eed-a9273b06dc0e"}]: dispatch 2024-04-25T14:32:41.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:41 smithi150 ceph-mon[32504]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e8effd43-adc0-472a-9eed-a9273b06dc0e"}]: dispatch 2024-04-25T14:32:41.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:41 smithi150 ceph-mon[32504]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e8effd43-adc0-472a-9eed-a9273b06dc0e"}]': finished 2024-04-25T14:32:41.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:41 smithi150 ceph-mon[32504]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T14:32:41.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:41 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:32:42.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:42 smithi183 ceph-mon[32584]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:32:42.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:42 smithi183 ceph-mon[32584]: from='client.? 172.21.15.183:0/762278765' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:32:42.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:42 smithi150 ceph-mon[32504]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:32:42.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:42 smithi150 ceph-mon[32504]: from='client.? 172.21.15.183:0/762278765' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:32:42.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:42 smithi053 ceph-mon[27773]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:32:42.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:42 smithi053 ceph-mon[27773]: from='client.? 172.21.15.183:0/762278765' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:32:43.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:43 smithi183 ceph-mon[32584]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:32:43.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:43 smithi150 ceph-mon[32504]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:32:43.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:43 smithi053 ceph-mon[27773]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:32:45.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:45 smithi053 ceph-mon[27773]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:32:45.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:45 smithi183 ceph-mon[32584]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:32:45.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:45 smithi150 ceph-mon[32504]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:32:47.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:47 smithi183 ceph-mon[32584]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:32:47.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:47 smithi053 ceph-mon[27773]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:32:47.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:47 smithi150 ceph-mon[32504]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:32:48.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:48 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T14:32:48.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:48 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:32:48.612 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:48 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T14:32:48.612 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:48 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:32:48.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:48 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T14:32:48.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:48 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:32:49.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:49 smithi053 ceph-mon[27773]: Deploying daemon osd.2 on smithi183 2024-04-25T14:32:49.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:49 smithi053 ceph-mon[27773]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:32:49.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:49 smithi150 ceph-mon[32504]: Deploying daemon osd.2 on smithi183 2024-04-25T14:32:49.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:49 smithi150 ceph-mon[32504]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:32:49.656 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:49 smithi183 ceph-mon[32584]: Deploying daemon osd.2 on smithi183 2024-04-25T14:32:49.656 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:49 smithi183 ceph-mon[32584]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:32:51.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:51 smithi183 ceph-mon[32584]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:32:51.561 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:51 smithi053 ceph-mon[27773]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:32:51.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:51 smithi150 ceph-mon[32504]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:32:53.589 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:53 smithi183 ceph-mon[32584]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:32:53.589 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:53 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:53.589 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:53 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:53.590 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:53 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:32:53.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:53 smithi150 ceph-mon[32504]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:32:53.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:53 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:53.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:53 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:53.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:53 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:32:53.786 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:53 smithi053 ceph-mon[27773]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:32:53.786 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:53 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:53.786 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:53 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:53.786 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:53 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:32:55.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:55 smithi183 ceph-mon[32584]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:32:55.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:55 smithi150 ceph-mon[32504]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:32:55.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:55 smithi053 ceph-mon[27773]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:32:56.281 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:56.281+0000 7f066bfff700 1 -- 172.21.15.183:0/2983986428 <== mgr.14152 v2:172.21.15.53:6800/4154933108 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0684062e80 con 0x7f066c0a3bb0 2024-04-25T14:32:56.281 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 2 on host 'smithi183' 2024-04-25T14:32:56.283 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:56.283+0000 7f0689211700 1 -- 172.21.15.183:0/2983986428 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f066c0a3bb0 msgr2=0x7f066c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:32:56.283 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:56.283+0000 7f0689211700 1 --2- 172.21.15.183:0/2983986428 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f066c0a3bb0 0x7f066c0a6070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f06780097b0 tx=0x7f0678006d20 comp rx=0 tx=0).stop 2024-04-25T14:32:56.283 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:56.283+0000 7f0689211700 1 -- 172.21.15.183:0/2983986428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0684102aa0 msgr2=0x7f0684198d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:32:56.283 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:56.283+0000 7f0689211700 1 --2- 172.21.15.183:0/2983986428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0684102aa0 0x7f0684198d20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0674000c00 tx=0x7f0674005ef0 comp rx=0 tx=0).stop 2024-04-25T14:32:56.283 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:56.283+0000 7f0689211700 1 -- 172.21.15.183:0/2983986428 shutdown_connections 2024-04-25T14:32:56.283 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:56.283+0000 7f0689211700 1 --2- 172.21.15.183:0/2983986428 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f066c0a3bb0 0x7f066c0a6070 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:32:56.283 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:56.283+0000 7f0689211700 1 --2- 172.21.15.183:0/2983986428 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0684104640 0x7f06841a02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:32:56.283 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:56.283+0000 7f0689211700 1 --2- 172.21.15.183:0/2983986428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0684102aa0 0x7f0684198d20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:32:56.284 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:56.283+0000 7f0689211700 1 --2- 172.21.15.183:0/2983986428 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0684103ca0 0x7f0684199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:32:56.284 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:56.283+0000 7f0689211700 1 -- 172.21.15.183:0/2983986428 >> 172.21.15.183:0/2983986428 conn(0x7f06840fe040 msgr2=0x7f0684107010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:32:56.284 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:56.284+0000 7f0689211700 1 -- 172.21.15.183:0/2983986428 shutdown_connections 2024-04-25T14:32:56.284 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:32:56.284+0000 7f0689211700 1 -- 172.21.15.183:0/2983986428 wait complete. 2024-04-25T14:32:56.869 DEBUG:teuthology.orchestra.run.smithi183:osd.2> sudo journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@osd.2.service 2024-04-25T14:32:56.872 INFO:tasks.cephadm:Deploying osd.3 on smithi053 with /dev/vg_nvme/lv_3... 2024-04-25T14:32:56.872 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T14:32:57.203 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 25 14:32:57 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-2[37718]: 2024-04-25T14:32:57.091+0000 7f4b7109e7c0 -1 osd.2 0 log_to_monitors true 2024-04-25T14:32:57.344 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:32:57.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:57 smithi053 ceph-mon[27773]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:32:57.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:57 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:57.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:57 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:57.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:57 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:57.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:57 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:57.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:57 smithi053 ceph-mon[27773]: from='osd.2 [v2:172.21.15.183:6800/574509311,v1:172.21.15.183:6801/574509311]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T14:32:57.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:57 smithi053 ceph-mon[27773]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T14:32:57.642 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:57 smithi183 ceph-mon[32584]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:32:57.643 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:57 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:57.643 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:57 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:57.643 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:57 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:57.643 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:57 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:57.643 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:57 smithi183 ceph-mon[32584]: from='osd.2 [v2:172.21.15.183:6800/574509311,v1:172.21.15.183:6801/574509311]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T14:32:57.643 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:57 smithi183 ceph-mon[32584]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T14:32:57.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:57 smithi150 ceph-mon[32504]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:32:57.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:57 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:57.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:57 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:57.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:57 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:57.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:57 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:32:57.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:57 smithi150 ceph-mon[32504]: from='osd.2 [v2:172.21.15.183:6800/574509311,v1:172.21.15.183:6801/574509311]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T14:32:57.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:57 smithi150 ceph-mon[32504]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T14:32:59.127 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:59 smithi053 ceph-mon[27773]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T14:32:59.128 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:59 smithi053 ceph-mon[27773]: from='osd.2 [v2:172.21.15.183:6800/574509311,v1:172.21.15.183:6801/574509311]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T14:32:59.128 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:59 smithi053 ceph-mon[27773]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T14:32:59.128 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:59 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:32:59.128 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:32:59 smithi053 ceph-mon[27773]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T14:32:59.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:59 smithi183 ceph-mon[32584]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T14:32:59.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:59 smithi183 ceph-mon[32584]: from='osd.2 [v2:172.21.15.183:6800/574509311,v1:172.21.15.183:6801/574509311]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T14:32:59.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:59 smithi183 ceph-mon[32584]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T14:32:59.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:59 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:32:59.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:32:59 smithi183 ceph-mon[32584]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T14:32:59.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:59 smithi150 ceph-mon[32504]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T14:32:59.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:59 smithi150 ceph-mon[32504]: from='osd.2 [v2:172.21.15.183:6800/574509311,v1:172.21.15.183:6801/574509311]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T14:32:59.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:59 smithi150 ceph-mon[32504]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T14:32:59.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:59 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:32:59.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:32:59 smithi150 ceph-mon[32504]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T14:33:00.079 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:00 smithi183 ceph-mon[32584]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:33:00.079 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:00 smithi183 ceph-mon[32584]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-25T14:33:00.079 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:00 smithi183 ceph-mon[32584]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T14:33:00.079 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:00 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:33:00.079 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:00 smithi183 ceph-mon[32584]: from='osd.2 ' entity='osd.2' 2024-04-25T14:33:00.079 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:00 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:33:00.080 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 25 14:32:59 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-2[37718]: 2024-04-25T14:32:59.939+0000 7f4b66f04700 -1 osd.2 0 waiting for initial osdmap 2024-04-25T14:33:00.080 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 25 14:33:00 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-2[37718]: 2024-04-25T14:33:00.078+0000 7f4b5e4ee700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:33:00.121 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:00 smithi053 ceph-mon[27773]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:33:00.121 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:00 smithi053 ceph-mon[27773]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-25T14:33:00.122 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:00 smithi053 ceph-mon[27773]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T14:33:00.122 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:00 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:33:00.122 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:00 smithi053 ceph-mon[27773]: from='osd.2 ' entity='osd.2' 2024-04-25T14:33:00.122 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:00 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:33:00.122 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T14:33:00.122 INFO:teuthology.orchestra.run.smithi053.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T14:33:00.122 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10+0 records in 2024-04-25T14:33:00.122 INFO:teuthology.orchestra.run.smithi053.stderr:10+0 records out 2024-04-25T14:33:00.122 INFO:teuthology.orchestra.run.smithi053.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0368471 s, 285 MB/s 2024-04-25T14:33:00.122 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping successful for: 2024-04-25T14:33:00.317 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:00 smithi150 ceph-mon[32504]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:33:00.317 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:00 smithi150 ceph-mon[32504]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-25T14:33:00.317 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:00 smithi150 ceph-mon[32504]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T14:33:00.317 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:00 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:33:00.317 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:00 smithi150 ceph-mon[32504]: from='osd.2 ' entity='osd.2' 2024-04-25T14:33:00.317 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:00 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:33:00.936 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi053:vg_nvme/lv_3 2024-04-25T14:33:01.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:01 smithi150 ceph-mon[32504]: purged_snaps scrub starts 2024-04-25T14:33:01.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:01 smithi150 ceph-mon[32504]: purged_snaps scrub ok 2024-04-25T14:33:01.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:01 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:01.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:01 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:01.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:01 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:33:01.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:01 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:01.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:01 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:33:01.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:01 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:33:01.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:01 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:01.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:01 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:33:01.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:01 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:33:01.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:01 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:33:01.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:01 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:01.212 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:01 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:33:01.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:01 smithi053 ceph-mon[27773]: purged_snaps scrub starts 2024-04-25T14:33:01.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:01 smithi053 ceph-mon[27773]: purged_snaps scrub ok 2024-04-25T14:33:01.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:01 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:01.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:01 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:01.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:01 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:33:01.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:01 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:01.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:01 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:33:01.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:01 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:33:01.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:01 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:01.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:01 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:33:01.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:01 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:33:01.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:01 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:33:01.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:01 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:01.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:01 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:33:01.353 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:33:01.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:01 smithi183 ceph-mon[32584]: purged_snaps scrub starts 2024-04-25T14:33:01.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:01 smithi183 ceph-mon[32584]: purged_snaps scrub ok 2024-04-25T14:33:01.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:01 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:01.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:01 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:01.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:01 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:33:01.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:01 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:01.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:01 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:33:01.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:01 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:33:01.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:01 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:01.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:01 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:33:01.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:01 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:33:01.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:01 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:33:01.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:01 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:01.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:01 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:33:02.251 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:02 smithi053 ceph-mon[27773]: pgmap v90: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:33:02.251 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:02 smithi053 ceph-mon[27773]: Detected new or changed devices on smithi183 2024-04-25T14:33:02.251 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:02 smithi053 ceph-mon[27773]: Adjusting osd_memory_target on smithi183 to 17088M 2024-04-25T14:33:02.251 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:02 smithi053 ceph-mon[27773]: osd.2 [v2:172.21.15.183:6800/574509311,v1:172.21.15.183:6801/574509311] boot 2024-04-25T14:33:02.251 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:02 smithi053 ceph-mon[27773]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T14:33:02.251 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:02 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:33:02.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:02 smithi183 ceph-mon[32584]: pgmap v90: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:33:02.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:02 smithi183 ceph-mon[32584]: Detected new or changed devices on smithi183 2024-04-25T14:33:02.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:02 smithi183 ceph-mon[32584]: Adjusting osd_memory_target on smithi183 to 17088M 2024-04-25T14:33:02.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:02 smithi183 ceph-mon[32584]: osd.2 [v2:172.21.15.183:6800/574509311,v1:172.21.15.183:6801/574509311] boot 2024-04-25T14:33:02.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:02 smithi183 ceph-mon[32584]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T14:33:02.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:02 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:33:02.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:02 smithi150 ceph-mon[32504]: pgmap v90: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:33:02.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:02 smithi150 ceph-mon[32504]: Detected new or changed devices on smithi183 2024-04-25T14:33:02.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:02 smithi150 ceph-mon[32504]: Adjusting osd_memory_target on smithi183 to 17088M 2024-04-25T14:33:02.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:02 smithi150 ceph-mon[32504]: osd.2 [v2:172.21.15.183:6800/574509311,v1:172.21.15.183:6801/574509311] boot 2024-04-25T14:33:02.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:02 smithi150 ceph-mon[32504]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T14:33:02.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:02 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:33:03.208 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:03 smithi053 ceph-mon[27773]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T14:33:03.208 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:03 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:33:03.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:03.386+0000 7f5e6ca00700 1 -- 172.21.15.53:0/2535540599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e6810ac80 msgr2=0x7f5e68111530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:33:03.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:03.386+0000 7f5e6ca00700 1 --2- 172.21.15.53:0/2535540599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e6810ac80 0x7f5e68111530 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5e6000b0d0 tx=0x7f5e6000b3e0 comp rx=0 tx=0).stop 2024-04-25T14:33:03.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:03.386+0000 7f5e6ca00700 1 -- 172.21.15.53:0/2535540599 shutdown_connections 2024-04-25T14:33:03.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:03.386+0000 7f5e6ca00700 1 --2- 172.21.15.53:0/2535540599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e6810ac80 0x7f5e68111530 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:33:03.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:03.386+0000 7f5e6ca00700 1 --2- 172.21.15.53:0/2535540599 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e681090e0 0x7f5e68109500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:33:03.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:03.386+0000 7f5e6ca00700 1 --2- 172.21.15.53:0/2535540599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e6810a2e0 0x7f5e6810a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:33:03.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:03.386+0000 7f5e6ca00700 1 -- 172.21.15.53:0/2535540599 >> 172.21.15.53:0/2535540599 conn(0x7f5e680780b0 msgr2=0x7f5e6807a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:33:03.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:03.386+0000 7f5e6ca00700 1 -- 172.21.15.53:0/2535540599 shutdown_connections 2024-04-25T14:33:03.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:03.386+0000 7f5e6ca00700 1 -- 172.21.15.53:0/2535540599 wait complete. 2024-04-25T14:33:03.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:03.387+0000 7f5e6ca00700 1 Processor -- start 2024-04-25T14:33:03.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:03.388+0000 7f5e6ca00700 1 -- start start 2024-04-25T14:33:03.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:03.388+0000 7f5e6ca00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e681090e0 0x7f5e6819f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:33:03.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:03.388+0000 7f5e6ca00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e6810a2e0 0x7f5e6819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:33:03.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:03.388+0000 7f5e6ca00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e6810ac80 0x7f5e681a6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:33:03.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:03.388+0000 7f5e6ca00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5e68114be0 con 0x7f5e6810ac80 2024-04-25T14:33:03.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:03.388+0000 7f5e6ca00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5e68114a60 con 0x7f5e681090e0 2024-04-25T14:33:03.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:03.388+0000 7f5e6ca00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5e68114d60 con 0x7f5e6810a2e0 2024-04-25T14:33:03.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:03.388+0000 7f5e65e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e681090e0 0x7f5e6819f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:33:03.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:03.388+0000 7f5e66800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e6810ac80 0x7f5e681a6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:33:03.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:03.388+0000 7f5e65e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e681090e0 0x7f5e6819f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.53:35354/0 (socket says 172.21.15.53:35354) 2024-04-25T14:33:03.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:03.388+0000 7f5e65e00700 1 -- 172.21.15.53:0/3320559682 learned_addr learned my addr 172.21.15.53:0/3320559682 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:33:03.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:03.389+0000 7f5e5fe00700 1 --2- 172.21.15.53:0/3320559682 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e6810a2e0 0x7f5e6819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:33:03.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:03.389+0000 7f5e66800700 1 -- 172.21.15.53:0/3320559682 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e6810a2e0 msgr2=0x7f5e6819f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:33:03.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:03.389+0000 7f5e66800700 1 --2- 172.21.15.53:0/3320559682 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e6810a2e0 0x7f5e6819f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:33:03.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:03.389+0000 7f5e66800700 1 -- 172.21.15.53:0/3320559682 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e681090e0 msgr2=0x7f5e6819f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:33:03.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:03.389+0000 7f5e66800700 1 --2- 172.21.15.53:0/3320559682 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e681090e0 0x7f5e6819f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:33:03.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:03.389+0000 7f5e66800700 1 -- 172.21.15.53:0/3320559682 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e681a6dc0 con 0x7f5e6810ac80 2024-04-25T14:33:03.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:03.389+0000 7f5e66800700 1 --2- 172.21.15.53:0/3320559682 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e6810ac80 0x7f5e681a6720 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5e6000b770 tx=0x7f5e60004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:33:03.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:03.390+0000 7f5e5ea00700 1 -- 172.21.15.53:0/3320559682 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6001f070 con 0x7f5e6810ac80 2024-04-25T14:33:03.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:03.390+0000 7f5e5ea00700 1 -- 172.21.15.53:0/3320559682 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e60007b80 con 0x7f5e6810ac80 2024-04-25T14:33:03.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:03.390+0000 7f5e6ca00700 1 -- 172.21.15.53:0/3320559682 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e681a7050 con 0x7f5e6810ac80 2024-04-25T14:33:03.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:03.390+0000 7f5e5ea00700 1 -- 172.21.15.53:0/3320559682 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6001b690 con 0x7f5e6810ac80 2024-04-25T14:33:03.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:03.390+0000 7f5e6ca00700 1 -- 172.21.15.53:0/3320559682 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e681a7500 con 0x7f5e6810ac80 2024-04-25T14:33:03.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:03.391+0000 7f5e6ca00700 1 -- 172.21.15.53:0/3320559682 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e68069730 con 0x7f5e6810ac80 2024-04-25T14:33:03.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:03.394+0000 7f5e5ea00700 1 -- 172.21.15.53:0/3320559682 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f5e60022070 con 0x7f5e6810ac80 2024-04-25T14:33:03.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:03.396+0000 7f5e5ea00700 1 --2- 172.21.15.53:0/3320559682 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5e4c0a3c30 0x7f5e4c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:33:03.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:03.396+0000 7f5e5ea00700 1 -- 172.21.15.53:0/3320559682 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3226+0+0 (secure 0 0 0) 0x7f5e6000f020 con 0x7f5e6810ac80 2024-04-25T14:33:03.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:03.396+0000 7f5e65e00700 1 --2- 172.21.15.53:0/3320559682 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5e4c0a3c30 0x7f5e4c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:33:03.398 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:03.397+0000 7f5e65e00700 1 --2- 172.21.15.53:0/3320559682 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5e4c0a3c30 0x7f5e4c0a60f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5e58005950 tx=0x7f5e58009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:33:03.403 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:03.400+0000 7f5e5ea00700 1 -- 172.21.15.53:0/3320559682 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5e60029020 con 0x7f5e6810ac80 2024-04-25T14:33:03.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:03 smithi183 ceph-mon[32584]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T14:33:03.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:03 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:33:03.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:03 smithi150 ceph-mon[32504]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T14:33:03.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:03 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:33:03.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:03.642+0000 7f5e6ca00700 1 -- 172.21.15.53:0/3320559682 --> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f5e68062f40 con 0x7f5e4c0a3c30 2024-04-25T14:33:04.372 INFO:journalctl@ceph.osd.0.smithi053.stdout:Apr 25 14:33:04 smithi053 sudo[46733]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T14:33:04.372 INFO:journalctl@ceph.osd.0.smithi053.stdout:Apr 25 14:33:04 smithi053 sudo[46733]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:33:04.372 INFO:journalctl@ceph.osd.0.smithi053.stdout:Apr 25 14:33:04 smithi053 sudo[46733]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:33:04.372 INFO:journalctl@ceph.osd.0.smithi053.stdout:Apr 25 14:33:04 smithi053 sudo[46733]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:33:04.372 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:04 smithi053 ceph-mon[27773]: pgmap v93: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:04.373 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:04 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T14:33:04.373 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:04 smithi053 ceph-mon[27773]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T14:33:04.373 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:04 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:33:04.373 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:04 smithi053 ceph-mon[27773]: from='client.14241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:33:04.373 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:04 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:33:04.373 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:04 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:33:04.373 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:04 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:33:04.423 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:04 smithi150 ceph-mon[32504]: pgmap v93: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:04.423 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:04 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T14:33:04.423 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:04 smithi150 ceph-mon[32504]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T14:33:04.423 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:04 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:33:04.423 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:04 smithi150 ceph-mon[32504]: from='client.14241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:33:04.423 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:04 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:33:04.423 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:04 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:33:04.423 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:04 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:33:04.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:04 smithi183 ceph-mon[32584]: pgmap v93: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:04.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:04 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T14:33:04.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:04 smithi183 ceph-mon[32584]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T14:33:04.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:04 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:33:04.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:04 smithi183 ceph-mon[32584]: from='client.14241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:33:04.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:04 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:33:04.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:04 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:33:04.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:04 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:33:04.646 INFO:journalctl@ceph.osd.0.smithi053.stdout:Apr 25 14:33:04 smithi053 sudo[46746]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T14:33:04.646 INFO:journalctl@ceph.osd.0.smithi053.stdout:Apr 25 14:33:04 smithi053 sudo[46746]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:33:04.646 INFO:journalctl@ceph.osd.0.smithi053.stdout:Apr 25 14:33:04 smithi053 sudo[46746]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:33:04.646 INFO:journalctl@ceph.osd.0.smithi053.stdout:Apr 25 14:33:04 smithi053 sudo[46746]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:33:04.647 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:04 smithi053 sudo[46792]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T14:33:04.647 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:04 smithi053 sudo[46792]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:33:04.647 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:04 smithi053 sudo[46792]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:33:04.647 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:04 smithi053 sudo[46792]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:33:04.647 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:04 smithi053 sudo[46795]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T14:33:04.647 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:04 smithi053 sudo[46795]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:33:04.647 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:04 smithi053 sudo[46795]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:33:04.647 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:04 smithi053 sudo[46795]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:33:04.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:04 smithi150 sudo[41092]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T14:33:04.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:04 smithi150 sudo[41092]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:33:04.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:04 smithi150 sudo[41092]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:33:04.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:04 smithi150 sudo[41092]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:33:04.684 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 14:33:04 smithi150 sudo[41086]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T14:33:04.684 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 14:33:04 smithi150 sudo[41086]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:33:04.684 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 14:33:04 smithi150 sudo[41086]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:33:04.684 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 14:33:04 smithi150 sudo[41086]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:33:04.684 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 14:33:04 smithi150 sudo[41089]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T14:33:04.684 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 14:33:04 smithi150 sudo[41089]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:33:04.684 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 14:33:04 smithi150 sudo[41089]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:33:04.684 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 14:33:04 smithi150 sudo[41089]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:33:04.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:04 smithi183 sudo[41348]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T14:33:04.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:04 smithi183 sudo[41348]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:33:04.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:04 smithi183 sudo[41348]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:33:04.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:04 smithi183 sudo[41348]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:33:04.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:04 smithi183 sudo[41351]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T14:33:04.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:04 smithi183 sudo[41351]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:33:04.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:04 smithi183 sudo[41351]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:33:04.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:04 smithi183 sudo[41351]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:33:04.955 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 25 14:33:04 smithi183 sudo[41342]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T14:33:04.955 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 25 14:33:04 smithi183 sudo[41342]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:33:04.955 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 25 14:33:04 smithi183 sudo[41342]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:33:04.955 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 25 14:33:04 smithi183 sudo[41342]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:33:04.955 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 25 14:33:04 smithi183 sudo[41345]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T14:33:04.955 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 25 14:33:04 smithi183 sudo[41345]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:33:04.955 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 25 14:33:04 smithi183 sudo[41345]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:33:04.955 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 25 14:33:04 smithi183 sudo[41345]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:33:04.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:04 smithi150 sudo[41095]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T14:33:04.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:04 smithi150 sudo[41095]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:33:04.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:04 smithi150 sudo[41095]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:33:04.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:04 smithi150 sudo[41095]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:33:05.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:05 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T14:33:05.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:05 smithi053 ceph-mon[27773]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T14:33:05.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:05 smithi053 ceph-mon[27773]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:05.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:05 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:05.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:05 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:05.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:05 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:05.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:05 smithi053 ceph-mon[27773]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T14:33:05.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:05 smithi053 ceph-mon[27773]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T14:33:05.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:05 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:33:05.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:05 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:33:05.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:05 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:33:05.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:05 smithi053 ceph-mon[27773]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T14:33:05.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:05 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:33:05.285 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:05 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:33:05.285 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:05 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:33:05.285 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:05 smithi053 ceph-mon[27773]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T14:33:05.285 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:05 smithi053 ceph-mon[27773]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T14:33:05.285 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:05 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:33:05.285 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:05 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:33:05.285 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:05 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:33:05.285 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:05 smithi053 ceph-mon[27773]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T14:33:05.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:05 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T14:33:05.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:05 smithi183 ceph-mon[32584]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T14:33:05.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:05 smithi183 ceph-mon[32584]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:05.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:05 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:05.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:05 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:05.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:05 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:05.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:05 smithi183 ceph-mon[32584]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T14:33:05.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:05 smithi183 ceph-mon[32584]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T14:33:05.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:05 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:33:05.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:05 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:33:05.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:05 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:33:05.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:05 smithi183 ceph-mon[32584]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T14:33:05.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:05 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:33:05.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:05 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:33:05.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:05 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:33:05.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:05 smithi183 ceph-mon[32584]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T14:33:05.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:05 smithi183 ceph-mon[32584]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T14:33:05.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:05 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:33:05.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:05 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:33:05.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:05 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:33:05.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:05 smithi183 ceph-mon[32584]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T14:33:05.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:05 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T14:33:05.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:05 smithi150 ceph-mon[32504]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T14:33:05.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:05 smithi150 ceph-mon[32504]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:05.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:05 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:05.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:05 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:05.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:05 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:05.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:05 smithi150 ceph-mon[32504]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T14:33:05.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:05 smithi150 ceph-mon[32504]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T14:33:05.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:05 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:33:05.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:05 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:33:05.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:05 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:33:05.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:05 smithi150 ceph-mon[32504]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T14:33:05.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:05 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:33:05.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:05 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:33:05.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:05 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:33:05.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:05 smithi150 ceph-mon[32504]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T14:33:05.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:05 smithi150 ceph-mon[32504]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T14:33:05.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:05 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:33:05.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:05 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:33:05.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:05 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:33:05.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:05 smithi150 ceph-mon[32504]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T14:33:06.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:06 smithi053 ceph-mon[27773]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T14:33:06.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:06 smithi183 ceph-mon[32584]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T14:33:06.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:06 smithi150 ceph-mon[32504]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T14:33:07.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:07.136+0000 7f5e5ea00700 1 -- 172.21.15.53:0/3320559682 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5e6009a220 con 0x7f5e6810ac80 2024-04-25T14:33:07.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:07 smithi183 ceph-mon[32584]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 93 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:07.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:07 smithi150 ceph-mon[32504]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 93 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:07.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:07 smithi053 ceph-mon[27773]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 93 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:08.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:08 smithi183 ceph-mon[32584]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T14:33:08.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:08 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/3313764265' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7cfced4-9b40-40de-a303-75ea8d5df4a1"}]: dispatch 2024-04-25T14:33:08.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:08 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/3313764265' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7cfced4-9b40-40de-a303-75ea8d5df4a1"}]': finished 2024-04-25T14:33:08.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:08 smithi183 ceph-mon[32584]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T14:33:08.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:08 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:33:08.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:08 smithi150 ceph-mon[32504]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T14:33:08.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:08 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/3313764265' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7cfced4-9b40-40de-a303-75ea8d5df4a1"}]: dispatch 2024-04-25T14:33:08.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:08 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/3313764265' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7cfced4-9b40-40de-a303-75ea8d5df4a1"}]': finished 2024-04-25T14:33:08.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:08 smithi150 ceph-mon[32504]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T14:33:08.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:08 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:33:08.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:08 smithi053 ceph-mon[27773]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T14:33:08.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:08 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/3313764265' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7cfced4-9b40-40de-a303-75ea8d5df4a1"}]: dispatch 2024-04-25T14:33:08.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:08 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/3313764265' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7cfced4-9b40-40de-a303-75ea8d5df4a1"}]': finished 2024-04-25T14:33:08.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:08 smithi053 ceph-mon[27773]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T14:33:08.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:08 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:33:09.434 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:09 smithi053 ceph-mon[27773]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:09.435 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:09 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/3774877515' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:33:09.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:09 smithi183 ceph-mon[32584]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:09.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:09 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/3774877515' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:33:09.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:09 smithi150 ceph-mon[32504]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:09.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:09 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/3774877515' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:33:11.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:11 smithi053 ceph-mon[27773]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:11.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:11 smithi183 ceph-mon[32584]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:11.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:11 smithi150 ceph-mon[32504]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:13.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:13 smithi053 ceph-mon[27773]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:13.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:13 smithi183 ceph-mon[32584]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:13.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:13 smithi150 ceph-mon[32504]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:15.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:15 smithi053 ceph-mon[27773]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:15.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:15 smithi183 ceph-mon[32584]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:15.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:15 smithi150 ceph-mon[32504]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:17.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:17 smithi053 ceph-mon[27773]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:17.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:17 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T14:33:17.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:17 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:33:17.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:17 smithi053 ceph-mon[27773]: Deploying daemon osd.3 on smithi053 2024-04-25T14:33:17.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:17 smithi183 ceph-mon[32584]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:17.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:17 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T14:33:17.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:17 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:33:17.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:17 smithi183 ceph-mon[32584]: Deploying daemon osd.3 on smithi053 2024-04-25T14:33:17.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:17 smithi150 ceph-mon[32504]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:17.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:17 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T14:33:17.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:17 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:33:17.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:17 smithi150 ceph-mon[32504]: Deploying daemon osd.3 on smithi053 2024-04-25T14:33:19.307 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:19 smithi053 ceph-mon[27773]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:19.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:19 smithi183 ceph-mon[32584]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:19.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:19 smithi150 ceph-mon[32504]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:21.349 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:21 smithi053 ceph-mon[27773]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:21.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:21 smithi183 ceph-mon[32584]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:21.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:21 smithi150 ceph-mon[32504]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:23.483 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:23 smithi053 ceph-mon[27773]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:23.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:23 smithi183 ceph-mon[32584]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:23.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:23 smithi150 ceph-mon[32504]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:25.837 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:25 smithi053 ceph-mon[27773]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:25.837 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:25 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:25.837 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:25 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:33:25.837 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:25 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:25.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:25 smithi183 ceph-mon[32584]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:25.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:25 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:25.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:25 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:33:25.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:25 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:25.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:25 smithi150 ceph-mon[32504]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:25.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:25 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:25.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:25 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:33:25.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:25 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:27.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:27 smithi183 ceph-mon[32584]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:27.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:27 smithi150 ceph-mon[32504]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:28.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:27 smithi053 ceph-mon[27773]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:29.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:28 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:29.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:28 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:29.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:28 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:33:29.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:28 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:33:29.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:28 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:29.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:28 smithi053 ceph-mon[27773]: from='osd.3 [v2:172.21.15.53:6810/570672219,v1:172.21.15.53:6811/570672219]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T14:33:29.197 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 3 on host 'smithi053' 2024-04-25T14:33:29.198 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:29.197+0000 7f5e5ea00700 1 -- 172.21.15.53:0/3320559682 <== mgr.14152 v2:172.21.15.53:6800/4154933108 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5e68062f40 con 0x7f5e4c0a3c30 2024-04-25T14:33:29.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:29.201+0000 7f5e6ca00700 1 -- 172.21.15.53:0/3320559682 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5e4c0a3c30 msgr2=0x7f5e4c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:33:29.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:29.201+0000 7f5e6ca00700 1 --2- 172.21.15.53:0/3320559682 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5e4c0a3c30 0x7f5e4c0a60f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5e58005950 tx=0x7f5e58009260 comp rx=0 tx=0).stop 2024-04-25T14:33:29.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:29.201+0000 7f5e6ca00700 1 -- 172.21.15.53:0/3320559682 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e6810ac80 msgr2=0x7f5e681a6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:33:29.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:29.201+0000 7f5e6ca00700 1 --2- 172.21.15.53:0/3320559682 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e6810ac80 0x7f5e681a6720 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5e6000b770 tx=0x7f5e60004c80 comp rx=0 tx=0).stop 2024-04-25T14:33:29.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:29.201+0000 7f5e6ca00700 1 -- 172.21.15.53:0/3320559682 shutdown_connections 2024-04-25T14:33:29.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:29.201+0000 7f5e6ca00700 1 --2- 172.21.15.53:0/3320559682 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5e4c0a3c30 0x7f5e4c0a60f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:33:29.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:29.201+0000 7f5e6ca00700 1 --2- 172.21.15.53:0/3320559682 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e6810ac80 0x7f5e681a6720 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:33:29.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:29.201+0000 7f5e6ca00700 1 --2- 172.21.15.53:0/3320559682 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e681090e0 0x7f5e6819f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:33:29.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:29.201+0000 7f5e6ca00700 1 --2- 172.21.15.53:0/3320559682 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e6810a2e0 0x7f5e6819f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:33:29.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:29.201+0000 7f5e6ca00700 1 -- 172.21.15.53:0/3320559682 >> 172.21.15.53:0/3320559682 conn(0x7f5e680780b0 msgr2=0x7f5e6810d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:33:29.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:29.201+0000 7f5e6ca00700 1 -- 172.21.15.53:0/3320559682 shutdown_connections 2024-04-25T14:33:29.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:33:29.201+0000 7f5e6ca00700 1 -- 172.21.15.53:0/3320559682 wait complete. 2024-04-25T14:33:29.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:28 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:29.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:28 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:29.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:28 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:33:29.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:28 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:33:29.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:28 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:29.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:28 smithi183 ceph-mon[32584]: from='osd.3 [v2:172.21.15.53:6810/570672219,v1:172.21.15.53:6811/570672219]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T14:33:29.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:28 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:29.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:28 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:29.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:28 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:33:29.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:28 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:33:29.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:28 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:29.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:28 smithi150 ceph-mon[32504]: from='osd.3 [v2:172.21.15.53:6810/570672219,v1:172.21.15.53:6811/570672219]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T14:33:29.803 DEBUG:teuthology.orchestra.run.smithi053:osd.3> sudo journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@osd.3.service 2024-04-25T14:33:29.810 INFO:tasks.cephadm:Deploying osd.4 on smithi150 with /dev/vg_nvme/lv_3... 2024-04-25T14:33:29.810 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T14:33:29.993 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:29 smithi053 ceph-mon[27773]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:29.993 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:29 smithi053 ceph-mon[27773]: from='osd.3 [v2:172.21.15.53:6810/570672219,v1:172.21.15.53:6811/570672219]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T14:33:29.993 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:29 smithi053 ceph-mon[27773]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T14:33:29.993 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:29 smithi053 ceph-mon[27773]: from='osd.3 [v2:172.21.15.53:6810/570672219,v1:172.21.15.53:6811/570672219]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-04-25T14:33:29.993 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:29 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:33:29.993 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:29 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:29.993 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:29 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:29.993 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:29 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:33:29.993 INFO:journalctl@ceph.osd.3.smithi053.stdout:Apr 25 14:33:29 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-3[48936]: 2024-04-25T14:33:29.982+0000 7f06dc800700 -1 osd.3 0 waiting for initial osdmap 2024-04-25T14:33:29.994 INFO:journalctl@ceph.osd.3.smithi053.stdout:Apr 25 14:33:29 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-3[48936]: 2024-04-25T14:33:29.992+0000 7f06d7600700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:33:30.060 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:33:30.182 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:29 smithi150 ceph-mon[32504]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:30.182 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:29 smithi150 ceph-mon[32504]: from='osd.3 [v2:172.21.15.53:6810/570672219,v1:172.21.15.53:6811/570672219]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T14:33:30.182 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:29 smithi150 ceph-mon[32504]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T14:33:30.182 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:29 smithi150 ceph-mon[32504]: from='osd.3 [v2:172.21.15.53:6810/570672219,v1:172.21.15.53:6811/570672219]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-04-25T14:33:30.182 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:29 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:33:30.182 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:29 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:30.182 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:29 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:30.182 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:29 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:33:30.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:29 smithi183 ceph-mon[32584]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:30.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:29 smithi183 ceph-mon[32584]: from='osd.3 [v2:172.21.15.53:6810/570672219,v1:172.21.15.53:6811/570672219]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T14:33:30.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:29 smithi183 ceph-mon[32584]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T14:33:30.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:29 smithi183 ceph-mon[32584]: from='osd.3 [v2:172.21.15.53:6810/570672219,v1:172.21.15.53:6811/570672219]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-04-25T14:33:30.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:29 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:33:30.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:29 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:30.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:29 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:30.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:29 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:33:31.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:30 smithi053 ceph-mon[27773]: from='osd.3 [v2:172.21.15.53:6810/570672219,v1:172.21.15.53:6811/570672219]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-04-25T14:33:31.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:30 smithi053 ceph-mon[27773]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T14:33:31.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:30 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:33:31.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:30 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:33:31.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:30 smithi053 ceph-mon[27773]: osd.3 [v2:172.21.15.53:6810/570672219,v1:172.21.15.53:6811/570672219] boot 2024-04-25T14:33:31.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:30 smithi053 ceph-mon[27773]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T14:33:31.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:30 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:33:31.298 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:30 smithi150 ceph-mon[32504]: from='osd.3 [v2:172.21.15.53:6810/570672219,v1:172.21.15.53:6811/570672219]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-04-25T14:33:31.299 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:30 smithi150 ceph-mon[32504]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T14:33:31.299 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:30 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:33:31.299 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:30 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:33:31.299 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:30 smithi150 ceph-mon[32504]: osd.3 [v2:172.21.15.53:6810/570672219,v1:172.21.15.53:6811/570672219] boot 2024-04-25T14:33:31.299 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:30 smithi150 ceph-mon[32504]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T14:33:31.299 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:30 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:33:31.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:30 smithi183 ceph-mon[32584]: from='osd.3 [v2:172.21.15.53:6810/570672219,v1:172.21.15.53:6811/570672219]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-04-25T14:33:31.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:30 smithi183 ceph-mon[32584]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T14:33:31.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:30 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:33:31.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:30 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:33:31.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:30 smithi183 ceph-mon[32584]: osd.3 [v2:172.21.15.53:6810/570672219,v1:172.21.15.53:6811/570672219] boot 2024-04-25T14:33:31.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:30 smithi183 ceph-mon[32584]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T14:33:31.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:30 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:33:31.980 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T14:33:31.980 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T14:33:31.980 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-04-25T14:33:31.980 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-04-25T14:33:31.980 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114061 s, 919 MB/s 2024-04-25T14:33:31.980 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-04-25T14:33:32.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:31 smithi053 ceph-mon[27773]: purged_snaps scrub starts 2024-04-25T14:33:32.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:31 smithi053 ceph-mon[27773]: purged_snaps scrub ok 2024-04-25T14:33:32.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:31 smithi053 ceph-mon[27773]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:32.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:31 smithi053 ceph-mon[27773]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T14:33:32.290 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:31 smithi150 ceph-mon[32504]: purged_snaps scrub starts 2024-04-25T14:33:32.290 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:31 smithi150 ceph-mon[32504]: purged_snaps scrub ok 2024-04-25T14:33:32.291 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:31 smithi150 ceph-mon[32504]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:32.291 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:31 smithi150 ceph-mon[32504]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T14:33:32.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:31 smithi183 ceph-mon[32584]: purged_snaps scrub starts 2024-04-25T14:33:32.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:31 smithi183 ceph-mon[32584]: purged_snaps scrub ok 2024-04-25T14:33:32.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:31 smithi183 ceph-mon[32584]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:33:32.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:31 smithi183 ceph-mon[32584]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T14:33:32.580 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_3 2024-04-25T14:33:32.824 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:33:33.824 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:33 smithi150 ceph-mon[32504]: pgmap v116: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:33:33.824 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:33 smithi150 ceph-mon[32504]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T14:33:33.890 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:33 smithi053 ceph-mon[27773]: pgmap v116: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:33:33.890 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:33 smithi053 ceph-mon[27773]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T14:33:33.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:33 smithi183 ceph-mon[32584]: pgmap v116: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:33:33.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:33 smithi183 ceph-mon[32584]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T14:33:34.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:34.312+0000 7ff0a877c700 1 -- 172.21.15.150:0/828119938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0a0103540 msgr2=0x7ff0a0105930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:33:34.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:34.312+0000 7ff0a877c700 1 --2- 172.21.15.150:0/828119938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0a0103540 0x7ff0a0105930 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff09c009990 tx=0x7ff09c009ca0 comp rx=0 tx=0).stop 2024-04-25T14:33:34.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:34.313+0000 7ff0a877c700 1 -- 172.21.15.150:0/828119938 shutdown_connections 2024-04-25T14:33:34.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:34.313+0000 7ff0a877c700 1 --2- 172.21.15.150:0/828119938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0a01087a0 0x7ff0a010ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:33:34.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:34.313+0000 7ff0a877c700 1 --2- 172.21.15.150:0/828119938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0a0103540 0x7ff0a0105930 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:33:34.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:34.313+0000 7ff0a877c700 1 --2- 172.21.15.150:0/828119938 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0a0105e70 0x7ff0a0108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:33:34.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:34.313+0000 7ff0a877c700 1 -- 172.21.15.150:0/828119938 >> 172.21.15.150:0/828119938 conn(0x7ff0a00fcf70 msgr2=0x7ff0a00ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:33:34.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:34.313+0000 7ff0a877c700 1 -- 172.21.15.150:0/828119938 shutdown_connections 2024-04-25T14:33:34.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:34.313+0000 7ff0a877c700 1 -- 172.21.15.150:0/828119938 wait complete. 2024-04-25T14:33:34.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:34.313+0000 7ff0a877c700 1 Processor -- start 2024-04-25T14:33:34.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:34.313+0000 7ff0a877c700 1 -- start start 2024-04-25T14:33:34.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:34.313+0000 7ff0a877c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0a0103540 0x7ff0a0198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:33:34.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:34.313+0000 7ff0a877c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0a0105e70 0x7ff0a01992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:33:34.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:34.313+0000 7ff0a877c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0a01087a0 0x7ff0a01a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:33:34.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:34.313+0000 7ff0a877c700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a010e630 con 0x7ff0a0105e70 2024-04-25T14:33:34.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:34.313+0000 7ff0a877c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a010e4b0 con 0x7ff0a0103540 2024-04-25T14:33:34.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:34.313+0000 7ff0a877c700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a010e7b0 con 0x7ff0a01087a0 2024-04-25T14:33:34.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:34.313+0000 7ff0a6518700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0a0103540 0x7ff0a0198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:33:34.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:34.313+0000 7ff0a6518700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0a0103540 0x7ff0a0198da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:39048/0 (socket says 172.21.15.150:39048) 2024-04-25T14:33:34.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:34.313+0000 7ff0a6518700 1 -- 172.21.15.150:0/2222621969 learned_addr learned my addr 172.21.15.150:0/2222621969 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-25T14:33:34.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:34.314+0000 7ff0a6d19700 1 --2- 172.21.15.150:0/2222621969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0a01087a0 0x7ff0a01a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:33:34.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:34.314+0000 7ff0a6518700 1 -- 172.21.15.150:0/2222621969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0a01087a0 msgr2=0x7ff0a01a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:33:34.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:34.314+0000 7ff0a6518700 1 --2- 172.21.15.150:0/2222621969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0a01087a0 0x7ff0a01a0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:33:34.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:34.314+0000 7ff0a6518700 1 -- 172.21.15.150:0/2222621969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0a0105e70 msgr2=0x7ff0a01992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:33:34.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:34.314+0000 7ff0a6518700 1 --2- 172.21.15.150:0/2222621969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0a0105e70 0x7ff0a01992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:33:34.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:34.314+0000 7ff0a6518700 1 -- 172.21.15.150:0/2222621969 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0a01a09f0 con 0x7ff0a0103540 2024-04-25T14:33:34.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:34.314+0000 7ff0a6518700 1 --2- 172.21.15.150:0/2222621969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0a0103540 0x7ff0a0198da0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff09c006010 tx=0x7ff09c0047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:33:34.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:34.314+0000 7ff0977fe700 1 -- 172.21.15.150:0/2222621969 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09c027070 con 0x7ff0a0103540 2024-04-25T14:33:34.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:34.314+0000 7ff0977fe700 1 -- 172.21.15.150:0/2222621969 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff09c022470 con 0x7ff0a0103540 2024-04-25T14:33:34.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:34.314+0000 7ff0a877c700 1 -- 172.21.15.150:0/2222621969 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0a01a0c80 con 0x7ff0a0103540 2024-04-25T14:33:34.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:34.314+0000 7ff0977fe700 1 -- 172.21.15.150:0/2222621969 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09c00f6c0 con 0x7ff0a0103540 2024-04-25T14:33:34.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:34.314+0000 7ff0a877c700 1 -- 172.21.15.150:0/2222621969 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0a01a1130 con 0x7ff0a0103540 2024-04-25T14:33:34.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:34.315+0000 7ff0a877c700 1 -- 172.21.15.150:0/2222621969 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0a0069730 con 0x7ff0a0103540 2024-04-25T14:33:34.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:34.316+0000 7ff0977fe700 1 -- 172.21.15.150:0/2222621969 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff09c02a070 con 0x7ff0a0103540 2024-04-25T14:33:34.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:34.316+0000 7ff0977fe700 1 --2- 172.21.15.150:0/2222621969 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff08c0a3cc0 0x7ff08c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:33:34.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:34.317+0000 7ff0977fe700 1 -- 172.21.15.150:0/2222621969 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3669+0+0 (secure 0 0 0) 0x7ff09c0d0000 con 0x7ff0a0103540 2024-04-25T14:33:34.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:34.317+0000 7ff0a5d17700 1 --2- 172.21.15.150:0/2222621969 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff08c0a3cc0 0x7ff08c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:33:34.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:34.318+0000 7ff0a5d17700 1 --2- 172.21.15.150:0/2222621969 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff08c0a3cc0 0x7ff08c0a6180 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff0900097b0 tx=0x7ff090006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:33:34.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:34.322+0000 7ff0977fe700 1 -- 172.21.15.150:0/2222621969 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff09c013070 con 0x7ff0a0103540 2024-04-25T14:33:34.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:34.527+0000 7ff0a877c700 1 -- 172.21.15.150:0/2222621969 --> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7ff0a0062f40 con 0x7ff08c0a3cc0 2024-04-25T14:33:34.623 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:34 smithi053 ceph-mon[27773]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T14:33:34.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:34 smithi183 ceph-mon[32584]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T14:33:34.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:34 smithi150 ceph-mon[32504]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T14:33:35.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:35 smithi150 ceph-mon[32504]: pgmap v119: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:33:35.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:35 smithi150 ceph-mon[32504]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:33:35.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:35 smithi150 ceph-mon[32504]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T14:33:35.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:35 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:33:35.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:35 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:33:35.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:35 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:33:35.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:35 smithi150 ceph-mon[32504]: Detected new or changed devices on smithi053 2024-04-25T14:33:35.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:35 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:35.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:35 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:35.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:35 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:33:35.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:35 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:33:35.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:35 smithi150 ceph-mon[32504]: Adjusting osd_memory_target on smithi053 to 8547M 2024-04-25T14:33:35.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:35 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:35.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:35 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:33:35.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:35 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:33:35.712 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:35 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:35.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:35 smithi183 ceph-mon[32584]: pgmap v119: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:33:35.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:35 smithi183 ceph-mon[32584]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:33:35.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:35 smithi183 ceph-mon[32584]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T14:33:35.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:35 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:33:35.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:35 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:33:35.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:35 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:33:35.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:35 smithi183 ceph-mon[32584]: Detected new or changed devices on smithi053 2024-04-25T14:33:35.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:35 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:35.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:35 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:35.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:35 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:33:35.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:35 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:33:35.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:35 smithi183 ceph-mon[32584]: Adjusting osd_memory_target on smithi053 to 8547M 2024-04-25T14:33:35.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:35 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:35.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:35 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:33:35.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:35 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:33:35.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:35 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:36.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:35 smithi053 ceph-mon[27773]: pgmap v119: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:33:36.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:35 smithi053 ceph-mon[27773]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:33:36.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:35 smithi053 ceph-mon[27773]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T14:33:36.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:35 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:33:36.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:35 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:33:36.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:35 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:33:36.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:35 smithi053 ceph-mon[27773]: Detected new or changed devices on smithi053 2024-04-25T14:33:36.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:35 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:36.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:35 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:36.034 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:35 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:33:36.034 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:35 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:33:36.034 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:35 smithi053 ceph-mon[27773]: Adjusting osd_memory_target on smithi053 to 8547M 2024-04-25T14:33:36.034 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:35 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:36.034 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:35 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:33:36.034 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:35 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:33:36.034 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:35 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:37.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:37 smithi183 ceph-mon[32584]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-04-25T14:33:37.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:37 smithi183 ceph-mon[32584]: from='client.? 172.21.15.150:0/3865686472' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "44a9e070-2282-43f1-9f05-b1f8ff81e540"}]: dispatch 2024-04-25T14:33:37.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:37 smithi183 ceph-mon[32584]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "44a9e070-2282-43f1-9f05-b1f8ff81e540"}]: dispatch 2024-04-25T14:33:37.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:37 smithi183 ceph-mon[32584]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "44a9e070-2282-43f1-9f05-b1f8ff81e540"}]': finished 2024-04-25T14:33:37.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:37 smithi183 ceph-mon[32584]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T14:33:37.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:37 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:33:37.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:37 smithi150 ceph-mon[32504]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-04-25T14:33:37.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:37 smithi150 ceph-mon[32504]: from='client.? 172.21.15.150:0/3865686472' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "44a9e070-2282-43f1-9f05-b1f8ff81e540"}]: dispatch 2024-04-25T14:33:37.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:37 smithi150 ceph-mon[32504]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "44a9e070-2282-43f1-9f05-b1f8ff81e540"}]: dispatch 2024-04-25T14:33:37.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:37 smithi150 ceph-mon[32504]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "44a9e070-2282-43f1-9f05-b1f8ff81e540"}]': finished 2024-04-25T14:33:37.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:37 smithi150 ceph-mon[32504]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T14:33:37.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:37 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:33:38.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:37 smithi053 ceph-mon[27773]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-04-25T14:33:38.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:37 smithi053 ceph-mon[27773]: from='client.? 172.21.15.150:0/3865686472' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "44a9e070-2282-43f1-9f05-b1f8ff81e540"}]: dispatch 2024-04-25T14:33:38.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:37 smithi053 ceph-mon[27773]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "44a9e070-2282-43f1-9f05-b1f8ff81e540"}]: dispatch 2024-04-25T14:33:38.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:37 smithi053 ceph-mon[27773]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "44a9e070-2282-43f1-9f05-b1f8ff81e540"}]': finished 2024-04-25T14:33:38.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:37 smithi053 ceph-mon[27773]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T14:33:38.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:37 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:33:38.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:38 smithi183 ceph-mon[32584]: from='client.? 172.21.15.150:0/169285626' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:33:38.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:38 smithi150 ceph-mon[32504]: from='client.? 172.21.15.150:0/169285626' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:33:39.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:38 smithi053 ceph-mon[27773]: from='client.? 172.21.15.150:0/169285626' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:33:39.934 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:39 smithi053 ceph-mon[27773]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-25T14:33:39.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:39 smithi183 ceph-mon[32584]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-25T14:33:39.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:39 smithi150 ceph-mon[32504]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-25T14:33:41.899 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:41 smithi150 ceph-mon[32504]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-25T14:33:41.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:41 smithi183 ceph-mon[32584]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-25T14:33:42.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:41 smithi053 ceph-mon[27773]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-25T14:33:43.875 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:43 smithi150 ceph-mon[32504]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T14:33:43.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:43 smithi183 ceph-mon[32584]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T14:33:44.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:43 smithi053 ceph-mon[27773]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T14:33:45.823 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:45 smithi150 ceph-mon[32504]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T14:33:45.823 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:45 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T14:33:45.823 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:45 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:33:45.823 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:45 smithi150 ceph-mon[32504]: Deploying daemon osd.4 on smithi150 2024-04-25T14:33:45.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:45 smithi183 ceph-mon[32584]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T14:33:45.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:45 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T14:33:45.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:45 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:33:45.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:45 smithi183 ceph-mon[32584]: Deploying daemon osd.4 on smithi150 2024-04-25T14:33:46.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:45 smithi053 ceph-mon[27773]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T14:33:46.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:45 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T14:33:46.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:45 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:33:46.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:45 smithi053 ceph-mon[27773]: Deploying daemon osd.4 on smithi150 2024-04-25T14:33:47.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:47 smithi183 ceph-mon[32584]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:33:47.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:47 smithi150 ceph-mon[32504]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:33:48.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:47 smithi053 ceph-mon[27773]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:33:49.682 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:49 smithi150 ceph-mon[32504]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:33:49.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:49 smithi183 ceph-mon[32584]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:33:50.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:49 smithi053 ceph-mon[27773]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:33:51.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:51 smithi150 ceph-mon[32504]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:33:51.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:51 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:51.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:51 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:51.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:51 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:33:51.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:51 smithi183 ceph-mon[32584]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:33:51.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:51 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:51.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:51 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:51.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:51 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:33:51.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:51 smithi053 ceph-mon[27773]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:33:51.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:51 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:51.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:51 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:51.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:51 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:33:54.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:54 smithi150 ceph-mon[32504]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:33:54.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:54 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:54.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:54 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:54.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:54 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:33:54.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:54 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:33:54.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:54 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:54.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:54.130+0000 7ff0977fe700 1 -- 172.21.15.150:0/2222621969 <== mgr.14152 v2:172.21.15.53:6800/4154933108 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff0a0062f40 con 0x7ff08c0a3cc0 2024-04-25T14:33:54.131 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 4 on host 'smithi150' 2024-04-25T14:33:54.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:54.133+0000 7ff0a877c700 1 -- 172.21.15.150:0/2222621969 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff08c0a3cc0 msgr2=0x7ff08c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:33:54.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:54.133+0000 7ff0a877c700 1 --2- 172.21.15.150:0/2222621969 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff08c0a3cc0 0x7ff08c0a6180 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff0900097b0 tx=0x7ff090006d20 comp rx=0 tx=0).stop 2024-04-25T14:33:54.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:54.133+0000 7ff0a877c700 1 -- 172.21.15.150:0/2222621969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0a0103540 msgr2=0x7ff0a0198da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:33:54.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:54.133+0000 7ff0a877c700 1 --2- 172.21.15.150:0/2222621969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0a0103540 0x7ff0a0198da0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff09c006010 tx=0x7ff09c0047b0 comp rx=0 tx=0).stop 2024-04-25T14:33:54.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:54.133+0000 7ff0a877c700 1 -- 172.21.15.150:0/2222621969 shutdown_connections 2024-04-25T14:33:54.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:54.133+0000 7ff0a877c700 1 --2- 172.21.15.150:0/2222621969 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff08c0a3cc0 0x7ff08c0a6180 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:33:54.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:54.133+0000 7ff0a877c700 1 --2- 172.21.15.150:0/2222621969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0a01087a0 0x7ff0a01a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:33:54.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:54.133+0000 7ff0a877c700 1 --2- 172.21.15.150:0/2222621969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0a0103540 0x7ff0a0198da0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:33:54.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:54.133+0000 7ff0a877c700 1 --2- 172.21.15.150:0/2222621969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0a0105e70 0x7ff0a01992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:33:54.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:54.133+0000 7ff0a877c700 1 -- 172.21.15.150:0/2222621969 >> 172.21.15.150:0/2222621969 conn(0x7ff0a00fcf70 msgr2=0x7ff0a0103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:33:54.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:54.133+0000 7ff0a877c700 1 -- 172.21.15.150:0/2222621969 shutdown_connections 2024-04-25T14:33:54.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:33:54.133+0000 7ff0a877c700 1 -- 172.21.15.150:0/2222621969 wait complete. 2024-04-25T14:33:54.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:54 smithi053 ceph-mon[27773]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:33:54.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:54 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:54.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:54 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:54.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:54 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:33:54.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:54 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:33:54.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:54 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:54.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:54 smithi183 ceph-mon[32584]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:33:54.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:54 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:54.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:54 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:54.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:54 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:33:54.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:54 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:33:54.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:54 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:54.716 DEBUG:teuthology.orchestra.run.smithi150:osd.4> sudo journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@osd.4.service 2024-04-25T14:33:54.721 INFO:tasks.cephadm:Deploying osd.5 on smithi183 with /dev/vg_nvme/lv_3... 2024-04-25T14:33:54.721 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T14:33:54.969 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:33:55.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:55 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:55.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:55 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:55.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:55 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:33:55.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:55 smithi150 ceph-mon[32504]: from='osd.4 [v2:172.21.15.150:6808/1526850103,v1:172.21.15.150:6809/1526850103]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T14:33:55.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:55 smithi150 ceph-mon[32504]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T14:33:55.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:55 smithi150 ceph-mon[32504]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:33:55.436 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:55 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:55.436 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:55 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:55.436 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:55 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:33:55.436 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:55 smithi183 ceph-mon[32584]: from='osd.4 [v2:172.21.15.150:6808/1526850103,v1:172.21.15.150:6809/1526850103]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T14:33:55.436 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:55 smithi183 ceph-mon[32584]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T14:33:55.437 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:55 smithi183 ceph-mon[32584]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:33:55.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:55 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:55.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:55 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:55.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:55 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:33:55.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:55 smithi053 ceph-mon[27773]: from='osd.4 [v2:172.21.15.150:6808/1526850103,v1:172.21.15.150:6809/1526850103]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T14:33:55.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:55 smithi053 ceph-mon[27773]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T14:33:55.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:55 smithi053 ceph-mon[27773]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:33:56.399 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:56 smithi183 ceph-mon[32584]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T14:33:56.399 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:56 smithi183 ceph-mon[32584]: from='osd.4 [v2:172.21.15.150:6808/1526850103,v1:172.21.15.150:6809/1526850103]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T14:33:56.399 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:56 smithi183 ceph-mon[32584]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T14:33:56.399 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:56 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:33:56.399 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:56 smithi183 ceph-mon[32584]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T14:33:56.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:56 smithi150 ceph-mon[32504]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T14:33:56.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:56 smithi150 ceph-mon[32504]: from='osd.4 [v2:172.21.15.150:6808/1526850103,v1:172.21.15.150:6809/1526850103]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T14:33:56.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:56 smithi150 ceph-mon[32504]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T14:33:56.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:56 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:33:56.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:56 smithi150 ceph-mon[32504]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T14:33:56.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:56 smithi053 ceph-mon[27773]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T14:33:56.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:56 smithi053 ceph-mon[27773]: from='osd.4 [v2:172.21.15.150:6808/1526850103,v1:172.21.15.150:6809/1526850103]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T14:33:56.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:56 smithi053 ceph-mon[27773]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T14:33:56.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:56 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:33:56.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:56 smithi053 ceph-mon[27773]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T14:33:56.941 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T14:33:56.941 INFO:teuthology.orchestra.run.smithi183.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T14:33:56.941 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10+0 records in 2024-04-25T14:33:56.941 INFO:teuthology.orchestra.run.smithi183.stderr:10+0 records out 2024-04-25T14:33:56.941 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0133172 s, 787 MB/s 2024-04-25T14:33:56.941 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping successful for: 2024-04-25T14:33:57.133 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 25 14:33:57 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-4[43891]: 2024-04-25T14:33:57.031+0000 7f6b918e3700 -1 osd.4 0 waiting for initial osdmap 2024-04-25T14:33:57.310 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:57 smithi183 ceph-mon[32584]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-25T14:33:57.310 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:57 smithi183 ceph-mon[32584]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T14:33:57.310 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:57 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:33:57.310 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:57 smithi183 ceph-mon[32584]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:33:57.310 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:57 smithi183 ceph-mon[32584]: from='osd.4 ' entity='osd.4' 2024-04-25T14:33:57.310 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:57 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:33:57.459 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 25 14:33:57 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-4[43891]: 2024-04-25T14:33:57.140+0000 7f6b896ce700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:33:57.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:57 smithi150 ceph-mon[32504]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-25T14:33:57.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:57 smithi150 ceph-mon[32504]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T14:33:57.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:57 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:33:57.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:57 smithi150 ceph-mon[32504]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:33:57.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:57 smithi150 ceph-mon[32504]: from='osd.4 ' entity='osd.4' 2024-04-25T14:33:57.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:57 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:33:57.519 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_3 2024-04-25T14:33:57.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:57 smithi053 ceph-mon[27773]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-25T14:33:57.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:57 smithi053 ceph-mon[27773]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T14:33:57.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:57 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:33:57.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:57 smithi053 ceph-mon[27773]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:33:57.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:57 smithi053 ceph-mon[27773]: from='osd.4 ' entity='osd.4' 2024-04-25T14:33:57.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:57 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:33:57.770 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:33:58.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:58 smithi183 ceph-mon[32584]: purged_snaps scrub starts 2024-04-25T14:33:58.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:58 smithi183 ceph-mon[32584]: purged_snaps scrub ok 2024-04-25T14:33:58.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:58 smithi183 ceph-mon[32584]: Detected new or changed devices on smithi150 2024-04-25T14:33:58.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:58 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:58.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:58 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:58.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:58 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:33:58.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:58 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:33:58.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:58 smithi183 ceph-mon[32584]: Adjusting osd_memory_target on smithi150 to 8544M 2024-04-25T14:33:58.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:58 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:58.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:58 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:33:58.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:58 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:33:58.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:58 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:58.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:58 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:33:58.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:58 smithi150 ceph-mon[32504]: purged_snaps scrub starts 2024-04-25T14:33:58.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:58 smithi150 ceph-mon[32504]: purged_snaps scrub ok 2024-04-25T14:33:58.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:58 smithi150 ceph-mon[32504]: Detected new or changed devices on smithi150 2024-04-25T14:33:58.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:58 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:58.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:58 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:58.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:58 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:33:58.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:58 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:33:58.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:58 smithi150 ceph-mon[32504]: Adjusting osd_memory_target on smithi150 to 8544M 2024-04-25T14:33:58.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:58 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:58.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:58 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:33:58.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:58 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:33:58.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:58 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:58.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:58 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:33:58.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:58 smithi053 ceph-mon[27773]: purged_snaps scrub starts 2024-04-25T14:33:58.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:58 smithi053 ceph-mon[27773]: purged_snaps scrub ok 2024-04-25T14:33:58.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:58 smithi053 ceph-mon[27773]: Detected new or changed devices on smithi150 2024-04-25T14:33:58.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:58 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:58.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:58 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:58.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:58 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:33:58.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:58 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:33:58.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:58 smithi053 ceph-mon[27773]: Adjusting osd_memory_target on smithi150 to 8544M 2024-04-25T14:33:58.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:58 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:58.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:58 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:33:58.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:58 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:33:58.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:58 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:33:58.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:58 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:33:59.285 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:33:59.284+0000 7f20b2035700 1 -- 172.21.15.183:0/3324473487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20ac102ba0 msgr2=0x7f20ac103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:33:59.285 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:33:59.284+0000 7f20b2035700 1 --2- 172.21.15.183:0/3324473487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20ac102ba0 0x7f20ac103000 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f20a0009990 tx=0x7f20a0009ca0 comp rx=0 tx=0).stop 2024-04-25T14:33:59.285 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:33:59.284+0000 7f20b2035700 1 -- 172.21.15.183:0/3324473487 shutdown_connections 2024-04-25T14:33:59.285 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:33:59.284+0000 7f20b2035700 1 --2- 172.21.15.183:0/3324473487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20ac103540 0x7f20ac109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:33:59.285 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:33:59.284+0000 7f20b2035700 1 --2- 172.21.15.183:0/3324473487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20ac1019a0 0x7f20ac101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:33:59.285 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:33:59.284+0000 7f20b2035700 1 --2- 172.21.15.183:0/3324473487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20ac102ba0 0x7f20ac103000 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:33:59.285 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:33:59.284+0000 7f20b2035700 1 -- 172.21.15.183:0/3324473487 >> 172.21.15.183:0/3324473487 conn(0x7f20ac0fcf40 msgr2=0x7f20ac0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:33:59.285 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:33:59.284+0000 7f20b2035700 1 -- 172.21.15.183:0/3324473487 shutdown_connections 2024-04-25T14:33:59.285 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:33:59.285+0000 7f20b2035700 1 -- 172.21.15.183:0/3324473487 wait complete. 2024-04-25T14:33:59.285 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:33:59.285+0000 7f20b2035700 1 Processor -- start 2024-04-25T14:33:59.287 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:33:59.285+0000 7f20b2035700 1 -- start start 2024-04-25T14:33:59.287 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:33:59.286+0000 7f20b2035700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20ac1019a0 0x7f20ac06b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:33:59.287 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:33:59.286+0000 7f20b2035700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20ac102ba0 0x7f20ac19da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:33:59.287 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:33:59.286+0000 7f20b2035700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20ac103540 0x7f20ac1a0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:33:59.287 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:33:59.286+0000 7f20b2035700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f20ac10d4b0 con 0x7f20ac103540 2024-04-25T14:33:59.288 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:33:59.286+0000 7f20b2035700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f20ac10d330 con 0x7f20ac1019a0 2024-04-25T14:33:59.288 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:33:59.286+0000 7f20b2035700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f20ac10d630 con 0x7f20ac102ba0 2024-04-25T14:33:59.288 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:33:59.286+0000 7f20aaffd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20ac102ba0 0x7f20ac19da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:33:59.288 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:33:59.286+0000 7f20aaffd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20ac102ba0 0x7f20ac19da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:41718/0 (socket says 172.21.15.183:41718) 2024-04-25T14:33:59.288 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:33:59.286+0000 7f20aaffd700 1 -- 172.21.15.183:0/2113646517 learned_addr learned my addr 172.21.15.183:0/2113646517 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-04-25T14:33:59.288 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:33:59.286+0000 7f20ab7fe700 1 --2- 172.21.15.183:0/2113646517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20ac1019a0 0x7f20ac06b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:33:59.288 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:33:59.286+0000 7f20aaffd700 1 -- 172.21.15.183:0/2113646517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20ac1019a0 msgr2=0x7f20ac06b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:33:59.288 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:33:59.286+0000 7f20aaffd700 1 --2- 172.21.15.183:0/2113646517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20ac1019a0 0x7f20ac06b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:33:59.288 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:33:59.286+0000 7f20aaffd700 1 -- 172.21.15.183:0/2113646517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20ac103540 msgr2=0x7f20ac1a0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:33:59.288 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:33:59.286+0000 7f20aaffd700 1 --2- 172.21.15.183:0/2113646517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20ac103540 0x7f20ac1a0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:33:59.288 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:33:59.286+0000 7f20aaffd700 1 -- 172.21.15.183:0/2113646517 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20ac1a0930 con 0x7f20ac102ba0 2024-04-25T14:33:59.288 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:33:59.286+0000 7f20aaffd700 1 --2- 172.21.15.183:0/2113646517 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20ac102ba0 0x7f20ac19da20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f20a0006010 tx=0x7f20a00047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:33:59.288 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:33:59.286+0000 7f20a8ff9700 1 -- 172.21.15.183:0/2113646517 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a0027070 con 0x7f20ac102ba0 2024-04-25T14:33:59.288 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:33:59.286+0000 7f20a8ff9700 1 -- 172.21.15.183:0/2113646517 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20a00041c0 con 0x7f20ac102ba0 2024-04-25T14:33:59.289 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:33:59.286+0000 7f20b2035700 1 -- 172.21.15.183:0/2113646517 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20ac1a0bc0 con 0x7f20ac102ba0 2024-04-25T14:33:59.289 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:33:59.286+0000 7f20a8ff9700 1 -- 172.21.15.183:0/2113646517 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a000f460 con 0x7f20ac102ba0 2024-04-25T14:33:59.289 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:33:59.286+0000 7f20b2035700 1 -- 172.21.15.183:0/2113646517 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20ac1a1010 con 0x7f20ac102ba0 2024-04-25T14:33:59.289 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:33:59.287+0000 7f20b2035700 1 -- 172.21.15.183:0/2113646517 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20ac069730 con 0x7f20ac102ba0 2024-04-25T14:33:59.289 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:33:59.288+0000 7f20a8ff9700 1 -- 172.21.15.183:0/2113646517 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f20a002a070 con 0x7f20ac102ba0 2024-04-25T14:33:59.290 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:33:59.289+0000 7f20a8ff9700 1 --2- 172.21.15.183:0/2113646517 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f20940a3c40 0x7f20940a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:33:59.290 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:33:59.289+0000 7f20a8ff9700 1 -- 172.21.15.183:0/2113646517 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4101+0+0 (secure 0 0 0) 0x7f20a00d0260 con 0x7f20ac102ba0 2024-04-25T14:33:59.290 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:33:59.290+0000 7f20ab7fe700 1 --2- 172.21.15.183:0/2113646517 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f20940a3c40 0x7f20940a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:33:59.290 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:33:59.291+0000 7f20ab7fe700 1 --2- 172.21.15.183:0/2113646517 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f20940a3c40 0x7f20940a6100 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f209c005950 tx=0x7f209c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:33:59.294 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:33:59.294+0000 7f20a8ff9700 1 -- 172.21.15.183:0/2113646517 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f20a0013070 con 0x7f20ac102ba0 2024-04-25T14:33:59.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:59 smithi183 ceph-mon[32584]: osd.4 [v2:172.21.15.150:6808/1526850103,v1:172.21.15.150:6809/1526850103] boot 2024-04-25T14:33:59.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:59 smithi183 ceph-mon[32584]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T14:33:59.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:59 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:33:59.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:33:59 smithi183 ceph-mon[32584]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:33:59.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:59 smithi150 ceph-mon[32504]: osd.4 [v2:172.21.15.150:6808/1526850103,v1:172.21.15.150:6809/1526850103] boot 2024-04-25T14:33:59.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:59 smithi150 ceph-mon[32504]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T14:33:59.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:59 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:33:59.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:33:59 smithi150 ceph-mon[32504]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:33:59.505 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:33:59.505+0000 7f20b2035700 1 -- 172.21.15.183:0/2113646517 --> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f20ac062e80 con 0x7f20940a3c40 2024-04-25T14:33:59.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:59 smithi053 ceph-mon[27773]: osd.4 [v2:172.21.15.150:6808/1526850103,v1:172.21.15.150:6809/1526850103] boot 2024-04-25T14:33:59.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:59 smithi053 ceph-mon[27773]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T14:33:59.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:59 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:33:59.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:33:59 smithi053 ceph-mon[27773]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:34:00.434 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:00 smithi183 ceph-mon[32584]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T14:34:00.434 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:00 smithi183 ceph-mon[32584]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:34:00.434 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:00 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:34:00.435 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:00 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:34:00.435 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:00 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:34:00.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:00 smithi150 ceph-mon[32504]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T14:34:00.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:00 smithi150 ceph-mon[32504]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:34:00.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:00 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:34:00.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:00 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:34:00.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:00 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:34:00.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:00 smithi053 ceph-mon[27773]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T14:34:00.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:00 smithi053 ceph-mon[27773]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:34:00.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:00 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:34:00.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:00 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:34:00.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:00 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:34:01.306 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:01 smithi183 ceph-mon[32584]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:34:01.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:01 smithi150 ceph-mon[32504]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:34:01.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:01 smithi053 ceph-mon[27773]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:34:02.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:02 smithi183 ceph-mon[32584]: from='client.? 172.21.15.183:0/2163409147' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c5619d2-c0f9-47d6-bf7e-2f4afd1b09f0"}]: dispatch 2024-04-25T14:34:02.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:02 smithi183 ceph-mon[32584]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c5619d2-c0f9-47d6-bf7e-2f4afd1b09f0"}]: dispatch 2024-04-25T14:34:02.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:02 smithi183 ceph-mon[32584]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c5619d2-c0f9-47d6-bf7e-2f4afd1b09f0"}]': finished 2024-04-25T14:34:02.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:02 smithi183 ceph-mon[32584]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T14:34:02.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:02 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:34:02.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:02 smithi150 ceph-mon[32504]: from='client.? 172.21.15.183:0/2163409147' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c5619d2-c0f9-47d6-bf7e-2f4afd1b09f0"}]: dispatch 2024-04-25T14:34:02.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:02 smithi150 ceph-mon[32504]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c5619d2-c0f9-47d6-bf7e-2f4afd1b09f0"}]: dispatch 2024-04-25T14:34:02.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:02 smithi150 ceph-mon[32504]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c5619d2-c0f9-47d6-bf7e-2f4afd1b09f0"}]': finished 2024-04-25T14:34:02.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:02 smithi150 ceph-mon[32504]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T14:34:02.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:02 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:34:02.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:02 smithi053 ceph-mon[27773]: from='client.? 172.21.15.183:0/2163409147' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c5619d2-c0f9-47d6-bf7e-2f4afd1b09f0"}]: dispatch 2024-04-25T14:34:02.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:02 smithi053 ceph-mon[27773]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c5619d2-c0f9-47d6-bf7e-2f4afd1b09f0"}]: dispatch 2024-04-25T14:34:02.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:02 smithi053 ceph-mon[27773]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c5619d2-c0f9-47d6-bf7e-2f4afd1b09f0"}]': finished 2024-04-25T14:34:02.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:02 smithi053 ceph-mon[27773]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T14:34:02.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:02 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:34:03.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:03 smithi150 ceph-mon[32504]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:34:03.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:03 smithi150 ceph-mon[32504]: from='client.? 172.21.15.183:0/3489906763' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:34:03.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:03 smithi053 ceph-mon[27773]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:34:03.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:03 smithi053 ceph-mon[27773]: from='client.? 172.21.15.183:0/3489906763' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:34:03.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:03 smithi183 ceph-mon[32584]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:34:03.706 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:03 smithi183 ceph-mon[32584]: from='client.? 172.21.15.183:0/3489906763' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:34:05.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:05 smithi183 ceph-mon[32584]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:34:05.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:05 smithi150 ceph-mon[32504]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:34:05.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:05 smithi053 ceph-mon[27773]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:34:07.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:07 smithi183 ceph-mon[32584]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:34:07.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:07 smithi150 ceph-mon[32504]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:34:07.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:07 smithi053 ceph-mon[27773]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:34:09.585 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:09 smithi053 ceph-mon[27773]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:34:09.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:09 smithi183 ceph-mon[32584]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:34:09.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:09 smithi150 ceph-mon[32504]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:34:10.443 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:10 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T14:34:10.443 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:10 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:34:10.443 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:10 smithi183 ceph-mon[32584]: Deploying daemon osd.5 on smithi183 2024-04-25T14:34:10.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:10 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T14:34:10.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:10 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:34:10.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:10 smithi150 ceph-mon[32504]: Deploying daemon osd.5 on smithi183 2024-04-25T14:34:10.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:10 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T14:34:10.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:10 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:34:10.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:10 smithi053 ceph-mon[27773]: Deploying daemon osd.5 on smithi183 2024-04-25T14:34:11.532 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:11 smithi183 ceph-mon[32584]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:34:11.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:11 smithi150 ceph-mon[32504]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:34:11.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:11 smithi053 ceph-mon[27773]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:34:13.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:13 smithi183 ceph-mon[32584]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:34:13.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:13 smithi150 ceph-mon[32504]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:34:13.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:13 smithi053 ceph-mon[27773]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:34:16.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:15 smithi183 ceph-mon[32584]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:34:16.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:15 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:16.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:15 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:34:16.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:15 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:16.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:15 smithi150 ceph-mon[32504]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:34:16.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:15 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:16.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:15 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:34:16.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:15 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:16.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:15 smithi053 ceph-mon[27773]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:34:16.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:15 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:16.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:15 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:34:16.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:15 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:17.988 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:34:17.988+0000 7f20a8ff9700 1 -- 172.21.15.183:0/2113646517 <== mgr.14152 v2:172.21.15.53:6800/4154933108 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f20ac062e80 con 0x7f20940a3c40 2024-04-25T14:34:17.988 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 5 on host 'smithi183' 2024-04-25T14:34:17.991 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:34:17.990+0000 7f20b2035700 1 -- 172.21.15.183:0/2113646517 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f20940a3c40 msgr2=0x7f20940a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:34:17.991 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:34:17.990+0000 7f20b2035700 1 --2- 172.21.15.183:0/2113646517 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f20940a3c40 0x7f20940a6100 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f209c005950 tx=0x7f209c0058e0 comp rx=0 tx=0).stop 2024-04-25T14:34:17.991 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:34:17.990+0000 7f20b2035700 1 -- 172.21.15.183:0/2113646517 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20ac102ba0 msgr2=0x7f20ac19da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:34:17.991 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:34:17.990+0000 7f20b2035700 1 --2- 172.21.15.183:0/2113646517 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20ac102ba0 0x7f20ac19da20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f20a0006010 tx=0x7f20a00047b0 comp rx=0 tx=0).stop 2024-04-25T14:34:17.991 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:34:17.990+0000 7f20b2035700 1 -- 172.21.15.183:0/2113646517 shutdown_connections 2024-04-25T14:34:17.991 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:34:17.990+0000 7f20b2035700 1 --2- 172.21.15.183:0/2113646517 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f20940a3c40 0x7f20940a6100 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:34:17.991 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:34:17.990+0000 7f20b2035700 1 --2- 172.21.15.183:0/2113646517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20ac103540 0x7f20ac1a0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:34:17.991 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:34:17.990+0000 7f20b2035700 1 --2- 172.21.15.183:0/2113646517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20ac1019a0 0x7f20ac06b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:34:17.991 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:34:17.990+0000 7f20b2035700 1 --2- 172.21.15.183:0/2113646517 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20ac102ba0 0x7f20ac19da20 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:34:17.992 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:34:17.990+0000 7f20b2035700 1 -- 172.21.15.183:0/2113646517 >> 172.21.15.183:0/2113646517 conn(0x7f20ac0fcf40 msgr2=0x7f20ac105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:34:17.992 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:34:17.990+0000 7f20b2035700 1 -- 172.21.15.183:0/2113646517 shutdown_connections 2024-04-25T14:34:17.992 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:34:17.990+0000 7f20b2035700 1 -- 172.21.15.183:0/2113646517 wait complete. 2024-04-25T14:34:18.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:17 smithi183 ceph-mon[32584]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:34:18.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:17 smithi150 ceph-mon[32504]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:34:18.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:17 smithi053 ceph-mon[27773]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:34:19.017 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:18 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:19.017 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:18 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:19.017 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:18 smithi183 ceph-mon[32584]: from='osd.5 [v2:172.21.15.183:6808/773726576,v1:172.21.15.183:6809/773726576]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T14:34:19.018 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:18 smithi183 ceph-mon[32584]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T14:34:19.019 DEBUG:teuthology.orchestra.run.smithi183:osd.5> sudo journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@osd.5.service 2024-04-25T14:34:19.021 INFO:tasks.cephadm:Deploying osd.6 on smithi053 with /dev/vg_nvme/lv_2... 2024-04-25T14:34:19.022 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T14:34:19.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:18 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:19.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:18 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:19.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:18 smithi150 ceph-mon[32504]: from='osd.5 [v2:172.21.15.183:6808/773726576,v1:172.21.15.183:6809/773726576]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T14:34:19.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:18 smithi150 ceph-mon[32504]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T14:34:19.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:18 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:19.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:18 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:19.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:18 smithi053 ceph-mon[27773]: from='osd.5 [v2:172.21.15.183:6808/773726576,v1:172.21.15.183:6809/773726576]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T14:34:19.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:18 smithi053 ceph-mon[27773]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T14:34:19.472 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:34:20.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:19 smithi053 ceph-mon[27773]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:34:20.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:19 smithi053 ceph-mon[27773]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T14:34:20.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:19 smithi053 ceph-mon[27773]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T14:34:20.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:19 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:34:20.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:19 smithi053 ceph-mon[27773]: from='osd.5 [v2:172.21.15.183:6808/773726576,v1:172.21.15.183:6809/773726576]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T14:34:20.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:19 smithi053 ceph-mon[27773]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T14:34:20.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:19 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:20.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:19 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:20.309 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:19 smithi150 ceph-mon[32504]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:34:20.309 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:19 smithi150 ceph-mon[32504]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T14:34:20.309 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:19 smithi150 ceph-mon[32504]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T14:34:20.309 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:19 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:34:20.309 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:19 smithi150 ceph-mon[32504]: from='osd.5 [v2:172.21.15.183:6808/773726576,v1:172.21.15.183:6809/773726576]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T14:34:20.309 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:19 smithi150 ceph-mon[32504]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T14:34:20.309 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:19 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:20.309 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:19 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:20.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:19 smithi183 ceph-mon[32584]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:34:20.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:19 smithi183 ceph-mon[32584]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T14:34:20.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:19 smithi183 ceph-mon[32584]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T14:34:20.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:19 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:34:20.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:19 smithi183 ceph-mon[32584]: from='osd.5 [v2:172.21.15.183:6808/773726576,v1:172.21.15.183:6809/773726576]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T14:34:20.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:19 smithi183 ceph-mon[32584]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T14:34:20.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:19 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:20.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:19 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:20.929 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:34:20 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5[44266]: 2024-04-25T14:34:20.873+0000 7f2f2b0ed700 -1 osd.5 0 waiting for initial osdmap 2024-04-25T14:34:21.203 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:34:21 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5[44266]: 2024-04-25T14:34:21.024+0000 7f2f226d7700 -1 osd.5 39 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:34:21.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:20 smithi183 ceph-mon[32584]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-25T14:34:21.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:20 smithi183 ceph-mon[32584]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T14:34:21.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:20 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:34:21.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:20 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:34:21.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:20 smithi183 ceph-mon[32584]: osdmap e39: 6 total, 5 up, 6 in 2024-04-25T14:34:21.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:20 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:34:21.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:20 smithi183 ceph-mon[32584]: from='osd.5 ' entity='osd.5' 2024-04-25T14:34:21.205 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:20 smithi053 ceph-mon[27773]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-25T14:34:21.205 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:20 smithi053 ceph-mon[27773]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T14:34:21.205 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:20 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:34:21.205 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:20 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:34:21.205 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:20 smithi053 ceph-mon[27773]: osdmap e39: 6 total, 5 up, 6 in 2024-04-25T14:34:21.205 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:20 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:34:21.205 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:20 smithi053 ceph-mon[27773]: from='osd.5 ' entity='osd.5' 2024-04-25T14:34:21.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:20 smithi150 ceph-mon[32504]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-25T14:34:21.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:20 smithi150 ceph-mon[32504]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T14:34:21.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:34:21.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:34:21.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:20 smithi150 ceph-mon[32504]: osdmap e39: 6 total, 5 up, 6 in 2024-04-25T14:34:21.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:34:21.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:20 smithi150 ceph-mon[32504]: from='osd.5 ' entity='osd.5' 2024-04-25T14:34:22.076 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:21 smithi183 ceph-mon[32584]: purged_snaps scrub starts 2024-04-25T14:34:22.077 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:21 smithi183 ceph-mon[32584]: purged_snaps scrub ok 2024-04-25T14:34:22.077 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:21 smithi183 ceph-mon[32584]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:34:22.077 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:21 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:34:22.077 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:21 smithi183 ceph-mon[32584]: osd.5 [v2:172.21.15.183:6808/773726576,v1:172.21.15.183:6809/773726576] boot 2024-04-25T14:34:22.077 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:21 smithi183 ceph-mon[32584]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T14:34:22.077 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:21 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:34:22.118 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:21 smithi053 ceph-mon[27773]: purged_snaps scrub starts 2024-04-25T14:34:22.118 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:21 smithi053 ceph-mon[27773]: purged_snaps scrub ok 2024-04-25T14:34:22.118 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:21 smithi053 ceph-mon[27773]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:34:22.118 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:21 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:34:22.118 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:21 smithi053 ceph-mon[27773]: osd.5 [v2:172.21.15.183:6808/773726576,v1:172.21.15.183:6809/773726576] boot 2024-04-25T14:34:22.118 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:21 smithi053 ceph-mon[27773]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T14:34:22.119 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:21 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:34:22.441 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T14:34:22.442 INFO:teuthology.orchestra.run.smithi053.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T14:34:22.442 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10+0 records in 2024-04-25T14:34:22.442 INFO:teuthology.orchestra.run.smithi053.stderr:10+0 records out 2024-04-25T14:34:22.442 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0626082 s, 167 MB/s 2024-04-25T14:34:22.442 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping successful for: 2024-04-25T14:34:22.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:21 smithi150 ceph-mon[32504]: purged_snaps scrub starts 2024-04-25T14:34:22.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:21 smithi150 ceph-mon[32504]: purged_snaps scrub ok 2024-04-25T14:34:22.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:21 smithi150 ceph-mon[32504]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:34:22.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:21 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:34:22.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:21 smithi150 ceph-mon[32504]: osd.5 [v2:172.21.15.183:6808/773726576,v1:172.21.15.183:6809/773726576] boot 2024-04-25T14:34:22.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:21 smithi150 ceph-mon[32504]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T14:34:22.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:21 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:34:23.491 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi053:vg_nvme/lv_2 2024-04-25T14:34:23.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:23 smithi183 ceph-mon[32584]: pgmap v154: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:34:23.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:23 smithi183 ceph-mon[32584]: Detected new or changed devices on smithi183 2024-04-25T14:34:23.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:23 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:23.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:23 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:23.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:23 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:34:23.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:23 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:34:23.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:23 smithi183 ceph-mon[32584]: Adjusting osd_memory_target on smithi183 to 8544M 2024-04-25T14:34:23.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:23 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:23.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:23 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:34:23.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:23 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:34:23.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:23 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:23.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:23 smithi183 ceph-mon[32584]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T14:34:23.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:23 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:34:23.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:23 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:34:23.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:23 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:34:23.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:23 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:23.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:23 smithi150 ceph-mon[32504]: pgmap v154: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:34:23.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:23 smithi150 ceph-mon[32504]: Detected new or changed devices on smithi183 2024-04-25T14:34:23.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:23 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:23.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:23 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:23.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:23 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:34:23.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:23 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:34:23.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:23 smithi150 ceph-mon[32504]: Adjusting osd_memory_target on smithi183 to 8544M 2024-04-25T14:34:23.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:23 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:23.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:23 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:34:23.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:23 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:34:23.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:23 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:23.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:23 smithi150 ceph-mon[32504]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T14:34:23.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:23 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:34:23.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:23 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:34:23.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:23 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:34:23.712 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:23 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:23.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:23 smithi053 ceph-mon[27773]: pgmap v154: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:34:23.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:23 smithi053 ceph-mon[27773]: Detected new or changed devices on smithi183 2024-04-25T14:34:23.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:23 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:23.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:23 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:23.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:23 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:34:23.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:23 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:34:23.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:23 smithi053 ceph-mon[27773]: Adjusting osd_memory_target on smithi183 to 8544M 2024-04-25T14:34:23.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:23 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:23.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:23 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:34:23.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:23 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:34:23.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:23 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:23.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:23 smithi053 ceph-mon[27773]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T14:34:23.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:23 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:34:23.784 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:23 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:34:23.784 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:23 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:34:23.784 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:23 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:23.954 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:34:24.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:24 smithi053 ceph-mon[27773]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T14:34:24.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:24 smithi183 ceph-mon[32584]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T14:34:24.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:24 smithi150 ceph-mon[32504]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T14:34:25.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:25 smithi053 ceph-mon[27773]: pgmap v157: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:34:25.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:25 smithi053 ceph-mon[27773]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T14:34:25.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:25 smithi183 ceph-mon[32584]: pgmap v157: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:34:25.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:25 smithi183 ceph-mon[32584]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T14:34:25.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:25 smithi150 ceph-mon[32504]: pgmap v157: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:34:25.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:25 smithi150 ceph-mon[32504]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T14:34:26.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:26.232+0000 7f885be00700 1 -- 172.21.15.53:0/288721716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f885c0773f0 msgr2=0x7f885c077810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:34:26.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:26.232+0000 7f885be00700 1 --2- 172.21.15.53:0/288721716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f885c0773f0 0x7f885c077810 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f884c009990 tx=0x7f884c009ca0 comp rx=0 tx=0).stop 2024-04-25T14:34:26.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:26.233+0000 7f885be00700 1 -- 172.21.15.53:0/288721716 shutdown_connections 2024-04-25T14:34:26.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:26.233+0000 7f885be00700 1 --2- 172.21.15.53:0/288721716 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f885c079000 0x7f885c079880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:34:26.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:26.233+0000 7f885be00700 1 --2- 172.21.15.53:0/288721716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f885c0773f0 0x7f885c077810 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:34:26.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:26.233+0000 7f885be00700 1 --2- 172.21.15.53:0/288721716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f885c078640 0x7f885c078ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:34:26.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:26.233+0000 7f885be00700 1 -- 172.21.15.53:0/288721716 >> 172.21.15.53:0/288721716 conn(0x7f885c100180 msgr2=0x7f885c1025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:34:26.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:26.233+0000 7f885be00700 1 -- 172.21.15.53:0/288721716 shutdown_connections 2024-04-25T14:34:26.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:26.233+0000 7f885be00700 1 -- 172.21.15.53:0/288721716 wait complete. 2024-04-25T14:34:26.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:26.234+0000 7f885be00700 1 Processor -- start 2024-04-25T14:34:26.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:26.234+0000 7f885be00700 1 -- start start 2024-04-25T14:34:26.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:26.234+0000 7f885be00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f885c0773f0 0x7f885c19f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:34:26.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:26.234+0000 7f885be00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f885c078640 0x7f885c19f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:34:26.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:26.234+0000 7f885be00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f885c079000 0x7f885c1a6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:34:26.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:26.234+0000 7f885be00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f885c114a70 con 0x7f885c0773f0 2024-04-25T14:34:26.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:26.234+0000 7f885be00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f885c1148f0 con 0x7f885c079000 2024-04-25T14:34:26.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:26.234+0000 7f885be00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f885c114bf0 con 0x7f885c078640 2024-04-25T14:34:26.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:26.235+0000 7f885aa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f885c0773f0 0x7f885c19f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:34:26.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:26.235+0000 7f885aa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f885c0773f0 0x7f885c19f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48016/0 (socket says 172.21.15.53:48016) 2024-04-25T14:34:26.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:26.235+0000 7f885aa00700 1 -- 172.21.15.53:0/1951639696 learned_addr learned my addr 172.21.15.53:0/1951639696 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:34:26.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:26.235+0000 7f885a000700 1 --2- 172.21.15.53:0/1951639696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f885c078640 0x7f885c19f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:34:26.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:26.235+0000 7f885b400700 1 --2- 172.21.15.53:0/1951639696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f885c079000 0x7f885c1a6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:34:26.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:26.235+0000 7f885aa00700 1 -- 172.21.15.53:0/1951639696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f885c078640 msgr2=0x7f885c19f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:34:26.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:26.235+0000 7f885aa00700 1 --2- 172.21.15.53:0/1951639696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f885c078640 0x7f885c19f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:34:26.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:26.235+0000 7f885aa00700 1 -- 172.21.15.53:0/1951639696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f885c079000 msgr2=0x7f885c1a6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:34:26.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:26.235+0000 7f885aa00700 1 --2- 172.21.15.53:0/1951639696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f885c079000 0x7f885c1a6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:34:26.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:26.235+0000 7f885aa00700 1 -- 172.21.15.53:0/1951639696 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f885c1a6d10 con 0x7f885c0773f0 2024-04-25T14:34:26.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:26.235+0000 7f885b400700 1 --2- 172.21.15.53:0/1951639696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f885c079000 0x7f885c1a6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:34:26.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:26.236+0000 7f885aa00700 1 --2- 172.21.15.53:0/1951639696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f885c0773f0 0x7f885c19f0b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f884c000c00 tx=0x7f884c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:34:26.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:26.236+0000 7f8860a00700 1 -- 172.21.15.53:0/1951639696 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884c027070 con 0x7f885c0773f0 2024-04-25T14:34:26.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:26.236+0000 7f8860a00700 1 -- 172.21.15.53:0/1951639696 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f884c022470 con 0x7f885c0773f0 2024-04-25T14:34:26.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:26.236+0000 7f8860a00700 1 -- 172.21.15.53:0/1951639696 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884c017650 con 0x7f885c0773f0 2024-04-25T14:34:26.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:26.236+0000 7f885be00700 1 -- 172.21.15.53:0/1951639696 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f885c1a6fa0 con 0x7f885c0773f0 2024-04-25T14:34:26.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:26.236+0000 7f885be00700 1 -- 172.21.15.53:0/1951639696 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f885c1a7340 con 0x7f885c0773f0 2024-04-25T14:34:26.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:26.238+0000 7f885be00700 1 -- 172.21.15.53:0/1951639696 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f885c069730 con 0x7f885c0773f0 2024-04-25T14:34:26.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:26.239+0000 7f8860a00700 1 -- 172.21.15.53:0/1951639696 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f884c02a070 con 0x7f885c0773f0 2024-04-25T14:34:26.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:26.241+0000 7f8860a00700 1 --2- 172.21.15.53:0/1951639696 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f88400a3c40 0x7f88400a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:34:26.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:26.241+0000 7f8860a00700 1 -- 172.21.15.53:0/1951639696 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4533+0+0 (secure 0 0 0) 0x7f884c0d0380 con 0x7f885c0773f0 2024-04-25T14:34:26.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:26.241+0000 7f885a000700 1 --2- 172.21.15.53:0/1951639696 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f88400a3c40 0x7f88400a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:34:26.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:26.242+0000 7f885a000700 1 --2- 172.21.15.53:0/1951639696 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f88400a3c40 0x7f88400a6100 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8848009c60 tx=0x7f8848009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:34:26.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:26.246+0000 7f8860a00700 1 -- 172.21.15.53:0/1951639696 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f884c013070 con 0x7f885c0773f0 2024-04-25T14:34:26.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:26.485+0000 7f885be00700 1 -- 172.21.15.53:0/1951639696 --> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f885c062e80 con 0x7f88400a3c40 2024-04-25T14:34:26.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:26 smithi053 ceph-mon[27773]: osdmap e44: 6 total, 6 up, 6 in 2024-04-25T14:34:26.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:26 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:34:26.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:26 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:34:26.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:26 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:34:26.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:26 smithi183 ceph-mon[32584]: osdmap e44: 6 total, 6 up, 6 in 2024-04-25T14:34:26.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:26 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:34:26.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:26 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:34:26.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:26 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:34:26.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:26 smithi150 ceph-mon[32504]: osdmap e44: 6 total, 6 up, 6 in 2024-04-25T14:34:26.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:26 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:34:26.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:26 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:34:26.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:26 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:34:27.726 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:27 smithi053 ceph-mon[27773]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T14:34:27.726 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:27 smithi053 ceph-mon[27773]: from='client.14271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:34:27.855 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:27 smithi183 ceph-mon[32584]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T14:34:27.855 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:27 smithi183 ceph-mon[32584]: from='client.14271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:34:27.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:27 smithi150 ceph-mon[32504]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T14:34:27.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:27 smithi150 ceph-mon[32504]: from='client.14271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:34:29.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:29 smithi053 ceph-mon[27773]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-25T14:34:29.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:29 smithi183 ceph-mon[32584]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-25T14:34:29.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:29 smithi150 ceph-mon[32504]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-25T14:34:31.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:31 smithi183 ceph-mon[32584]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-25T14:34:31.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:31 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/1635218701' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1cf74d94-c118-4738-b6db-040af458a8eb"}]: dispatch 2024-04-25T14:34:31.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:31 smithi183 ceph-mon[32584]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1cf74d94-c118-4738-b6db-040af458a8eb"}]: dispatch 2024-04-25T14:34:31.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:31 smithi183 ceph-mon[32584]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1cf74d94-c118-4738-b6db-040af458a8eb"}]': finished 2024-04-25T14:34:31.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:31 smithi183 ceph-mon[32584]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T14:34:31.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:31 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:34:31.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:31 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/1120995956' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:34:31.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:31 smithi150 ceph-mon[32504]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-25T14:34:31.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:31 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/1635218701' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1cf74d94-c118-4738-b6db-040af458a8eb"}]: dispatch 2024-04-25T14:34:31.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:31 smithi150 ceph-mon[32504]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1cf74d94-c118-4738-b6db-040af458a8eb"}]: dispatch 2024-04-25T14:34:31.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:31 smithi150 ceph-mon[32504]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1cf74d94-c118-4738-b6db-040af458a8eb"}]': finished 2024-04-25T14:34:31.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:31 smithi150 ceph-mon[32504]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T14:34:31.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:31 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:34:31.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:31 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/1120995956' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:34:32.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:31 smithi053 ceph-mon[27773]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-25T14:34:32.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:31 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1635218701' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1cf74d94-c118-4738-b6db-040af458a8eb"}]: dispatch 2024-04-25T14:34:32.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:31 smithi053 ceph-mon[27773]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1cf74d94-c118-4738-b6db-040af458a8eb"}]: dispatch 2024-04-25T14:34:32.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:31 smithi053 ceph-mon[27773]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1cf74d94-c118-4738-b6db-040af458a8eb"}]': finished 2024-04-25T14:34:32.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:31 smithi053 ceph-mon[27773]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T14:34:32.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:31 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:34:32.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:31 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1120995956' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:34:33.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:33 smithi183 ceph-mon[32584]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-25T14:34:33.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:33 smithi150 ceph-mon[32504]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-25T14:34:34.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:33 smithi053 ceph-mon[27773]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-25T14:34:35.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:35 smithi183 ceph-mon[32584]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-25T14:34:36.053 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:35 smithi053 ceph-mon[27773]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-25T14:34:36.054 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:35 smithi150 ceph-mon[32504]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-25T14:34:37.757 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:37 smithi053 ceph-mon[27773]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:34:37.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:37 smithi183 ceph-mon[32584]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:34:37.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:37 smithi150 ceph-mon[32504]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:34:39.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:39 smithi183 ceph-mon[32584]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:34:39.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:39 smithi150 ceph-mon[32504]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:34:40.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:39 smithi053 ceph-mon[27773]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:34:40.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:40 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T14:34:40.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:40 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:34:40.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:40 smithi183 ceph-mon[32584]: Deploying daemon osd.6 on smithi053 2024-04-25T14:34:40.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:40 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T14:34:40.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:40 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:34:40.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:40 smithi150 ceph-mon[32504]: Deploying daemon osd.6 on smithi053 2024-04-25T14:34:40.993 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:40 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T14:34:40.994 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:40 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:34:40.994 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:40 smithi053 ceph-mon[27773]: Deploying daemon osd.6 on smithi053 2024-04-25T14:34:41.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:41 smithi183 ceph-mon[32584]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:34:41.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:41 smithi150 ceph-mon[32504]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:34:42.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:41 smithi053 ceph-mon[27773]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:34:43.934 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:43 smithi053 ceph-mon[27773]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:34:43.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:43 smithi183 ceph-mon[32584]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:34:43.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:43 smithi150 ceph-mon[32504]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:34:45.765 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:45 smithi053 ceph-mon[27773]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:34:45.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:45 smithi183 ceph-mon[32584]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:34:45.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:45 smithi150 ceph-mon[32504]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:34:47.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:47 smithi183 ceph-mon[32584]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:34:47.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:47 smithi150 ceph-mon[32504]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:34:47.976 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:47 smithi053 ceph-mon[27773]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:34:49.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:49 smithi053 ceph-mon[27773]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:34:49.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:49 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:49.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:49 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:34:49.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:49 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:49.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:49 smithi183 ceph-mon[32584]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:34:49.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:49 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:49.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:49 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:34:49.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:49 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:49.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:49 smithi150 ceph-mon[32504]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:34:49.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:49 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:49.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:49 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:34:49.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:49 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:51.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:51 smithi183 ceph-mon[32584]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:34:51.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:51 smithi150 ceph-mon[32504]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:34:52.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:51 smithi053 ceph-mon[27773]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:34:53.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:53.076+0000 7f8860a00700 1 -- 172.21.15.53:0/1951639696 <== mgr.14152 v2:172.21.15.53:6800/4154933108 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f885c062e80 con 0x7f88400a3c40 2024-04-25T14:34:53.078 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 6 on host 'smithi053' 2024-04-25T14:34:53.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:53.080+0000 7f885be00700 1 -- 172.21.15.53:0/1951639696 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f88400a3c40 msgr2=0x7f88400a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:34:53.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:53.080+0000 7f885be00700 1 --2- 172.21.15.53:0/1951639696 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f88400a3c40 0x7f88400a6100 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8848009c60 tx=0x7f8848009260 comp rx=0 tx=0).stop 2024-04-25T14:34:53.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:53.080+0000 7f885be00700 1 -- 172.21.15.53:0/1951639696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f885c0773f0 msgr2=0x7f885c19f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:34:53.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:53.080+0000 7f885be00700 1 --2- 172.21.15.53:0/1951639696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f885c0773f0 0x7f885c19f0b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f884c000c00 tx=0x7f884c0047b0 comp rx=0 tx=0).stop 2024-04-25T14:34:53.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:53.080+0000 7f885be00700 1 -- 172.21.15.53:0/1951639696 shutdown_connections 2024-04-25T14:34:53.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:53.080+0000 7f885be00700 1 --2- 172.21.15.53:0/1951639696 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f88400a3c40 0x7f88400a6100 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:34:53.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:53.080+0000 7f885be00700 1 --2- 172.21.15.53:0/1951639696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f885c079000 0x7f885c1a6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:34:53.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:53.080+0000 7f885be00700 1 --2- 172.21.15.53:0/1951639696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f885c0773f0 0x7f885c19f0b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:34:53.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:53.080+0000 7f885be00700 1 --2- 172.21.15.53:0/1951639696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f885c078640 0x7f885c19f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:34:53.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:53.080+0000 7f885be00700 1 -- 172.21.15.53:0/1951639696 >> 172.21.15.53:0/1951639696 conn(0x7f885c100180 msgr2=0x7f885c101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:34:53.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:53.080+0000 7f885be00700 1 -- 172.21.15.53:0/1951639696 shutdown_connections 2024-04-25T14:34:53.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:34:53.080+0000 7f885be00700 1 -- 172.21.15.53:0/1951639696 wait complete. 2024-04-25T14:34:53.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:53 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:53.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:53 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:53.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:53 smithi053 ceph-mon[27773]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:34:53.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:53 smithi053 ceph-mon[27773]: from='osd.6 [v2:172.21.15.53:6818/355884319,v1:172.21.15.53:6819/355884319]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T14:34:53.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:53 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:53.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:53 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:53.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:53 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:53.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:53 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:53.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:53 smithi183 ceph-mon[32584]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:34:53.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:53 smithi183 ceph-mon[32584]: from='osd.6 [v2:172.21.15.53:6818/355884319,v1:172.21.15.53:6819/355884319]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T14:34:53.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:53 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:53.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:53 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:53.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:53 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:53.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:53 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:53.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:53 smithi150 ceph-mon[32504]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:34:53.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:53 smithi150 ceph-mon[32504]: from='osd.6 [v2:172.21.15.53:6818/355884319,v1:172.21.15.53:6819/355884319]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T14:34:53.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:53 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:53.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:53 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:53.775 DEBUG:teuthology.orchestra.run.smithi053:osd.6> sudo journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@osd.6.service 2024-04-25T14:34:53.781 INFO:tasks.cephadm:Deploying osd.7 on smithi150 with /dev/vg_nvme/lv_2... 2024-04-25T14:34:53.781 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T14:34:54.027 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:34:54.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:54 smithi183 ceph-mon[32584]: from='osd.6 [v2:172.21.15.53:6818/355884319,v1:172.21.15.53:6819/355884319]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T14:34:54.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:54 smithi183 ceph-mon[32584]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T14:34:54.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:54 smithi183 ceph-mon[32584]: from='osd.6 [v2:172.21.15.53:6818/355884319,v1:172.21.15.53:6819/355884319]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-04-25T14:34:54.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:54 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:34:54.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:54 smithi150 ceph-mon[32504]: from='osd.6 [v2:172.21.15.53:6818/355884319,v1:172.21.15.53:6819/355884319]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T14:34:54.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:54 smithi150 ceph-mon[32504]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T14:34:54.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:54 smithi150 ceph-mon[32504]: from='osd.6 [v2:172.21.15.53:6818/355884319,v1:172.21.15.53:6819/355884319]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-04-25T14:34:54.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:54 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:34:54.491 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:54 smithi053 ceph-mon[27773]: from='osd.6 [v2:172.21.15.53:6818/355884319,v1:172.21.15.53:6819/355884319]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T14:34:54.491 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:54 smithi053 ceph-mon[27773]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T14:34:54.491 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:54 smithi053 ceph-mon[27773]: from='osd.6 [v2:172.21.15.53:6818/355884319,v1:172.21.15.53:6819/355884319]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-04-25T14:34:54.491 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:54 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:34:54.491 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:34:54 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:34:54.178+0000 7f2689600700 -1 osd.6 0 waiting for initial osdmap 2024-04-25T14:34:54.492 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:34:54 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:34:54.190+0000 7f2684400700 -1 osd.6 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:34:55.369 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:55 smithi150 ceph-mon[32504]: from='osd.6 [v2:172.21.15.53:6818/355884319,v1:172.21.15.53:6819/355884319]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-04-25T14:34:55.369 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:55 smithi150 ceph-mon[32504]: osdmap e47: 7 total, 6 up, 7 in 2024-04-25T14:34:55.369 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:55 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:34:55.369 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:55 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:34:55.369 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:55 smithi150 ceph-mon[32504]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:34:55.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:55 smithi183 ceph-mon[32584]: from='osd.6 [v2:172.21.15.53:6818/355884319,v1:172.21.15.53:6819/355884319]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-04-25T14:34:55.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:55 smithi183 ceph-mon[32584]: osdmap e47: 7 total, 6 up, 7 in 2024-04-25T14:34:55.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:55 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:34:55.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:55 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:34:55.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:55 smithi183 ceph-mon[32584]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:34:55.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:55 smithi053 ceph-mon[27773]: from='osd.6 [v2:172.21.15.53:6818/355884319,v1:172.21.15.53:6819/355884319]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-04-25T14:34:55.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:55 smithi053 ceph-mon[27773]: osdmap e47: 7 total, 6 up, 7 in 2024-04-25T14:34:55.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:55 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:34:55.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:55 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:34:55.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:55 smithi053 ceph-mon[27773]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:34:55.959 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T14:34:55.959 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T14:34:55.959 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-04-25T14:34:55.960 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-04-25T14:34:55.960 INFO:teuthology.orchestra.run.smithi150.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114618 s, 915 MB/s 2024-04-25T14:34:55.960 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-04-25T14:34:56.278 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:56 smithi150 ceph-mon[32504]: purged_snaps scrub starts 2024-04-25T14:34:56.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:56 smithi150 ceph-mon[32504]: purged_snaps scrub ok 2024-04-25T14:34:56.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:56 smithi150 ceph-mon[32504]: osd.6 [v2:172.21.15.53:6818/355884319,v1:172.21.15.53:6819/355884319] boot 2024-04-25T14:34:56.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:56 smithi150 ceph-mon[32504]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T14:34:56.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:56 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:34:56.386 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:56 smithi053 ceph-mon[27773]: purged_snaps scrub starts 2024-04-25T14:34:56.386 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:56 smithi053 ceph-mon[27773]: purged_snaps scrub ok 2024-04-25T14:34:56.386 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:56 smithi053 ceph-mon[27773]: osd.6 [v2:172.21.15.53:6818/355884319,v1:172.21.15.53:6819/355884319] boot 2024-04-25T14:34:56.386 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:56 smithi053 ceph-mon[27773]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T14:34:56.386 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:56 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:34:56.459 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:56 smithi183 ceph-mon[32584]: purged_snaps scrub starts 2024-04-25T14:34:56.459 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:56 smithi183 ceph-mon[32584]: purged_snaps scrub ok 2024-04-25T14:34:56.460 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:56 smithi183 ceph-mon[32584]: osd.6 [v2:172.21.15.53:6818/355884319,v1:172.21.15.53:6819/355884319] boot 2024-04-25T14:34:56.460 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:56 smithi183 ceph-mon[32584]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T14:34:56.460 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:56 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:34:56.575 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_2 2024-04-25T14:34:56.818 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:34:57.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:57 smithi150 ceph-mon[32504]: osdmap e49: 7 total, 7 up, 7 in 2024-04-25T14:34:57.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:57 smithi150 ceph-mon[32504]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:34:57.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:57 smithi183 ceph-mon[32584]: osdmap e49: 7 total, 7 up, 7 in 2024-04-25T14:34:57.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:57 smithi183 ceph-mon[32584]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:34:57.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:57 smithi053 ceph-mon[27773]: osdmap e49: 7 total, 7 up, 7 in 2024-04-25T14:34:57.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:57 smithi053 ceph-mon[27773]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:34:58.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:34:58.384+0000 7f616f52b700 1 -- 172.21.15.150:0/3025962785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6168108790 msgr2=0x7f616810ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:34:58.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:34:58.384+0000 7f616f52b700 1 --2- 172.21.15.150:0/3025962785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6168108790 0x7f616810ac00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f616000b0a0 tx=0x7f616000b3b0 comp rx=0 tx=0).stop 2024-04-25T14:34:58.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:34:58.384+0000 7f616f52b700 1 -- 172.21.15.150:0/3025962785 shutdown_connections 2024-04-25T14:34:58.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:34:58.384+0000 7f616f52b700 1 --2- 172.21.15.150:0/3025962785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6168108790 0x7f616810ac00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:34:58.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:34:58.384+0000 7f616f52b700 1 --2- 172.21.15.150:0/3025962785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6168103530 0x7f6168105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:34:58.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:34:58.384+0000 7f616f52b700 1 --2- 172.21.15.150:0/3025962785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6168105e60 0x7f6168108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:34:58.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:34:58.384+0000 7f616f52b700 1 -- 172.21.15.150:0/3025962785 >> 172.21.15.150:0/3025962785 conn(0x7f61680fcf40 msgr2=0x7f61680ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:34:58.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:34:58.385+0000 7f616f52b700 1 -- 172.21.15.150:0/3025962785 shutdown_connections 2024-04-25T14:34:58.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:34:58.385+0000 7f616f52b700 1 -- 172.21.15.150:0/3025962785 wait complete. 2024-04-25T14:34:58.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:34:58.385+0000 7f616f52b700 1 Processor -- start 2024-04-25T14:34:58.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:34:58.385+0000 7f616f52b700 1 -- start start 2024-04-25T14:34:58.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:34:58.385+0000 7f616f52b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6168103530 0x7f6168198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:34:58.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:34:58.385+0000 7f616f52b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6168105e60 0x7f6168199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:34:58.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:34:58.385+0000 7f616f52b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6168108790 0x7f61681a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:34:58.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:34:58.385+0000 7f616f52b700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f616810e580 con 0x7f6168105e60 2024-04-25T14:34:58.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:34:58.385+0000 7f616f52b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f616810e400 con 0x7f6168108790 2024-04-25T14:34:58.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:34:58.385+0000 7f616f52b700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f616810e700 con 0x7f6168103530 2024-04-25T14:34:58.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:34:58.385+0000 7f616dac8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6168108790 0x7f61681a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:34:58.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:34:58.385+0000 7f616dac8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6168108790 0x7f61681a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:42220/0 (socket says 172.21.15.150:42220) 2024-04-25T14:34:58.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:34:58.385+0000 7f616d2c7700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6168103530 0x7f6168198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:34:58.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:34:58.385+0000 7f616dac8700 1 -- 172.21.15.150:0/1041476328 learned_addr learned my addr 172.21.15.150:0/1041476328 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-25T14:34:58.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:34:58.386+0000 7f616dac8700 1 -- 172.21.15.150:0/1041476328 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6168103530 msgr2=0x7f6168198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:34:58.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:34:58.386+0000 7f616dac8700 1 --2- 172.21.15.150:0/1041476328 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6168103530 0x7f6168198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:34:58.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:34:58.386+0000 7f616dac8700 1 -- 172.21.15.150:0/1041476328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6168105e60 msgr2=0x7f6168199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:34:58.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:34:58.386+0000 7f616dac8700 1 --2- 172.21.15.150:0/1041476328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6168105e60 0x7f6168199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:34:58.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:34:58.386+0000 7f616dac8700 1 -- 172.21.15.150:0/1041476328 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61681a0830 con 0x7f6168108790 2024-04-25T14:34:58.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:34:58.386+0000 7f616d2c7700 1 --2- 172.21.15.150:0/1041476328 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6168103530 0x7f6168198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:34:58.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:34:58.386+0000 7f616dac8700 1 --2- 172.21.15.150:0/1041476328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6168108790 0x7f61681a0190 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f616000b070 tx=0x7f6160004dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:34:58.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:34:58.386+0000 7f615e7fc700 1 -- 172.21.15.150:0/1041476328 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6160021070 con 0x7f6168108790 2024-04-25T14:34:58.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:34:58.386+0000 7f615e7fc700 1 -- 172.21.15.150:0/1041476328 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6160007c00 con 0x7f6168108790 2024-04-25T14:34:58.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:34:58.386+0000 7f616f52b700 1 -- 172.21.15.150:0/1041476328 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61681a0ac0 con 0x7f6168108790 2024-04-25T14:34:58.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:34:58.386+0000 7f615e7fc700 1 -- 172.21.15.150:0/1041476328 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6160004770 con 0x7f6168108790 2024-04-25T14:34:58.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:34:58.386+0000 7f616f52b700 1 -- 172.21.15.150:0/1041476328 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61681a1080 con 0x7f6168108790 2024-04-25T14:34:58.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:34:58.387+0000 7f616f52b700 1 -- 172.21.15.150:0/1041476328 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6168069730 con 0x7f6168108790 2024-04-25T14:34:58.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:34:58.389+0000 7f615e7fc700 1 -- 172.21.15.150:0/1041476328 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6160003a00 con 0x7f6168108790 2024-04-25T14:34:58.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:34:58.389+0000 7f615e7fc700 1 --2- 172.21.15.150:0/1041476328 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f61540a3c40 0x7f61540a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:34:58.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:34:58.390+0000 7f615e7fc700 1 -- 172.21.15.150:0/1041476328 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4965+0+0 (secure 0 0 0) 0x7f616000f020 con 0x7f6168108790 2024-04-25T14:34:58.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:34:58.390+0000 7f616d2c7700 1 --2- 172.21.15.150:0/1041476328 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f61540a3c40 0x7f61540a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:34:58.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:34:58.391+0000 7f616d2c7700 1 --2- 172.21.15.150:0/1041476328 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f61540a3c40 0x7f61540a6100 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6158011490 tx=0x7f6158009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:34:58.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:34:58.393+0000 7f615e7fc700 1 -- 172.21.15.150:0/1041476328 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f616002a070 con 0x7f6168108790 2024-04-25T14:34:58.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:34:58.600+0000 7f616f52b700 1 -- 172.21.15.150:0/1041476328 --> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f6168062e80 con 0x7f61540a3c40 2024-04-25T14:34:59.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:59 smithi053 ceph-mon[27773]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:34:59.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:59 smithi053 ceph-mon[27773]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:34:59.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:59 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:34:59.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:59 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:34:59.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:59 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:34:59.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:59 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:59.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:59 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:59.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:59 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:34:59.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:59 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:34:59.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:59 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:34:59.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:59 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:59.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:59 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:34:59.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:59 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:34:59.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:34:59 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:59.567 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:59 smithi150 ceph-mon[32504]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:34:59.567 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:59 smithi150 ceph-mon[32504]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:34:59.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:59 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:34:59.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:59 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:34:59.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:59 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:34:59.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:59 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:59.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:59 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:59.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:59 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:34:59.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:59 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:34:59.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:59 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:34:59.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:59 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:59.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:59 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:34:59.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:59 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:34:59.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:34:59 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:59.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:59 smithi183 ceph-mon[32584]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:34:59.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:59 smithi183 ceph-mon[32584]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:34:59.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:59 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:34:59.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:59 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:34:59.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:59 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:34:59.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:59 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:59.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:59 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:59.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:59 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:34:59.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:59 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:34:59.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:59 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:34:59.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:59 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:34:59.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:59 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:34:59.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:59 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:34:59.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:34:59 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:00.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:00 smithi150 ceph-mon[32504]: Detected new or changed devices on smithi053 2024-04-25T14:35:00.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:00 smithi150 ceph-mon[32504]: Adjusting osd_memory_target on smithi053 to 5698M 2024-04-25T14:35:00.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:00 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:35:00.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:00 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:35:00.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:00 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:35:00.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:00 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:00.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:00 smithi183 ceph-mon[32584]: Detected new or changed devices on smithi053 2024-04-25T14:35:00.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:00 smithi183 ceph-mon[32584]: Adjusting osd_memory_target on smithi053 to 5698M 2024-04-25T14:35:00.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:00 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:35:00.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:00 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:35:00.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:00 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:35:00.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:00 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:00.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:00 smithi053 ceph-mon[27773]: Detected new or changed devices on smithi053 2024-04-25T14:35:00.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:00 smithi053 ceph-mon[27773]: Adjusting osd_memory_target on smithi053 to 5698M 2024-04-25T14:35:00.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:00 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:35:00.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:00 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:35:00.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:00 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:35:00.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:00 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:01.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:01 smithi053 ceph-mon[27773]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:35:01.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:01 smithi053 ceph-mon[27773]: from='client.? 172.21.15.150:0/1386339803' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ce97c04-cd41-4080-bbbc-be9cf5fadc02"}]: dispatch 2024-04-25T14:35:01.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:01 smithi053 ceph-mon[27773]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ce97c04-cd41-4080-bbbc-be9cf5fadc02"}]: dispatch 2024-04-25T14:35:01.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:01 smithi053 ceph-mon[27773]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0ce97c04-cd41-4080-bbbc-be9cf5fadc02"}]': finished 2024-04-25T14:35:01.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:01 smithi053 ceph-mon[27773]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T14:35:01.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:01 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:35:01.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:01 smithi183 ceph-mon[32584]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:35:01.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:01 smithi183 ceph-mon[32584]: from='client.? 172.21.15.150:0/1386339803' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ce97c04-cd41-4080-bbbc-be9cf5fadc02"}]: dispatch 2024-04-25T14:35:01.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:01 smithi183 ceph-mon[32584]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ce97c04-cd41-4080-bbbc-be9cf5fadc02"}]: dispatch 2024-04-25T14:35:01.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:01 smithi183 ceph-mon[32584]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0ce97c04-cd41-4080-bbbc-be9cf5fadc02"}]': finished 2024-04-25T14:35:01.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:01 smithi183 ceph-mon[32584]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T14:35:01.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:01 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:35:01.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:01 smithi150 ceph-mon[32504]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:35:01.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:01 smithi150 ceph-mon[32504]: from='client.? 172.21.15.150:0/1386339803' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ce97c04-cd41-4080-bbbc-be9cf5fadc02"}]: dispatch 2024-04-25T14:35:01.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:01 smithi150 ceph-mon[32504]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ce97c04-cd41-4080-bbbc-be9cf5fadc02"}]: dispatch 2024-04-25T14:35:01.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:01 smithi150 ceph-mon[32504]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0ce97c04-cd41-4080-bbbc-be9cf5fadc02"}]': finished 2024-04-25T14:35:01.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:01 smithi150 ceph-mon[32504]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T14:35:01.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:01 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:35:02.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:02 smithi183 ceph-mon[32584]: from='client.? 172.21.15.150:0/1800241856' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:35:02.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:02 smithi150 ceph-mon[32504]: from='client.? 172.21.15.150:0/1800241856' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:35:03.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:02 smithi053 ceph-mon[27773]: from='client.? 172.21.15.150:0/1800241856' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:35:03.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:03 smithi183 ceph-mon[32584]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:35:03.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:03 smithi150 ceph-mon[32504]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:35:04.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:03 smithi053 ceph-mon[27773]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:35:05.830 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:05 smithi150 ceph-mon[32504]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:35:05.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:05 smithi183 ceph-mon[32584]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:35:06.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:05 smithi053 ceph-mon[27773]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:35:07.674 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:07 smithi150 ceph-mon[32504]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:35:07.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:07 smithi183 ceph-mon[32584]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:35:08.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:07 smithi053 ceph-mon[27773]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:35:09.629 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:09 smithi150 ceph-mon[32504]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:35:09.629 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:09 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T14:35:09.629 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:09 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:35:09.629 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:09 smithi150 ceph-mon[32504]: Deploying daemon osd.7 on smithi150 2024-04-25T14:35:09.835 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:09 smithi053 ceph-mon[27773]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:35:09.835 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:09 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T14:35:09.836 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:09 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:35:09.836 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:09 smithi053 ceph-mon[27773]: Deploying daemon osd.7 on smithi150 2024-04-25T14:35:09.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:09 smithi183 ceph-mon[32584]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:35:09.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:09 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T14:35:09.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:09 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:35:09.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:09 smithi183 ceph-mon[32584]: Deploying daemon osd.7 on smithi150 2024-04-25T14:35:11.863 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:11 smithi150 ceph-mon[32504]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:35:11.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:11 smithi183 ceph-mon[32584]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:35:12.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:11 smithi053 ceph-mon[27773]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:35:13.622 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:13 smithi150 ceph-mon[32504]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:35:13.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:13 smithi183 ceph-mon[32584]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:35:14.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:13 smithi053 ceph-mon[27773]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:35:15.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:15 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:15.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:15 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:15.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:15 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:35:15.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:15 smithi183 ceph-mon[32584]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:35:15.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:15 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:15.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:15 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:15.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:15 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:35:15.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:15 smithi150 ceph-mon[32504]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:35:15.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:15 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:15.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:15 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:15.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:15 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:35:15.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:15 smithi053 ceph-mon[27773]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:35:17.232 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 7 on host 'smithi150' 2024-04-25T14:35:17.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:35:17.231+0000 7f615e7fc700 1 -- 172.21.15.150:0/1041476328 <== mgr.14152 v2:172.21.15.53:6800/4154933108 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6168062e80 con 0x7f61540a3c40 2024-04-25T14:35:17.236 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:35:17.234+0000 7f616f52b700 1 -- 172.21.15.150:0/1041476328 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f61540a3c40 msgr2=0x7f61540a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:35:17.236 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:35:17.234+0000 7f616f52b700 1 --2- 172.21.15.150:0/1041476328 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f61540a3c40 0x7f61540a6100 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6158011490 tx=0x7f6158009260 comp rx=0 tx=0).stop 2024-04-25T14:35:17.236 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:35:17.234+0000 7f616f52b700 1 -- 172.21.15.150:0/1041476328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6168108790 msgr2=0x7f61681a0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:35:17.236 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:35:17.234+0000 7f616f52b700 1 --2- 172.21.15.150:0/1041476328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6168108790 0x7f61681a0190 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f616000b070 tx=0x7f6160004dc0 comp rx=0 tx=0).stop 2024-04-25T14:35:17.236 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:35:17.234+0000 7f616f52b700 1 -- 172.21.15.150:0/1041476328 shutdown_connections 2024-04-25T14:35:17.237 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:35:17.234+0000 7f616f52b700 1 --2- 172.21.15.150:0/1041476328 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f61540a3c40 0x7f61540a6100 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:35:17.237 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:35:17.234+0000 7f616f52b700 1 --2- 172.21.15.150:0/1041476328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6168108790 0x7f61681a0190 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:35:17.237 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:35:17.234+0000 7f616f52b700 1 --2- 172.21.15.150:0/1041476328 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6168103530 0x7f6168198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:35:17.237 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:35:17.234+0000 7f616f52b700 1 --2- 172.21.15.150:0/1041476328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6168105e60 0x7f6168199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:35:17.237 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:35:17.234+0000 7f616f52b700 1 -- 172.21.15.150:0/1041476328 >> 172.21.15.150:0/1041476328 conn(0x7f61680fcf40 msgr2=0x7f6168103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:35:17.237 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:35:17.234+0000 7f616f52b700 1 -- 172.21.15.150:0/1041476328 shutdown_connections 2024-04-25T14:35:17.237 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:35:17.234+0000 7f616f52b700 1 -- 172.21.15.150:0/1041476328 wait complete. 2024-04-25T14:35:17.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:17 smithi150 ceph-mon[32504]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:35:17.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:17 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:17.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:17 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:17.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:17 smithi183 ceph-mon[32584]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:35:17.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:17 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:17.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:17 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:17.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:17 smithi053 ceph-mon[27773]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:35:17.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:17 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:17.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:17 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:17.889 DEBUG:teuthology.orchestra.run.smithi150:osd.7> sudo journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@osd.7.service 2024-04-25T14:35:17.891 INFO:tasks.cephadm:Deploying osd.8 on smithi183 with /dev/vg_nvme/lv_2... 2024-04-25T14:35:17.892 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T14:35:18.137 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:35:18.210 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:35:18 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:35:18.036+0000 7fc6be1b57c0 -1 osd.7 0 log_to_monitors true 2024-04-25T14:35:18.565 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:18 smithi183 ceph-mon[32584]: from='osd.7 [v2:172.21.15.150:6816/229022562,v1:172.21.15.150:6817/229022562]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T14:35:18.566 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:18 smithi183 ceph-mon[32584]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T14:35:18.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:18 smithi150 ceph-mon[32504]: from='osd.7 [v2:172.21.15.150:6816/229022562,v1:172.21.15.150:6817/229022562]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T14:35:18.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:18 smithi150 ceph-mon[32504]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T14:35:18.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:18 smithi053 ceph-mon[27773]: from='osd.7 [v2:172.21.15.150:6816/229022562,v1:172.21.15.150:6817/229022562]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T14:35:18.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:18 smithi053 ceph-mon[27773]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T14:35:19.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:19 smithi183 ceph-mon[32584]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:35:19.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:19 smithi183 ceph-mon[32584]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T14:35:19.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:19 smithi183 ceph-mon[32584]: from='osd.7 [v2:172.21.15.150:6816/229022562,v1:172.21.15.150:6817/229022562]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T14:35:19.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:19 smithi183 ceph-mon[32584]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T14:35:19.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:19 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:35:19.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:19 smithi183 ceph-mon[32584]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T14:35:19.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:19 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:19.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:19 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:19.710 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:35:19 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:35:19.323+0000 7fc6b381a700 -1 osd.7 0 waiting for initial osdmap 2024-04-25T14:35:19.710 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:35:19 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:35:19.335+0000 7fc6af60d700 -1 osd.7 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:35:19.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:19 smithi150 ceph-mon[32504]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:35:19.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:19 smithi150 ceph-mon[32504]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T14:35:19.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:19 smithi150 ceph-mon[32504]: from='osd.7 [v2:172.21.15.150:6816/229022562,v1:172.21.15.150:6817/229022562]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T14:35:19.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:19 smithi150 ceph-mon[32504]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T14:35:19.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:19 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:35:19.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:19 smithi150 ceph-mon[32504]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T14:35:19.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:19 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:19.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:19 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:19.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:19 smithi053 ceph-mon[27773]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:35:19.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:19 smithi053 ceph-mon[27773]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T14:35:19.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:19 smithi053 ceph-mon[27773]: from='osd.7 [v2:172.21.15.150:6816/229022562,v1:172.21.15.150:6817/229022562]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T14:35:19.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:19 smithi053 ceph-mon[27773]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T14:35:19.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:19 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:35:19.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:19 smithi053 ceph-mon[27773]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T14:35:19.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:19 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:19.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:19 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:20.044 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T14:35:20.044 INFO:teuthology.orchestra.run.smithi183.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T14:35:20.044 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10+0 records in 2024-04-25T14:35:20.044 INFO:teuthology.orchestra.run.smithi183.stderr:10+0 records out 2024-04-25T14:35:20.044 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113948 s, 920 MB/s 2024-04-25T14:35:20.044 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping successful for: 2024-04-25T14:35:20.634 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:20 smithi183 ceph-mon[32584]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-25T14:35:20.634 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:20 smithi183 ceph-mon[32584]: osdmap e52: 8 total, 7 up, 8 in 2024-04-25T14:35:20.634 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:20 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:35:20.662 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_2 2024-04-25T14:35:20.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:20 smithi150 ceph-mon[32504]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-25T14:35:20.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:20 smithi150 ceph-mon[32504]: osdmap e52: 8 total, 7 up, 8 in 2024-04-25T14:35:20.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:20 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:35:20.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:20 smithi053 ceph-mon[27773]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-25T14:35:20.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:20 smithi053 ceph-mon[27773]: osdmap e52: 8 total, 7 up, 8 in 2024-04-25T14:35:20.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:20 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:35:20.905 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:35:21.584 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:21 smithi150 ceph-mon[32504]: purged_snaps scrub starts 2024-04-25T14:35:21.585 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:21 smithi150 ceph-mon[32504]: purged_snaps scrub ok 2024-04-25T14:35:21.585 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:21 smithi150 ceph-mon[32504]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:35:21.585 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:21 smithi150 ceph-mon[32504]: osd.7 [v2:172.21.15.150:6816/229022562,v1:172.21.15.150:6817/229022562] boot 2024-04-25T14:35:21.585 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:21 smithi150 ceph-mon[32504]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T14:35:21.585 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:21 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:35:21.585 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:21 smithi150 ceph-mon[32504]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T14:35:21.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:21 smithi183 ceph-mon[32584]: purged_snaps scrub starts 2024-04-25T14:35:21.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:21 smithi183 ceph-mon[32584]: purged_snaps scrub ok 2024-04-25T14:35:21.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:21 smithi183 ceph-mon[32584]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:35:21.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:21 smithi183 ceph-mon[32584]: osd.7 [v2:172.21.15.150:6816/229022562,v1:172.21.15.150:6817/229022562] boot 2024-04-25T14:35:21.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:21 smithi183 ceph-mon[32584]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T14:35:21.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:21 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:35:21.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:21 smithi183 ceph-mon[32584]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T14:35:21.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:21 smithi053 ceph-mon[27773]: purged_snaps scrub starts 2024-04-25T14:35:21.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:21 smithi053 ceph-mon[27773]: purged_snaps scrub ok 2024-04-25T14:35:21.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:21 smithi053 ceph-mon[27773]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:35:21.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:21 smithi053 ceph-mon[27773]: osd.7 [v2:172.21.15.150:6816/229022562,v1:172.21.15.150:6817/229022562] boot 2024-04-25T14:35:21.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:21 smithi053 ceph-mon[27773]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T14:35:21.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:21 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:35:21.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:21 smithi053 ceph-mon[27773]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T14:35:22.491 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:22.488+0000 7f4641b4a700 1 -- 172.21.15.183:0/2573168679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f463c102aa0 msgr2=0x7f463c102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:35:22.491 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:22.488+0000 7f4641b4a700 1 --2- 172.21.15.183:0/2573168679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f463c102aa0 0x7f463c102ec0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f462c009990 tx=0x7f462c009ca0 comp rx=0 tx=0).stop 2024-04-25T14:35:22.491 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:22.488+0000 7f4641b4a700 1 -- 172.21.15.183:0/2573168679 shutdown_connections 2024-04-25T14:35:22.491 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:22.488+0000 7f4641b4a700 1 --2- 172.21.15.183:0/2573168679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f463c104640 0x7f463c10aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:35:22.491 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:22.488+0000 7f4641b4a700 1 --2- 172.21.15.183:0/2573168679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f463c102aa0 0x7f463c102ec0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:35:22.491 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:22.488+0000 7f4641b4a700 1 --2- 172.21.15.183:0/2573168679 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f463c103ca0 0x7f463c104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:35:22.492 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:22.488+0000 7f4641b4a700 1 -- 172.21.15.183:0/2573168679 >> 172.21.15.183:0/2573168679 conn(0x7f463c0fe040 msgr2=0x7f463c100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:35:22.492 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:22.488+0000 7f4641b4a700 1 -- 172.21.15.183:0/2573168679 shutdown_connections 2024-04-25T14:35:22.492 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:22.488+0000 7f4641b4a700 1 -- 172.21.15.183:0/2573168679 wait complete. 2024-04-25T14:35:22.492 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:22.488+0000 7f4641b4a700 1 Processor -- start 2024-04-25T14:35:22.492 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:22.488+0000 7f4641b4a700 1 -- start start 2024-04-25T14:35:22.492 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:22.489+0000 7f4641b4a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f463c102aa0 0x7f463c198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:35:22.492 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:22.489+0000 7f4641b4a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f463c103ca0 0x7f463c199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:35:22.492 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:22.489+0000 7f4641b4a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f463c104640 0x7f463c1a0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:35:22.492 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:22.489+0000 7f4641b4a700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f463c10e5b0 con 0x7f463c102aa0 2024-04-25T14:35:22.492 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:22.489+0000 7f4641b4a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f463c10e430 con 0x7f463c103ca0 2024-04-25T14:35:22.492 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:22.489+0000 7f4641b4a700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f463c10e730 con 0x7f463c104640 2024-04-25T14:35:22.492 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:22.489+0000 7f463bfff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f463c104640 0x7f463c1a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:35:22.492 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:22.489+0000 7f463bfff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f463c104640 0x7f463c1a0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:38908/0 (socket says 172.21.15.183:38908) 2024-04-25T14:35:22.492 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:22.489+0000 7f463bfff700 1 -- 172.21.15.183:0/2581081672 learned_addr learned my addr 172.21.15.183:0/2581081672 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-04-25T14:35:22.492 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:22.489+0000 7f4632ffd700 1 --2- 172.21.15.183:0/2581081672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f463c103ca0 0x7f463c199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:35:22.493 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:22.489+0000 7f463bfff700 1 -- 172.21.15.183:0/2581081672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f463c103ca0 msgr2=0x7f463c199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:35:22.493 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:22.489+0000 7f463bfff700 1 --2- 172.21.15.183:0/2581081672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f463c103ca0 0x7f463c199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:35:22.493 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:22.489+0000 7f463bfff700 1 -- 172.21.15.183:0/2581081672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f463c102aa0 msgr2=0x7f463c198ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:35:22.493 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:22.489+0000 7f463bfff700 1 --2- 172.21.15.183:0/2581081672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f463c102aa0 0x7f463c198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:35:22.493 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:22.489+0000 7f463bfff700 1 -- 172.21.15.183:0/2581081672 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f463c1a0910 con 0x7f463c104640 2024-04-25T14:35:22.493 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:22.489+0000 7f4632ffd700 1 --2- 172.21.15.183:0/2581081672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f463c103ca0 0x7f463c199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:35:22.493 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:22.489+0000 7f463bfff700 1 --2- 172.21.15.183:0/2581081672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f463c104640 0x7f463c1a0240 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f463400b970 tx=0x7f463400bd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:35:22.493 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:22.490+0000 7f46397fa700 1 -- 172.21.15.183:0/2581081672 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4634019070 con 0x7f463c104640 2024-04-25T14:35:22.493 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:22.490+0000 7f4641b4a700 1 -- 172.21.15.183:0/2581081672 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f463c1a0c00 con 0x7f463c104640 2024-04-25T14:35:22.493 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:22.490+0000 7f46397fa700 1 -- 172.21.15.183:0/2581081672 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4634014440 con 0x7f463c104640 2024-04-25T14:35:22.493 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:22.490+0000 7f46397fa700 1 -- 172.21.15.183:0/2581081672 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463400ce60 con 0x7f463c104640 2024-04-25T14:35:22.493 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:22.490+0000 7f4641b4a700 1 -- 172.21.15.183:0/2581081672 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f463c1a1140 con 0x7f463c104640 2024-04-25T14:35:22.495 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:22.491+0000 7f4641b4a700 1 -- 172.21.15.183:0/2581081672 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f463c069730 con 0x7f463c104640 2024-04-25T14:35:22.495 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:22.492+0000 7f46397fa700 1 -- 172.21.15.183:0/2581081672 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f463401c070 con 0x7f463c104640 2024-04-25T14:35:22.495 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:22.492+0000 7f46397fa700 1 --2- 172.21.15.183:0/2581081672 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f46280a3c40 0x7f46280a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:35:22.495 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:22.493+0000 7f46397fa700 1 -- 172.21.15.183:0/2581081672 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5426+0+0 (secure 0 0 0) 0x7f46340cf1b0 con 0x7f463c104640 2024-04-25T14:35:22.495 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:22.493+0000 7f463b7fe700 1 --2- 172.21.15.183:0/2581081672 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f46280a3c40 0x7f46280a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:35:22.495 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:22.494+0000 7f463b7fe700 1 --2- 172.21.15.183:0/2581081672 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f46280a3c40 0x7f46280a6100 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f462c00b5c0 tx=0x7f462c003860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:35:22.496 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:22.496+0000 7f46397fa700 1 -- 172.21.15.183:0/2581081672 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f463409c6d0 con 0x7f463c104640 2024-04-25T14:35:22.703 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:22.702+0000 7f4641b4a700 1 -- 172.21.15.183:0/2581081672 --> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f463c062e80 con 0x7f46280a3c40 2024-04-25T14:35:22.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:22 smithi183 ceph-mon[32584]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T14:35:22.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:22 smithi183 ceph-mon[32584]: Detected new or changed devices on smithi150 2024-04-25T14:35:22.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:22 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:22.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:22 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:22.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:22 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:35:22.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:22 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:35:22.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:22 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:35:22.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:22 smithi183 ceph-mon[32584]: Adjusting osd_memory_target on smithi150 to 5696M 2024-04-25T14:35:22.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:22 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:22.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:22 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:35:22.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:22 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:35:22.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:22 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:22.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:22 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:35:22.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:22 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:35:22.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:22 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:35:22.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:22 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:22.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:22 smithi150 ceph-mon[32504]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T14:35:22.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:22 smithi150 ceph-mon[32504]: Detected new or changed devices on smithi150 2024-04-25T14:35:22.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:22 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:22.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:22 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:22.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:22 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:35:22.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:22 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:35:22.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:22 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:35:22.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:22 smithi150 ceph-mon[32504]: Adjusting osd_memory_target on smithi150 to 5696M 2024-04-25T14:35:22.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:22 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:22.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:22 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:35:22.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:22 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:35:22.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:22 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:22.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:22 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:35:22.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:22 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:35:22.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:22 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:35:22.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:22 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:23.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:22 smithi053 ceph-mon[27773]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T14:35:23.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:22 smithi053 ceph-mon[27773]: Detected new or changed devices on smithi150 2024-04-25T14:35:23.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:22 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:23.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:22 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:23.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:22 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:35:23.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:22 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:35:23.034 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:22 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:35:23.034 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:22 smithi053 ceph-mon[27773]: Adjusting osd_memory_target on smithi150 to 5696M 2024-04-25T14:35:23.034 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:22 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:23.034 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:22 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:35:23.034 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:22 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:35:23.034 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:22 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:23.034 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:22 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:35:23.034 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:22 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:35:23.034 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:22 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:35:23.034 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:22 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:23.663 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:23 smithi183 ceph-mon[32584]: pgmap v199: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T14:35:23.663 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:23 smithi183 ceph-mon[32584]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:35:23.663 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:23 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:35:23.663 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:23 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:35:23.663 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:23 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:35:23.663 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:23 smithi183 ceph-mon[32584]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-04-25T14:35:23.663 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:23 smithi183 ceph-mon[32584]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-25T14:35:23.663 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:23 smithi183 ceph-mon[32584]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T14:35:23.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:23 smithi150 ceph-mon[32504]: pgmap v199: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T14:35:23.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:23 smithi150 ceph-mon[32504]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:35:23.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:23 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:35:23.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:23 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:35:23.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:23 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:35:23.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:23 smithi150 ceph-mon[32504]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-04-25T14:35:23.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:23 smithi150 ceph-mon[32504]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-25T14:35:23.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:23 smithi150 ceph-mon[32504]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T14:35:24.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:23 smithi053 ceph-mon[27773]: pgmap v199: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T14:35:24.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:23 smithi053 ceph-mon[27773]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:35:24.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:23 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:35:24.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:23 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:35:24.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:23 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:35:24.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:23 smithi053 ceph-mon[27773]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-04-25T14:35:24.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:23 smithi053 ceph-mon[27773]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-25T14:35:24.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:23 smithi053 ceph-mon[27773]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T14:35:25.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:25 smithi053 ceph-mon[27773]: osdmap e57: 8 total, 8 up, 8 in 2024-04-25T14:35:25.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:25 smithi183 ceph-mon[32584]: osdmap e57: 8 total, 8 up, 8 in 2024-04-25T14:35:25.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:25 smithi150 ceph-mon[32504]: osdmap e57: 8 total, 8 up, 8 in 2024-04-25T14:35:26.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:26 smithi053 ceph-mon[27773]: pgmap v202: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T14:35:26.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:26 smithi053 ceph-mon[27773]: from='client.? 172.21.15.183:0/2342273552' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d57787d-2a7c-4514-bd01-357ced413f68"}]: dispatch 2024-04-25T14:35:26.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:26 smithi053 ceph-mon[27773]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d57787d-2a7c-4514-bd01-357ced413f68"}]: dispatch 2024-04-25T14:35:26.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:26 smithi053 ceph-mon[27773]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d57787d-2a7c-4514-bd01-357ced413f68"}]': finished 2024-04-25T14:35:26.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:26 smithi053 ceph-mon[27773]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T14:35:26.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:26 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:35:26.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:26 smithi053 ceph-mon[27773]: from='client.? 172.21.15.183:0/1009950280' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:35:26.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:26 smithi183 ceph-mon[32584]: pgmap v202: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T14:35:26.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:26 smithi183 ceph-mon[32584]: from='client.? 172.21.15.183:0/2342273552' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d57787d-2a7c-4514-bd01-357ced413f68"}]: dispatch 2024-04-25T14:35:26.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:26 smithi183 ceph-mon[32584]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d57787d-2a7c-4514-bd01-357ced413f68"}]: dispatch 2024-04-25T14:35:26.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:26 smithi183 ceph-mon[32584]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d57787d-2a7c-4514-bd01-357ced413f68"}]': finished 2024-04-25T14:35:26.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:26 smithi183 ceph-mon[32584]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T14:35:26.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:26 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:35:26.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:26 smithi183 ceph-mon[32584]: from='client.? 172.21.15.183:0/1009950280' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:35:26.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:26 smithi150 ceph-mon[32504]: pgmap v202: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T14:35:26.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:26 smithi150 ceph-mon[32504]: from='client.? 172.21.15.183:0/2342273552' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d57787d-2a7c-4514-bd01-357ced413f68"}]: dispatch 2024-04-25T14:35:26.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:26 smithi150 ceph-mon[32504]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d57787d-2a7c-4514-bd01-357ced413f68"}]: dispatch 2024-04-25T14:35:26.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:26 smithi150 ceph-mon[32504]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d57787d-2a7c-4514-bd01-357ced413f68"}]': finished 2024-04-25T14:35:26.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:26 smithi150 ceph-mon[32504]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T14:35:26.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:26 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:35:26.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:26 smithi150 ceph-mon[32504]: from='client.? 172.21.15.183:0/1009950280' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:35:27.355 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:27 smithi183 ceph-mon[32584]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-04-25T14:35:27.355 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:27 smithi183 ceph-mon[32584]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-25T14:35:27.355 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:27 smithi183 ceph-mon[32584]: Cluster is now healthy 2024-04-25T14:35:27.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:27 smithi150 ceph-mon[32504]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-04-25T14:35:27.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:27 smithi150 ceph-mon[32504]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-25T14:35:27.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:27 smithi150 ceph-mon[32504]: Cluster is now healthy 2024-04-25T14:35:27.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:27 smithi053 ceph-mon[27773]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-04-25T14:35:27.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:27 smithi053 ceph-mon[27773]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-25T14:35:27.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:27 smithi053 ceph-mon[27773]: Cluster is now healthy 2024-04-25T14:35:28.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:28 smithi183 ceph-mon[32584]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:35:28.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:28 smithi150 ceph-mon[32504]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:35:28.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:28 smithi053 ceph-mon[27773]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:35:30.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:30 smithi183 ceph-mon[32584]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:35:30.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:30 smithi150 ceph-mon[32504]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:35:30.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:30 smithi053 ceph-mon[27773]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:35:32.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:32 smithi183 ceph-mon[32584]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:35:32.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:32 smithi150 ceph-mon[32504]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:35:32.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:32 smithi053 ceph-mon[27773]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:35:33.446 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:33 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T14:35:33.447 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:33 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:35:33.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:33 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T14:35:33.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:33 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:35:33.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:33 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T14:35:33.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:33 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:35:34.392 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:34 smithi183 ceph-mon[32584]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:35:34.392 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:34 smithi183 ceph-mon[32584]: Deploying daemon osd.8 on smithi183 2024-04-25T14:35:34.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:34 smithi150 ceph-mon[32504]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:35:34.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:34 smithi150 ceph-mon[32504]: Deploying daemon osd.8 on smithi183 2024-04-25T14:35:34.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:34 smithi053 ceph-mon[27773]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:35:34.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:34 smithi053 ceph-mon[27773]: Deploying daemon osd.8 on smithi183 2024-04-25T14:35:35.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:35 smithi183 ceph-mon[32584]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:35:35.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:35 smithi150 ceph-mon[32504]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:35:35.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:35 smithi053 ceph-mon[27773]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:35:37.397 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:37 smithi183 ceph-mon[32584]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:35:37.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:37 smithi150 ceph-mon[32504]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:35:37.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:37 smithi053 ceph-mon[27773]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:35:39.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:39 smithi053 ceph-mon[27773]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:35:39.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:39 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:39.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:39 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:39.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:39 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:35:39.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:39 smithi183 ceph-mon[32584]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:35:39.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:39 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:39.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:39 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:39.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:39 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:35:39.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:39 smithi150 ceph-mon[32504]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:35:39.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:39 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:39.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:39 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:39.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:39 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:35:41.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:41 smithi183 ceph-mon[32584]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:35:41.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:41 smithi150 ceph-mon[32504]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:35:41.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:41 smithi053 ceph-mon[27773]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:35:41.790 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:41.790+0000 7f46397fa700 1 -- 172.21.15.183:0/2581081672 <== mgr.14152 v2:172.21.15.53:6800/4154933108 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f463c062e80 con 0x7f46280a3c40 2024-04-25T14:35:41.791 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 8 on host 'smithi183' 2024-04-25T14:35:41.793 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:41.793+0000 7f4641b4a700 1 -- 172.21.15.183:0/2581081672 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f46280a3c40 msgr2=0x7f46280a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:35:41.793 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:41.793+0000 7f4641b4a700 1 --2- 172.21.15.183:0/2581081672 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f46280a3c40 0x7f46280a6100 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f462c00b5c0 tx=0x7f462c003860 comp rx=0 tx=0).stop 2024-04-25T14:35:41.793 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:41.793+0000 7f4641b4a700 1 -- 172.21.15.183:0/2581081672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f463c104640 msgr2=0x7f463c1a0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:35:41.793 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:41.793+0000 7f4641b4a700 1 --2- 172.21.15.183:0/2581081672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f463c104640 0x7f463c1a0240 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f463400b970 tx=0x7f463400bd30 comp rx=0 tx=0).stop 2024-04-25T14:35:41.793 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:41.793+0000 7f4641b4a700 1 -- 172.21.15.183:0/2581081672 shutdown_connections 2024-04-25T14:35:41.793 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:41.793+0000 7f4641b4a700 1 --2- 172.21.15.183:0/2581081672 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f46280a3c40 0x7f46280a6100 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:35:41.794 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:41.793+0000 7f4641b4a700 1 --2- 172.21.15.183:0/2581081672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f463c104640 0x7f463c1a0240 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:35:41.794 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:41.793+0000 7f4641b4a700 1 --2- 172.21.15.183:0/2581081672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f463c102aa0 0x7f463c198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:35:41.794 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:41.793+0000 7f4641b4a700 1 --2- 172.21.15.183:0/2581081672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f463c103ca0 0x7f463c199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:35:41.794 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:41.793+0000 7f4641b4a700 1 -- 172.21.15.183:0/2581081672 >> 172.21.15.183:0/2581081672 conn(0x7f463c0fe040 msgr2=0x7f463c1070c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:35:41.794 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:41.793+0000 7f4641b4a700 1 -- 172.21.15.183:0/2581081672 shutdown_connections 2024-04-25T14:35:41.794 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:35:41.793+0000 7f4641b4a700 1 -- 172.21.15.183:0/2581081672 wait complete. 2024-04-25T14:35:42.403 DEBUG:teuthology.orchestra.run.smithi183:osd.8> sudo journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@osd.8.service 2024-04-25T14:35:42.405 INFO:tasks.cephadm:Deploying osd.9 on smithi053 with /dev/vg_nvme/lv_1... 2024-04-25T14:35:42.405 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T14:35:42.885 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:35:42.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:42 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:42.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:42 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:42.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:42 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:42.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:42 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:42.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:42 smithi183 ceph-mon[32584]: from='osd.8 [v2:172.21.15.183:6816/301003844,v1:172.21.15.183:6817/301003844]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T14:35:42.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:42 smithi183 ceph-mon[32584]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T14:35:43.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:42 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:43.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:42 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:43.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:42 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:43.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:42 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:43.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:42 smithi053 ceph-mon[27773]: from='osd.8 [v2:172.21.15.183:6816/301003844,v1:172.21.15.183:6817/301003844]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T14:35:43.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:42 smithi053 ceph-mon[27773]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T14:35:43.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:42 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:43.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:42 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:43.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:42 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:43.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:42 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:43.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:42 smithi150 ceph-mon[32504]: from='osd.8 [v2:172.21.15.183:6816/301003844,v1:172.21.15.183:6817/301003844]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T14:35:43.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:42 smithi150 ceph-mon[32504]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T14:35:44.117 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:43 smithi053 ceph-mon[27773]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:35:44.118 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:43 smithi053 ceph-mon[27773]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T14:35:44.118 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:43 smithi053 ceph-mon[27773]: from='osd.8 [v2:172.21.15.183:6816/301003844,v1:172.21.15.183:6817/301003844]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T14:35:44.118 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:43 smithi053 ceph-mon[27773]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T14:35:44.118 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:43 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:35:44.118 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:43 smithi053 ceph-mon[27773]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T14:35:44.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:43 smithi183 ceph-mon[32584]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:35:44.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:43 smithi183 ceph-mon[32584]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T14:35:44.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:43 smithi183 ceph-mon[32584]: from='osd.8 [v2:172.21.15.183:6816/301003844,v1:172.21.15.183:6817/301003844]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T14:35:44.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:43 smithi183 ceph-mon[32584]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T14:35:44.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:43 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:35:44.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:43 smithi183 ceph-mon[32584]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T14:35:44.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:43 smithi150 ceph-mon[32504]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:35:44.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:43 smithi150 ceph-mon[32504]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T14:35:44.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:43 smithi150 ceph-mon[32504]: from='osd.8 [v2:172.21.15.183:6816/301003844,v1:172.21.15.183:6817/301003844]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T14:35:44.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:43 smithi150 ceph-mon[32504]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T14:35:44.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:43 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:35:44.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:43 smithi150 ceph-mon[32504]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T14:35:44.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:44 smithi183 ceph-mon[32584]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-25T14:35:44.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:44 smithi183 ceph-mon[32584]: osdmap e60: 9 total, 8 up, 9 in 2024-04-25T14:35:44.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:44 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:35:44.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:44 smithi183 ceph-mon[32584]: from='osd.8 ' entity='osd.8' 2024-04-25T14:35:44.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:44 smithi183 ceph-mon[32584]: osdmap e61: 9 total, 8 up, 9 in 2024-04-25T14:35:44.954 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:35:44 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:35:44.729+0000 7f2ec1ff6700 -1 osd.8 0 waiting for initial osdmap 2024-04-25T14:35:44.954 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:35:44 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:35:44.902+0000 7f2eb9de1700 -1 osd.8 61 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:35:45.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:44 smithi150 ceph-mon[32504]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-25T14:35:45.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:44 smithi150 ceph-mon[32504]: osdmap e60: 9 total, 8 up, 9 in 2024-04-25T14:35:45.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:44 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:35:45.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:44 smithi150 ceph-mon[32504]: from='osd.8 ' entity='osd.8' 2024-04-25T14:35:45.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:44 smithi150 ceph-mon[32504]: osdmap e61: 9 total, 8 up, 9 in 2024-04-25T14:35:45.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:44 smithi053 ceph-mon[27773]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-25T14:35:45.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:44 smithi053 ceph-mon[27773]: osdmap e60: 9 total, 8 up, 9 in 2024-04-25T14:35:45.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:44 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:35:45.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:44 smithi053 ceph-mon[27773]: from='osd.8 ' entity='osd.8' 2024-04-25T14:35:45.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:44 smithi053 ceph-mon[27773]: osdmap e61: 9 total, 8 up, 9 in 2024-04-25T14:35:45.725 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T14:35:45.730 INFO:teuthology.orchestra.run.smithi053.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T14:35:45.730 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10+0 records in 2024-04-25T14:35:45.730 INFO:teuthology.orchestra.run.smithi053.stderr:10+0 records out 2024-04-25T14:35:45.730 INFO:teuthology.orchestra.run.smithi053.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0343152 s, 306 MB/s 2024-04-25T14:35:45.730 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping successful for: 2024-04-25T14:35:46.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:45 smithi053 ceph-mon[27773]: purged_snaps scrub starts 2024-04-25T14:35:46.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:45 smithi053 ceph-mon[27773]: purged_snaps scrub ok 2024-04-25T14:35:46.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:45 smithi053 ceph-mon[27773]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:35:46.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:45 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:35:46.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:45 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:46.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:45 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:46.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:45 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:35:46.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:45 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:35:46.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:45 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:35:46.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:45 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:46.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:45 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:35:46.034 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:45 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:35:46.034 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:45 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:46.034 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:45 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:35:46.034 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:45 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:35:46.034 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:45 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:35:46.034 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:45 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:46.034 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:45 smithi053 ceph-mon[27773]: osd.8 [v2:172.21.15.183:6816/301003844,v1:172.21.15.183:6817/301003844] boot 2024-04-25T14:35:46.034 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:45 smithi053 ceph-mon[27773]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T14:35:46.034 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:45 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:35:46.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:45 smithi183 ceph-mon[32584]: purged_snaps scrub starts 2024-04-25T14:35:46.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:45 smithi183 ceph-mon[32584]: purged_snaps scrub ok 2024-04-25T14:35:46.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:45 smithi183 ceph-mon[32584]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:35:46.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:45 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:35:46.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:45 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:46.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:45 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:46.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:45 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:35:46.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:45 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:35:46.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:45 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:35:46.206 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:45 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:46.206 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:45 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:35:46.206 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:45 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:35:46.206 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:45 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:46.206 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:45 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:35:46.206 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:45 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:35:46.206 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:45 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:35:46.206 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:45 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:46.206 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:45 smithi183 ceph-mon[32584]: osd.8 [v2:172.21.15.183:6816/301003844,v1:172.21.15.183:6817/301003844] boot 2024-04-25T14:35:46.206 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:45 smithi183 ceph-mon[32584]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T14:35:46.206 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:45 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:35:46.213 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:45 smithi150 ceph-mon[32504]: purged_snaps scrub starts 2024-04-25T14:35:46.213 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:45 smithi150 ceph-mon[32504]: purged_snaps scrub ok 2024-04-25T14:35:46.213 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:45 smithi150 ceph-mon[32504]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:35:46.214 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:45 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:35:46.214 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:45 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:46.214 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:45 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:46.214 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:45 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:35:46.214 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:45 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:35:46.214 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:45 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:35:46.214 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:45 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:46.214 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:45 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:35:46.214 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:45 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:35:46.214 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:45 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:46.214 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:45 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:35:46.214 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:45 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:35:46.214 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:45 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:35:46.214 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:45 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:35:46.215 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:45 smithi150 ceph-mon[32504]: osd.8 [v2:172.21.15.183:6816/301003844,v1:172.21.15.183:6817/301003844] boot 2024-04-25T14:35:46.215 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:45 smithi150 ceph-mon[32504]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T14:35:46.215 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:45 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:35:46.618 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi053:vg_nvme/lv_1 2024-04-25T14:35:47.046 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:35:47.197 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:46 smithi053 ceph-mon[27773]: Detected new or changed devices on smithi183 2024-04-25T14:35:47.197 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:46 smithi053 ceph-mon[27773]: Adjusting osd_memory_target on smithi183 to 5696M 2024-04-25T14:35:47.197 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:46 smithi053 ceph-mon[27773]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T14:35:47.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:46 smithi183 ceph-mon[32584]: Detected new or changed devices on smithi183 2024-04-25T14:35:47.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:46 smithi183 ceph-mon[32584]: Adjusting osd_memory_target on smithi183 to 5696M 2024-04-25T14:35:47.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:46 smithi183 ceph-mon[32584]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T14:35:47.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:46 smithi150 ceph-mon[32504]: Detected new or changed devices on smithi183 2024-04-25T14:35:47.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:46 smithi150 ceph-mon[32504]: Adjusting osd_memory_target on smithi183 to 5696M 2024-04-25T14:35:47.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:46 smithi150 ceph-mon[32504]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T14:35:48.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:47 smithi183 ceph-mon[32584]: pgmap v218: 1 pgs: 1 remapped+peering; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:35:48.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:47 smithi183 ceph-mon[32584]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T14:35:48.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:47 smithi150 ceph-mon[32504]: pgmap v218: 1 pgs: 1 remapped+peering; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:35:48.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:47 smithi150 ceph-mon[32504]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T14:35:48.229 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:47 smithi053 ceph-mon[27773]: pgmap v218: 1 pgs: 1 remapped+peering; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:35:48.230 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:47 smithi053 ceph-mon[27773]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T14:35:49.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:35:49.108+0000 7f31bca00700 1 -- 172.21.15.53:0/2575234586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31b8076a70 msgr2=0x7f31b8074ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:35:49.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:35:49.108+0000 7f31bca00700 1 --2- 172.21.15.53:0/2575234586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31b8076a70 0x7f31b8074ed0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f31a800b0a0 tx=0x7f31a800b3b0 comp rx=0 tx=0).stop 2024-04-25T14:35:49.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:35:49.108+0000 7f31bca00700 1 -- 172.21.15.53:0/2575234586 shutdown_connections 2024-04-25T14:35:49.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:35:49.108+0000 7f31bca00700 1 --2- 172.21.15.53:0/2575234586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31b810a8a0 0x7f31b810cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:35:49.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:35:49.108+0000 7f31bca00700 1 --2- 172.21.15.53:0/2575234586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31b8076a70 0x7f31b8074ed0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:35:49.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:35:49.108+0000 7f31bca00700 1 --2- 172.21.15.53:0/2575234586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31b80754a0 0x7f31b8075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:35:49.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:35:49.108+0000 7f31bca00700 1 -- 172.21.15.53:0/2575234586 >> 172.21.15.53:0/2575234586 conn(0x7f31b80fdf70 msgr2=0x7f31b81003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:35:49.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:35:49.108+0000 7f31bca00700 1 -- 172.21.15.53:0/2575234586 shutdown_connections 2024-04-25T14:35:49.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:35:49.108+0000 7f31bca00700 1 -- 172.21.15.53:0/2575234586 wait complete. 2024-04-25T14:35:49.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:35:49.109+0000 7f31bca00700 1 Processor -- start 2024-04-25T14:35:49.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:35:49.110+0000 7f31bca00700 1 -- start start 2024-04-25T14:35:49.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:35:49.110+0000 7f31bca00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31b80754a0 0x7f31b819ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:35:49.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:35:49.110+0000 7f31bca00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31b8076a70 0x7f31b819b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:35:49.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:35:49.110+0000 7f31bca00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31b810a8a0 0x7f31b81a2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:35:49.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:35:49.110+0000 7f31bca00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f31b8110690 con 0x7f31b80754a0 2024-04-25T14:35:49.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:35:49.110+0000 7f31bca00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f31b8110510 con 0x7f31b810a8a0 2024-04-25T14:35:49.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:35:49.110+0000 7f31bca00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f31b8110810 con 0x7f31b8076a70 2024-04-25T14:35:49.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:35:49.110+0000 7f31b6800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31b80754a0 0x7f31b819ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:35:49.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:35:49.110+0000 7f31b6800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31b80754a0 0x7f31b819ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39120/0 (socket says 172.21.15.53:39120) 2024-04-25T14:35:49.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:35:49.110+0000 7f31b6800700 1 -- 172.21.15.53:0/3898771682 learned_addr learned my addr 172.21.15.53:0/3898771682 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:35:49.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:35:49.111+0000 7f31b7200700 1 --2- 172.21.15.53:0/3898771682 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31b810a8a0 0x7f31b81a2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:35:49.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:35:49.111+0000 7f31ade00700 1 --2- 172.21.15.53:0/3898771682 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31b8076a70 0x7f31b819b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:35:49.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:35:49.111+0000 7f31b6800700 1 -- 172.21.15.53:0/3898771682 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31b8076a70 msgr2=0x7f31b819b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:35:49.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:35:49.111+0000 7f31b6800700 1 --2- 172.21.15.53:0/3898771682 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31b8076a70 0x7f31b819b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:35:49.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:35:49.111+0000 7f31b6800700 1 -- 172.21.15.53:0/3898771682 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31b810a8a0 msgr2=0x7f31b81a2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:35:49.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:35:49.111+0000 7f31b6800700 1 --2- 172.21.15.53:0/3898771682 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31b810a8a0 0x7f31b81a2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:35:49.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:35:49.111+0000 7f31b6800700 1 -- 172.21.15.53:0/3898771682 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31b81a29a0 con 0x7f31b80754a0 2024-04-25T14:35:49.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:35:49.111+0000 7f31b7200700 1 --2- 172.21.15.53:0/3898771682 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31b810a8a0 0x7f31b81a2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:35:49.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:35:49.111+0000 7f31b6800700 1 --2- 172.21.15.53:0/3898771682 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31b80754a0 0x7f31b819ad90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f31a8004ce0 tx=0x7f31a8004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:35:49.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:35:49.112+0000 7f31afe00700 1 -- 172.21.15.53:0/3898771682 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a8021070 con 0x7f31b80754a0 2024-04-25T14:35:49.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:35:49.112+0000 7f31afe00700 1 -- 172.21.15.53:0/3898771682 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31a8007b80 con 0x7f31b80754a0 2024-04-25T14:35:49.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:35:49.112+0000 7f31bca00700 1 -- 172.21.15.53:0/3898771682 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31b81a2c30 con 0x7f31b80754a0 2024-04-25T14:35:49.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:35:49.112+0000 7f31afe00700 1 -- 172.21.15.53:0/3898771682 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a80052a0 con 0x7f31b80754a0 2024-04-25T14:35:49.115 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:35:49.112+0000 7f31bca00700 1 -- 172.21.15.53:0/3898771682 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31b81a31c0 con 0x7f31b80754a0 2024-04-25T14:35:49.115 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:35:49.113+0000 7f31bca00700 1 -- 172.21.15.53:0/3898771682 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31b8069730 con 0x7f31b80754a0 2024-04-25T14:35:49.116 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:35:49.116+0000 7f31afe00700 1 -- 172.21.15.53:0/3898771682 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f31a8024070 con 0x7f31b80754a0 2024-04-25T14:35:49.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:35:49.117+0000 7f31afe00700 1 --2- 172.21.15.53:0/3898771682 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f319c0a3cc0 0x7f319c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:35:49.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:35:49.117+0000 7f31afe00700 1 -- 172.21.15.53:0/3898771682 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5829+0+0 (secure 0 0 0) 0x7f31a800f020 con 0x7f31b80754a0 2024-04-25T14:35:49.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:35:49.117+0000 7f31ade00700 1 --2- 172.21.15.53:0/3898771682 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f319c0a3cc0 0x7f319c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:35:49.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:35:49.118+0000 7f31ade00700 1 --2- 172.21.15.53:0/3898771682 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f319c0a3cc0 0x7f319c0a6180 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f31b80767a0 tx=0x7f31a0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:35:49.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:35:49.120+0000 7f31afe00700 1 -- 172.21.15.53:0/3898771682 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f31a802b070 con 0x7f31b80754a0 2024-04-25T14:35:49.363 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:35:49.362+0000 7f31bca00700 1 -- 172.21.15.53:0/3898771682 --> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f31b8062e80 con 0x7f319c0a3cc0 2024-04-25T14:35:50.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:49 smithi183 ceph-mon[32584]: pgmap v221: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:35:50.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:49 smithi183 ceph-mon[32584]: osdmap e65: 9 total, 9 up, 9 in 2024-04-25T14:35:50.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:49 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:35:50.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:49 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:35:50.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:49 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:35:50.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:49 smithi150 ceph-mon[32504]: pgmap v221: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:35:50.213 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:49 smithi150 ceph-mon[32504]: osdmap e65: 9 total, 9 up, 9 in 2024-04-25T14:35:50.214 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:49 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:35:50.214 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:49 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:35:50.214 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:49 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:35:50.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:49 smithi053 ceph-mon[27773]: pgmap v221: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:35:50.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:49 smithi053 ceph-mon[27773]: osdmap e65: 9 total, 9 up, 9 in 2024-04-25T14:35:50.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:49 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:35:50.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:49 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:35:50.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:49 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:35:51.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:50 smithi053 ceph-mon[27773]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:35:51.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:50 smithi053 ceph-mon[27773]: osdmap e66: 9 total, 9 up, 9 in 2024-04-25T14:35:51.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:50 smithi183 ceph-mon[32584]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:35:51.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:50 smithi183 ceph-mon[32584]: osdmap e66: 9 total, 9 up, 9 in 2024-04-25T14:35:51.209 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:50 smithi150 ceph-mon[32504]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:35:51.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:50 smithi150 ceph-mon[32504]: osdmap e66: 9 total, 9 up, 9 in 2024-04-25T14:35:52.189 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:51 smithi053 ceph-mon[27773]: pgmap v224: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:35:52.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:51 smithi183 ceph-mon[32584]: pgmap v224: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:35:52.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:51 smithi150 ceph-mon[32504]: pgmap v224: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:35:54.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:53 smithi183 ceph-mon[32584]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-04-25T14:35:54.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:53 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/4080468598' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e11dd8eb-c55a-4d70-8c58-457540b7a50a"}]: dispatch 2024-04-25T14:35:54.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:53 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/4080468598' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e11dd8eb-c55a-4d70-8c58-457540b7a50a"}]': finished 2024-04-25T14:35:54.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:53 smithi183 ceph-mon[32584]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T14:35:54.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:53 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:35:54.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:53 smithi150 ceph-mon[32504]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-04-25T14:35:54.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:53 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/4080468598' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e11dd8eb-c55a-4d70-8c58-457540b7a50a"}]: dispatch 2024-04-25T14:35:54.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:53 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/4080468598' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e11dd8eb-c55a-4d70-8c58-457540b7a50a"}]': finished 2024-04-25T14:35:54.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:53 smithi150 ceph-mon[32504]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T14:35:54.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:53 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:35:54.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:53 smithi053 ceph-mon[27773]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-04-25T14:35:54.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:53 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/4080468598' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e11dd8eb-c55a-4d70-8c58-457540b7a50a"}]: dispatch 2024-04-25T14:35:54.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:53 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/4080468598' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e11dd8eb-c55a-4d70-8c58-457540b7a50a"}]': finished 2024-04-25T14:35:54.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:53 smithi053 ceph-mon[27773]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T14:35:54.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:53 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:35:55.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:54 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/1801012774' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:35:55.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:54 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/1801012774' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:35:55.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:54 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1801012774' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:35:56.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:55 smithi183 ceph-mon[32584]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T14:35:56.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:55 smithi150 ceph-mon[32504]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T14:35:56.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:55 smithi053 ceph-mon[27773]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T14:35:58.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:57 smithi053 ceph-mon[27773]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-25T14:35:58.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:57 smithi183 ceph-mon[32584]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-25T14:35:58.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:57 smithi150 ceph-mon[32504]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-25T14:36:00.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:35:59 smithi183 ceph-mon[32584]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-25T14:36:00.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:35:59 smithi150 ceph-mon[32504]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-25T14:36:00.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:35:59 smithi053 ceph-mon[27773]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-25T14:36:02.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:01 smithi183 ceph-mon[32584]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T14:36:02.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:01 smithi150 ceph-mon[32504]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T14:36:02.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:01 smithi053 ceph-mon[27773]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T14:36:03.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:02 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T14:36:03.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:02 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:36:03.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:02 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T14:36:03.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:02 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:36:03.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:02 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T14:36:03.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:02 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:36:04.146 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:03 smithi053 ceph-mon[27773]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:36:04.146 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:03 smithi053 ceph-mon[27773]: Deploying daemon osd.9 on smithi053 2024-04-25T14:36:04.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:03 smithi183 ceph-mon[32584]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:36:04.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:03 smithi183 ceph-mon[32584]: Deploying daemon osd.9 on smithi053 2024-04-25T14:36:04.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:03 smithi150 ceph-mon[32504]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:36:04.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:03 smithi150 ceph-mon[32504]: Deploying daemon osd.9 on smithi053 2024-04-25T14:36:06.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:05 smithi183 ceph-mon[32584]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:36:06.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:05 smithi150 ceph-mon[32504]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:36:06.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:05 smithi053 ceph-mon[27773]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:36:08.203 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:07 smithi053 ceph-mon[27773]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:36:08.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:07 smithi183 ceph-mon[32584]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:36:08.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:07 smithi150 ceph-mon[32504]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:36:10.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:09 smithi183 ceph-mon[32584]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:36:10.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:09 smithi150 ceph-mon[32504]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:36:10.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:09 smithi053 ceph-mon[27773]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:36:12.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:11 smithi183 ceph-mon[32584]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:36:12.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:12.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:12.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:36:12.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:11 smithi150 ceph-mon[32504]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:36:12.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:11 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:12.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:11 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:12.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:11 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:36:12.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:11 smithi053 ceph-mon[27773]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:36:12.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:11 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:12.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:11 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:12.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:11 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:36:14.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:13 smithi183 ceph-mon[32584]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:36:14.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:13 smithi150 ceph-mon[32504]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:36:14.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:13 smithi053 ceph-mon[27773]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:36:16.199 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:15 smithi053 ceph-mon[27773]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:36:16.199 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:15 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:16.199 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:15 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:16.199 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:15 smithi053 ceph-mon[27773]: from='osd.9 [v2:172.21.15.53:6826/3449260978,v1:172.21.15.53:6827/3449260978]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T14:36:16.200 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:15 smithi053 ceph-mon[27773]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T14:36:16.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:15 smithi183 ceph-mon[32584]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:36:16.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:15 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:16.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:15 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:16.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:15 smithi183 ceph-mon[32584]: from='osd.9 [v2:172.21.15.53:6826/3449260978,v1:172.21.15.53:6827/3449260978]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T14:36:16.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:15 smithi183 ceph-mon[32584]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T14:36:16.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:15 smithi150 ceph-mon[32504]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:36:16.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:15 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:16.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:15 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:16.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:15 smithi150 ceph-mon[32504]: from='osd.9 [v2:172.21.15.53:6826/3449260978,v1:172.21.15.53:6827/3449260978]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T14:36:16.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:15 smithi150 ceph-mon[32504]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T14:36:16.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:36:16.334+0000 7f31afe00700 1 -- 172.21.15.53:0/3898771682 <== mgr.14152 v2:172.21.15.53:6800/4154933108 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f31b8062e80 con 0x7f319c0a3cc0 2024-04-25T14:36:16.394 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 9 on host 'smithi053' 2024-04-25T14:36:16.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:36:16.338+0000 7f31bca00700 1 -- 172.21.15.53:0/3898771682 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f319c0a3cc0 msgr2=0x7f319c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:36:16.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:36:16.338+0000 7f31bca00700 1 --2- 172.21.15.53:0/3898771682 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f319c0a3cc0 0x7f319c0a6180 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f31b80767a0 tx=0x7f31a0009260 comp rx=0 tx=0).stop 2024-04-25T14:36:16.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:36:16.338+0000 7f31bca00700 1 -- 172.21.15.53:0/3898771682 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31b80754a0 msgr2=0x7f31b819ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:36:16.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:36:16.338+0000 7f31bca00700 1 --2- 172.21.15.53:0/3898771682 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31b80754a0 0x7f31b819ad90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f31a8004ce0 tx=0x7f31a8004d10 comp rx=0 tx=0).stop 2024-04-25T14:36:16.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:36:16.338+0000 7f31bca00700 1 -- 172.21.15.53:0/3898771682 shutdown_connections 2024-04-25T14:36:16.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:36:16.338+0000 7f31bca00700 1 --2- 172.21.15.53:0/3898771682 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f319c0a3cc0 0x7f319c0a6180 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:36:16.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:36:16.338+0000 7f31bca00700 1 --2- 172.21.15.53:0/3898771682 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31b810a8a0 0x7f31b81a2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:36:16.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:36:16.338+0000 7f31bca00700 1 --2- 172.21.15.53:0/3898771682 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31b80754a0 0x7f31b819ad90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:36:16.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:36:16.338+0000 7f31bca00700 1 --2- 172.21.15.53:0/3898771682 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31b8076a70 0x7f31b819b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:36:16.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:36:16.338+0000 7f31bca00700 1 -- 172.21.15.53:0/3898771682 >> 172.21.15.53:0/3898771682 conn(0x7f31b80fdf70 msgr2=0x7f31b81067a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:36:16.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:36:16.338+0000 7f31bca00700 1 -- 172.21.15.53:0/3898771682 shutdown_connections 2024-04-25T14:36:16.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:36:16.338+0000 7f31bca00700 1 -- 172.21.15.53:0/3898771682 wait complete. 2024-04-25T14:36:17.004 DEBUG:teuthology.orchestra.run.smithi053:osd.9> sudo journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@osd.9.service 2024-04-25T14:36:17.011 INFO:tasks.cephadm:Deploying osd.10 on smithi150 with /dev/vg_nvme/lv_1... 2024-04-25T14:36:17.011 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T14:36:17.259 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:36:17.541 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:17 smithi150 ceph-mon[32504]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:36:17.541 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:17 smithi150 ceph-mon[32504]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T14:36:17.541 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:17 smithi150 ceph-mon[32504]: from='osd.9 [v2:172.21.15.53:6826/3449260978,v1:172.21.15.53:6827/3449260978]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-04-25T14:36:17.541 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:17 smithi150 ceph-mon[32504]: osdmap e68: 10 total, 9 up, 10 in 2024-04-25T14:36:17.541 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:17 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:36:17.542 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:17 smithi150 ceph-mon[32504]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-04-25T14:36:17.542 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:17 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:17.542 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:17 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:17.610 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:17 smithi053 ceph-mon[27773]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:36:17.610 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:17 smithi053 ceph-mon[27773]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T14:36:17.610 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:17 smithi053 ceph-mon[27773]: from='osd.9 [v2:172.21.15.53:6826/3449260978,v1:172.21.15.53:6827/3449260978]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-04-25T14:36:17.610 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:17 smithi053 ceph-mon[27773]: osdmap e68: 10 total, 9 up, 10 in 2024-04-25T14:36:17.610 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:17 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:36:17.610 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:17 smithi053 ceph-mon[27773]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-04-25T14:36:17.610 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:17 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:17.610 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:17 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:17.610 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:36:17 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:36:17.311+0000 7fcdd0400700 -1 osd.9 0 waiting for initial osdmap 2024-04-25T14:36:17.611 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:36:17 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:36:17.334+0000 7fcdcb200700 -1 osd.9 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:36:17.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:17 smithi183 ceph-mon[32584]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:36:17.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:17 smithi183 ceph-mon[32584]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T14:36:17.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:17 smithi183 ceph-mon[32584]: from='osd.9 [v2:172.21.15.53:6826/3449260978,v1:172.21.15.53:6827/3449260978]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-04-25T14:36:17.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:17 smithi183 ceph-mon[32584]: osdmap e68: 10 total, 9 up, 10 in 2024-04-25T14:36:17.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:17 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:36:17.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:17 smithi183 ceph-mon[32584]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-04-25T14:36:17.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:17 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:17.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:17 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:18.658 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:18 smithi150 ceph-mon[32504]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-04-25T14:36:18.658 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:18 smithi150 ceph-mon[32504]: osdmap e69: 10 total, 9 up, 10 in 2024-04-25T14:36:18.658 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:18 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:36:18.658 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:18 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:36:18.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:18 smithi183 ceph-mon[32584]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-04-25T14:36:18.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:18 smithi183 ceph-mon[32584]: osdmap e69: 10 total, 9 up, 10 in 2024-04-25T14:36:18.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:18 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:36:18.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:18 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:36:18.710 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:18 smithi053 ceph-mon[27773]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-04-25T14:36:18.710 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:18 smithi053 ceph-mon[27773]: osdmap e69: 10 total, 9 up, 10 in 2024-04-25T14:36:18.710 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:18 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:36:18.710 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:18 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:36:19.196 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T14:36:19.196 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T14:36:19.196 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-04-25T14:36:19.196 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-04-25T14:36:19.196 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113257 s, 926 MB/s 2024-04-25T14:36:19.196 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-04-25T14:36:19.497 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:19 smithi150 ceph-mon[32504]: purged_snaps scrub starts 2024-04-25T14:36:19.497 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:19 smithi150 ceph-mon[32504]: purged_snaps scrub ok 2024-04-25T14:36:19.497 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:19 smithi150 ceph-mon[32504]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:36:19.497 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:19 smithi150 ceph-mon[32504]: osd.9 [v2:172.21.15.53:6826/3449260978,v1:172.21.15.53:6827/3449260978] boot 2024-04-25T14:36:19.497 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:19 smithi150 ceph-mon[32504]: osdmap e70: 10 total, 10 up, 10 in 2024-04-25T14:36:19.497 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:19 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:36:19.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:19 smithi183 ceph-mon[32584]: purged_snaps scrub starts 2024-04-25T14:36:19.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:19 smithi183 ceph-mon[32584]: purged_snaps scrub ok 2024-04-25T14:36:19.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:19 smithi183 ceph-mon[32584]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:36:19.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:19 smithi183 ceph-mon[32584]: osd.9 [v2:172.21.15.53:6826/3449260978,v1:172.21.15.53:6827/3449260978] boot 2024-04-25T14:36:19.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:19 smithi183 ceph-mon[32584]: osdmap e70: 10 total, 10 up, 10 in 2024-04-25T14:36:19.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:19 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:36:19.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:19 smithi053 ceph-mon[27773]: purged_snaps scrub starts 2024-04-25T14:36:19.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:19 smithi053 ceph-mon[27773]: purged_snaps scrub ok 2024-04-25T14:36:19.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:19 smithi053 ceph-mon[27773]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:36:19.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:19 smithi053 ceph-mon[27773]: osd.9 [v2:172.21.15.53:6826/3449260978,v1:172.21.15.53:6827/3449260978] boot 2024-04-25T14:36:19.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:19 smithi053 ceph-mon[27773]: osdmap e70: 10 total, 10 up, 10 in 2024-04-25T14:36:19.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:19 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:36:19.892 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_1 2024-04-25T14:36:20.139 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:36:20.466 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:20 smithi150 ceph-mon[32504]: osdmap e71: 10 total, 10 up, 10 in 2024-04-25T14:36:20.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:20 smithi183 ceph-mon[32584]: osdmap e71: 10 total, 10 up, 10 in 2024-04-25T14:36:20.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:20 smithi053 ceph-mon[27773]: osdmap e71: 10 total, 10 up, 10 in 2024-04-25T14:36:21.498 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:21 smithi150 ceph-mon[32504]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:36:21.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:21.693+0000 7fcc1e2b2700 1 -- 172.21.15.150:0/2863037612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc18108790 msgr2=0x7fcc1810ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:36:21.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:21.693+0000 7fcc1e2b2700 1 --2- 172.21.15.150:0/2863037612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc18108790 0x7fcc1810ac00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcc1000b0a0 tx=0x7fcc1000b3b0 comp rx=0 tx=0).stop 2024-04-25T14:36:21.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:21.693+0000 7fcc1e2b2700 1 -- 172.21.15.150:0/2863037612 shutdown_connections 2024-04-25T14:36:21.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:21.693+0000 7fcc1e2b2700 1 --2- 172.21.15.150:0/2863037612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc18108790 0x7fcc1810ac00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:36:21.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:21.693+0000 7fcc1e2b2700 1 --2- 172.21.15.150:0/2863037612 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc18103530 0x7fcc18105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:36:21.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:21.693+0000 7fcc1e2b2700 1 --2- 172.21.15.150:0/2863037612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc18105e60 0x7fcc18108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:36:21.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:21.693+0000 7fcc1e2b2700 1 -- 172.21.15.150:0/2863037612 >> 172.21.15.150:0/2863037612 conn(0x7fcc180fcf40 msgr2=0x7fcc180ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:36:21.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:21.693+0000 7fcc1e2b2700 1 -- 172.21.15.150:0/2863037612 shutdown_connections 2024-04-25T14:36:21.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:21.693+0000 7fcc1e2b2700 1 -- 172.21.15.150:0/2863037612 wait complete. 2024-04-25T14:36:21.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:21.694+0000 7fcc1e2b2700 1 Processor -- start 2024-04-25T14:36:21.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:21.694+0000 7fcc1e2b2700 1 -- start start 2024-04-25T14:36:21.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:21.694+0000 7fcc1e2b2700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc18103530 0x7fcc1819afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:36:21.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:21.694+0000 7fcc1e2b2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc18105e60 0x7fcc1819b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:36:21.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:21.694+0000 7fcc1e2b2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc18108790 0x7fcc181a2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:36:21.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:21.694+0000 7fcc1e2b2700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1810e650 con 0x7fcc18105e60 2024-04-25T14:36:21.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:21.694+0000 7fcc1e2b2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1810e4d0 con 0x7fcc18108790 2024-04-25T14:36:21.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:21.694+0000 7fcc1e2b2700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1810e7d0 con 0x7fcc18103530 2024-04-25T14:36:21.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:21.694+0000 7fcc1c84f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc18108790 0x7fcc181a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:36:21.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:21.694+0000 7fcc1c84f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc18108790 0x7fcc181a2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:60516/0 (socket says 172.21.15.150:60516) 2024-04-25T14:36:21.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:21.694+0000 7fcc1c84f700 1 -- 172.21.15.150:0/4209282675 learned_addr learned my addr 172.21.15.150:0/4209282675 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-25T14:36:21.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:21.694+0000 7fcc17fff700 1 --2- 172.21.15.150:0/4209282675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc18103530 0x7fcc1819afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:36:21.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:21.694+0000 7fcc1c84f700 1 -- 172.21.15.150:0/4209282675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc18103530 msgr2=0x7fcc1819afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:36:21.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:21.694+0000 7fcc1c84f700 1 --2- 172.21.15.150:0/4209282675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc18103530 0x7fcc1819afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:36:21.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:21.694+0000 7fcc1c84f700 1 -- 172.21.15.150:0/4209282675 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc18105e60 msgr2=0x7fcc1819b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:36:21.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:21.694+0000 7fcc1c84f700 1 --2- 172.21.15.150:0/4209282675 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc18105e60 0x7fcc1819b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:36:21.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:21.695+0000 7fcc1c84f700 1 -- 172.21.15.150:0/4209282675 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc181a2c30 con 0x7fcc18108790 2024-04-25T14:36:21.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:21.695+0000 7fcc1c84f700 1 --2- 172.21.15.150:0/4209282675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc18108790 0x7fcc181a2590 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009fd0 tx=0x7fcc1000cf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:36:21.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:21.695+0000 7fcc157fa700 1 -- 172.21.15.150:0/4209282675 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc10021030 con 0x7fcc18108790 2024-04-25T14:36:21.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:21.695+0000 7fcc1e2b2700 1 -- 172.21.15.150:0/4209282675 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc181a2ec0 con 0x7fcc18108790 2024-04-25T14:36:21.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:21.695+0000 7fcc157fa700 1 -- 172.21.15.150:0/4209282675 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc10007cb0 con 0x7fcc18108790 2024-04-25T14:36:21.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:21.695+0000 7fcc157fa700 1 -- 172.21.15.150:0/4209282675 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc10004850 con 0x7fcc18108790 2024-04-25T14:36:21.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:21.695+0000 7fcc1e2b2700 1 -- 172.21.15.150:0/4209282675 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc181a33f0 con 0x7fcc18108790 2024-04-25T14:36:21.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:21.696+0000 7fcc1e2b2700 1 -- 172.21.15.150:0/4209282675 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc18069730 con 0x7fcc18108790 2024-04-25T14:36:21.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:21.697+0000 7fcc157fa700 1 -- 172.21.15.150:0/4209282675 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fcc10024070 con 0x7fcc18108790 2024-04-25T14:36:21.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:21.697+0000 7fcc157fa700 1 --2- 172.21.15.150:0/4209282675 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fcc000a3cc0 0x7fcc000a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:36:21.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:21.698+0000 7fcc157fa700 1 -- 172.21.15.150:0/4209282675 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6261+0+0 (secure 0 0 0) 0x7fcc1000f020 con 0x7fcc18108790 2024-04-25T14:36:21.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:21.698+0000 7fcc17fff700 1 --2- 172.21.15.150:0/4209282675 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fcc000a3cc0 0x7fcc000a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:36:21.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:21.699+0000 7fcc17fff700 1 --2- 172.21.15.150:0/4209282675 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fcc000a3cc0 0x7fcc000a6180 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcc08009a10 tx=0x7fcc08009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:36:21.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:21.702+0000 7fcc157fa700 1 -- 172.21.15.150:0/4209282675 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcc10024360 con 0x7fcc18108790 2024-04-25T14:36:21.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:21 smithi183 ceph-mon[32584]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:36:21.762 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:21 smithi053 ceph-mon[27773]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:36:21.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:21.911+0000 7fcc1e2b2700 1 -- 172.21.15.150:0/4209282675 --> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fcc18062e80 con 0x7fcc000a3cc0 2024-04-25T14:36:22.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:22 smithi053 ceph-mon[27773]: from='client.24350 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:36:22.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:22 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:36:22.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:22 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:36:22.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:22 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:36:22.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:22 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:22.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:22 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:22.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:22 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:36:22.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:22 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:36:22.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:22 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:36:22.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:22 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:36:22.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:22 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:22.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:22 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:36:22.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:22 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:36:22.607 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:22 smithi150 ceph-mon[32504]: from='client.24350 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:36:22.607 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:22 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:36:22.607 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:22 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:36:22.607 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:22 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:36:22.607 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:22 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:22.608 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:22 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:22.608 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:22 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:36:22.608 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:22 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:36:22.608 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:22 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:36:22.608 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:22 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:36:22.608 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:22 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:22.608 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:22 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:36:22.608 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:22 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:36:22.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:22 smithi183 ceph-mon[32584]: from='client.24350 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:36:22.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:22 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:36:22.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:22 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:36:22.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:22 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:36:22.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:22 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:22.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:22 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:22.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:22 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:36:22.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:22 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:36:22.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:22 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:36:22.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:22 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:36:22.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:22 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:22.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:22 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:36:22.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:22 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:36:23.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:23 smithi150 ceph-mon[32504]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:36:23.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:23 smithi150 ceph-mon[32504]: Detected new or changed devices on smithi053 2024-04-25T14:36:23.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:23 smithi150 ceph-mon[32504]: Adjusting osd_memory_target on smithi053 to 4273M 2024-04-25T14:36:23.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:23 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:23.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:23 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:36:23.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:23 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:36:23.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:23 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:36:23.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:23 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:23.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:23 smithi183 ceph-mon[32584]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:36:23.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:23 smithi183 ceph-mon[32584]: Detected new or changed devices on smithi053 2024-04-25T14:36:23.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:23 smithi183 ceph-mon[32584]: Adjusting osd_memory_target on smithi053 to 4273M 2024-04-25T14:36:23.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:23 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:23.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:23 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:36:23.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:23 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:36:23.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:23 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:36:23.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:23 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:23.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:23 smithi053 ceph-mon[27773]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:36:23.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:23 smithi053 ceph-mon[27773]: Detected new or changed devices on smithi053 2024-04-25T14:36:23.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:23 smithi053 ceph-mon[27773]: Adjusting osd_memory_target on smithi053 to 4273M 2024-04-25T14:36:23.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:23 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:23.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:23 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:36:23.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:23 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:36:23.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:23 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:36:23.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:23 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:25.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:25 smithi183 ceph-mon[32584]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:36:25.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:25 smithi183 ceph-mon[32584]: from='client.? 172.21.15.150:0/2618966730' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5de4803d-ad37-4a90-add5-0048f44fc4ca"}]: dispatch 2024-04-25T14:36:25.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:25 smithi183 ceph-mon[32584]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5de4803d-ad37-4a90-add5-0048f44fc4ca"}]: dispatch 2024-04-25T14:36:25.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:25 smithi183 ceph-mon[32584]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5de4803d-ad37-4a90-add5-0048f44fc4ca"}]': finished 2024-04-25T14:36:25.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:25 smithi183 ceph-mon[32584]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T14:36:25.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:25 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:36:25.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:25 smithi183 ceph-mon[32584]: from='client.? 172.21.15.150:0/2001168214' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:36:25.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:25 smithi150 ceph-mon[32504]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:36:25.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:25 smithi150 ceph-mon[32504]: from='client.? 172.21.15.150:0/2618966730' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5de4803d-ad37-4a90-add5-0048f44fc4ca"}]: dispatch 2024-04-25T14:36:25.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:25 smithi150 ceph-mon[32504]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5de4803d-ad37-4a90-add5-0048f44fc4ca"}]: dispatch 2024-04-25T14:36:25.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:25 smithi150 ceph-mon[32504]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5de4803d-ad37-4a90-add5-0048f44fc4ca"}]': finished 2024-04-25T14:36:25.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:25 smithi150 ceph-mon[32504]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T14:36:25.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:25 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:36:25.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:25 smithi150 ceph-mon[32504]: from='client.? 172.21.15.150:0/2001168214' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:36:25.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:25 smithi053 ceph-mon[27773]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:36:25.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:25 smithi053 ceph-mon[27773]: from='client.? 172.21.15.150:0/2618966730' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5de4803d-ad37-4a90-add5-0048f44fc4ca"}]: dispatch 2024-04-25T14:36:25.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:25 smithi053 ceph-mon[27773]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5de4803d-ad37-4a90-add5-0048f44fc4ca"}]: dispatch 2024-04-25T14:36:25.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:25 smithi053 ceph-mon[27773]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5de4803d-ad37-4a90-add5-0048f44fc4ca"}]': finished 2024-04-25T14:36:25.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:25 smithi053 ceph-mon[27773]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T14:36:25.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:25 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:36:25.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:25 smithi053 ceph-mon[27773]: from='client.? 172.21.15.150:0/2001168214' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:36:27.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:27 smithi183 ceph-mon[32584]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:36:27.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:27 smithi150 ceph-mon[32504]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:36:27.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:27 smithi053 ceph-mon[27773]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:36:29.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:29 smithi183 ceph-mon[32584]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:36:29.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:29 smithi150 ceph-mon[32504]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:36:29.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:29 smithi053 ceph-mon[27773]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:36:31.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:31 smithi183 ceph-mon[32584]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:36:31.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:31 smithi150 ceph-mon[32504]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:36:31.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:31 smithi053 ceph-mon[27773]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:36:32.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:32 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T14:36:32.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:32 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:36:32.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:32 smithi150 ceph-mon[32504]: Deploying daemon osd.10 on smithi150 2024-04-25T14:36:32.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:32 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T14:36:32.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:32 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:36:32.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:32 smithi183 ceph-mon[32584]: Deploying daemon osd.10 on smithi150 2024-04-25T14:36:32.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:32 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T14:36:32.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:32 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:36:32.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:32 smithi053 ceph-mon[27773]: Deploying daemon osd.10 on smithi150 2024-04-25T14:36:33.658 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:33 smithi150 ceph-mon[32504]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:36:33.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:33 smithi183 ceph-mon[32584]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:36:33.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:33 smithi053 ceph-mon[27773]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:36:35.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:35 smithi183 ceph-mon[32584]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:36:35.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:35 smithi150 ceph-mon[32504]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:36:35.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:35 smithi053 ceph-mon[27773]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:36:37.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:37 smithi183 ceph-mon[32584]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:36:37.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:37 smithi150 ceph-mon[32504]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:36:37.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:37 smithi053 ceph-mon[27773]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:36:38.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:38 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:38.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:38 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:38.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:38 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:36:38.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:38 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:38.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:38 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:38.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:38 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:36:39.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:38 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:39.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:38 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:39.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:38 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:36:39.887 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:39 smithi150 ceph-mon[32504]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:36:39.934 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:39 smithi053 ceph-mon[27773]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:36:39.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:39 smithi183 ceph-mon[32584]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:36:41.248 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:41.248+0000 7fcc157fa700 1 -- 172.21.15.150:0/4209282675 <== mgr.14152 v2:172.21.15.53:6800/4154933108 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fcc18062e80 con 0x7fcc000a3cc0 2024-04-25T14:36:41.248 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 10 on host 'smithi150' 2024-04-25T14:36:41.250 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:41.250+0000 7fcc1e2b2700 1 -- 172.21.15.150:0/4209282675 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fcc000a3cc0 msgr2=0x7fcc000a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:36:41.250 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:41.250+0000 7fcc1e2b2700 1 --2- 172.21.15.150:0/4209282675 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fcc000a3cc0 0x7fcc000a6180 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcc08009a10 tx=0x7fcc08009190 comp rx=0 tx=0).stop 2024-04-25T14:36:41.250 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:41.250+0000 7fcc1e2b2700 1 -- 172.21.15.150:0/4209282675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc18108790 msgr2=0x7fcc181a2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:36:41.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:41.250+0000 7fcc1e2b2700 1 --2- 172.21.15.150:0/4209282675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc18108790 0x7fcc181a2590 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009fd0 tx=0x7fcc1000cf40 comp rx=0 tx=0).stop 2024-04-25T14:36:41.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:41.251+0000 7fcc1e2b2700 1 -- 172.21.15.150:0/4209282675 shutdown_connections 2024-04-25T14:36:41.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:41.251+0000 7fcc1e2b2700 1 --2- 172.21.15.150:0/4209282675 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fcc000a3cc0 0x7fcc000a6180 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:36:41.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:41.251+0000 7fcc1e2b2700 1 --2- 172.21.15.150:0/4209282675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc18108790 0x7fcc181a2590 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:36:41.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:41.251+0000 7fcc1e2b2700 1 --2- 172.21.15.150:0/4209282675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc18103530 0x7fcc1819afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:36:41.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:41.251+0000 7fcc1e2b2700 1 --2- 172.21.15.150:0/4209282675 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc18105e60 0x7fcc1819b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:36:41.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:41.251+0000 7fcc1e2b2700 1 -- 172.21.15.150:0/4209282675 >> 172.21.15.150:0/4209282675 conn(0x7fcc180fcf40 msgr2=0x7fcc18102d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:36:41.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:41.251+0000 7fcc1e2b2700 1 -- 172.21.15.150:0/4209282675 shutdown_connections 2024-04-25T14:36:41.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T14:36:41.251+0000 7fcc1e2b2700 1 -- 172.21.15.150:0/4209282675 wait complete. 2024-04-25T14:36:41.797 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:41 smithi150 ceph-mon[32504]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:36:41.797 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:41 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:41.797 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:41 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:41.798 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:41 smithi150 ceph-mon[32504]: from='osd.10 [v2:172.21.15.150:6824/3435992308,v1:172.21.15.150:6825/3435992308]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T14:36:41.798 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:41 smithi150 ceph-mon[32504]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T14:36:41.823 DEBUG:teuthology.orchestra.run.smithi150:osd.10> sudo journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@osd.10.service 2024-04-25T14:36:41.825 INFO:tasks.cephadm:Deploying osd.11 on smithi183 with /dev/vg_nvme/lv_1... 2024-04-25T14:36:41.826 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T14:36:41.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:41 smithi183 ceph-mon[32584]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:36:41.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:41 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:41.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:41 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:41.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:41 smithi183 ceph-mon[32584]: from='osd.10 [v2:172.21.15.150:6824/3435992308,v1:172.21.15.150:6825/3435992308]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T14:36:41.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:41 smithi183 ceph-mon[32584]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T14:36:42.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:41 smithi053 ceph-mon[27773]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:36:42.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:41 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:42.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:41 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:42.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:41 smithi053 ceph-mon[27773]: from='osd.10 [v2:172.21.15.150:6824/3435992308,v1:172.21.15.150:6825/3435992308]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T14:36:42.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:41 smithi053 ceph-mon[27773]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T14:36:42.075 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:36:43.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:43 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:43.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:43 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:43.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:43 smithi150 ceph-mon[32504]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T14:36:43.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:43 smithi150 ceph-mon[32504]: from='osd.10 [v2:172.21.15.150:6824/3435992308,v1:172.21.15.150:6825/3435992308]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T14:36:43.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:43 smithi150 ceph-mon[32504]: osdmap e73: 11 total, 10 up, 11 in 2024-04-25T14:36:43.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:43 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:36:43.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:43 smithi150 ceph-mon[32504]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T14:36:43.386 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:43 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:43.386 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:43 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:43.386 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:43 smithi183 ceph-mon[32584]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T14:36:43.386 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:43 smithi183 ceph-mon[32584]: from='osd.10 [v2:172.21.15.150:6824/3435992308,v1:172.21.15.150:6825/3435992308]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T14:36:43.386 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:43 smithi183 ceph-mon[32584]: osdmap e73: 11 total, 10 up, 11 in 2024-04-25T14:36:43.386 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:43 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:36:43.386 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:43 smithi183 ceph-mon[32584]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T14:36:43.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:43 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:43.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:43 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:43.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:43 smithi053 ceph-mon[27773]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T14:36:43.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:43 smithi053 ceph-mon[27773]: from='osd.10 [v2:172.21.15.150:6824/3435992308,v1:172.21.15.150:6825/3435992308]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T14:36:43.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:43 smithi053 ceph-mon[27773]: osdmap e73: 11 total, 10 up, 11 in 2024-04-25T14:36:43.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:43 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:36:43.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:43 smithi053 ceph-mon[27773]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T14:36:43.671 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:36:43 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:36:43.210+0000 7fbd7228e700 -1 osd.10 0 waiting for initial osdmap 2024-04-25T14:36:43.671 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:36:43 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:36:43.226+0000 7fbd6e081700 -1 osd.10 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:36:44.132 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T14:36:44.132 INFO:teuthology.orchestra.run.smithi183.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T14:36:44.132 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10+0 records in 2024-04-25T14:36:44.132 INFO:teuthology.orchestra.run.smithi183.stderr:10+0 records out 2024-04-25T14:36:44.133 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114752 s, 914 MB/s 2024-04-25T14:36:44.133 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping successful for: 2024-04-25T14:36:44.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:44 smithi150 ceph-mon[32504]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:36:44.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:44 smithi150 ceph-mon[32504]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-25T14:36:44.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:44 smithi150 ceph-mon[32504]: osdmap e74: 11 total, 10 up, 11 in 2024-04-25T14:36:44.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:44 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:36:44.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:44 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:36:44.435 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:44 smithi183 ceph-mon[32584]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:36:44.435 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:44 smithi183 ceph-mon[32584]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-25T14:36:44.436 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:44 smithi183 ceph-mon[32584]: osdmap e74: 11 total, 10 up, 11 in 2024-04-25T14:36:44.436 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:44 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:36:44.436 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:44 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:36:44.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:44 smithi053 ceph-mon[27773]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:36:44.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:44 smithi053 ceph-mon[27773]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-25T14:36:44.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:44 smithi053 ceph-mon[27773]: osdmap e74: 11 total, 10 up, 11 in 2024-04-25T14:36:44.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:44 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:36:44.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:44 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:36:44.725 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_1 2024-04-25T14:36:44.967 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:36:45.317 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:45 smithi183 ceph-mon[32584]: purged_snaps scrub starts 2024-04-25T14:36:45.318 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:45 smithi183 ceph-mon[32584]: purged_snaps scrub ok 2024-04-25T14:36:45.318 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:45 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:36:45.318 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:45 smithi183 ceph-mon[32584]: osd.10 [v2:172.21.15.150:6824/3435992308,v1:172.21.15.150:6825/3435992308] boot 2024-04-25T14:36:45.318 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:45 smithi183 ceph-mon[32584]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T14:36:45.318 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:45 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:36:45.445 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:45 smithi150 ceph-mon[32504]: purged_snaps scrub starts 2024-04-25T14:36:45.445 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:45 smithi150 ceph-mon[32504]: purged_snaps scrub ok 2024-04-25T14:36:45.445 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:45 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:36:45.445 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:45 smithi150 ceph-mon[32504]: osd.10 [v2:172.21.15.150:6824/3435992308,v1:172.21.15.150:6825/3435992308] boot 2024-04-25T14:36:45.445 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:45 smithi150 ceph-mon[32504]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T14:36:45.445 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:45 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:36:45.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:45 smithi053 ceph-mon[27773]: purged_snaps scrub starts 2024-04-25T14:36:45.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:45 smithi053 ceph-mon[27773]: purged_snaps scrub ok 2024-04-25T14:36:45.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:45 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:36:45.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:45 smithi053 ceph-mon[27773]: osd.10 [v2:172.21.15.150:6824/3435992308,v1:172.21.15.150:6825/3435992308] boot 2024-04-25T14:36:45.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:45 smithi053 ceph-mon[27773]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T14:36:45.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:45 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:36:46.323 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:46 smithi183 ceph-mon[32584]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:36:46.323 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:46 smithi183 ceph-mon[32584]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T14:36:46.323 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:46 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:46.323 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:46 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:46.323 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:46 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:36:46.323 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:46 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:36:46.324 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:46 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:36:46.324 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:46 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:36:46.324 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:46 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:46.324 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:46 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:36:46.324 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:46 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:36:46.324 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:46 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:46.324 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:46 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:36:46.324 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:46 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:36:46.324 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:46 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:36:46.324 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:46 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:46.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:46 smithi150 ceph-mon[32504]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:36:46.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:46 smithi150 ceph-mon[32504]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T14:36:46.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:46 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:46.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:46 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:46.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:46 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:36:46.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:46 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:36:46.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:46 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:36:46.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:46 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:36:46.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:46 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:46.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:46 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:36:46.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:46 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:36:46.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:46 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:46.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:46 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:36:46.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:46 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:36:46.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:46 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:36:46.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:46 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:46.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:46 smithi053 ceph-mon[27773]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:36:46.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:46 smithi053 ceph-mon[27773]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T14:36:46.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:46 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:46.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:46 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:46.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:46 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:36:46.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:46 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:36:46.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:46 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:36:46.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:46 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:36:46.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:46 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:46.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:46 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:36:46.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:46 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:36:46.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:46 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:46.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:46 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:36:46.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:46 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:36:46.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:46 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:36:46.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:46 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:36:46.544 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:36:46.543+0000 7f4bf0d19700 1 -- 172.21.15.183:0/780578965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bec103540 msgr2=0x7f4bec109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:36:46.544 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:36:46.543+0000 7f4bf0d19700 1 --2- 172.21.15.183:0/780578965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bec103540 0x7f4bec109df0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4be400b0a0 tx=0x7f4be400b3b0 comp rx=0 tx=0).stop 2024-04-25T14:36:46.544 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:36:46.543+0000 7f4bf0d19700 1 -- 172.21.15.183:0/780578965 shutdown_connections 2024-04-25T14:36:46.544 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:36:46.543+0000 7f4bf0d19700 1 --2- 172.21.15.183:0/780578965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bec103540 0x7f4bec109df0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:36:46.544 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:36:46.543+0000 7f4bf0d19700 1 --2- 172.21.15.183:0/780578965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bec1019a0 0x7f4bec101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:36:46.544 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:36:46.543+0000 7f4bf0d19700 1 --2- 172.21.15.183:0/780578965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bec102ba0 0x7f4bec103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:36:46.544 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:36:46.543+0000 7f4bf0d19700 1 -- 172.21.15.183:0/780578965 >> 172.21.15.183:0/780578965 conn(0x7f4bec0fcf40 msgr2=0x7f4bec0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:36:46.544 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:36:46.543+0000 7f4bf0d19700 1 -- 172.21.15.183:0/780578965 shutdown_connections 2024-04-25T14:36:46.544 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:36:46.543+0000 7f4bf0d19700 1 -- 172.21.15.183:0/780578965 wait complete. 2024-04-25T14:36:46.546 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:36:46.544+0000 7f4bf0d19700 1 Processor -- start 2024-04-25T14:36:46.546 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:36:46.544+0000 7f4bf0d19700 1 -- start start 2024-04-25T14:36:46.546 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:36:46.544+0000 7f4bf0d19700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bec1019a0 0x7f4bec198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:36:46.546 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:36:46.544+0000 7f4bf0d19700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bec102ba0 0x7f4bec1991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:36:46.546 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:36:46.544+0000 7f4bf0d19700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bec103540 0x7f4bec1a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:36:46.546 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:36:46.544+0000 7f4bf0d19700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4bec10d4d0 con 0x7f4bec103540 2024-04-25T14:36:46.546 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:36:46.544+0000 7f4bf0d19700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4bec10d350 con 0x7f4bec1019a0 2024-04-25T14:36:46.546 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:36:46.544+0000 7f4bf0d19700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4bec10d650 con 0x7f4bec102ba0 2024-04-25T14:36:46.547 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:36:46.544+0000 7f4be1d9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bec102ba0 0x7f4bec1991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:36:46.547 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:36:46.544+0000 7f4be1d9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bec102ba0 0x7f4bec1991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:36764/0 (socket says 172.21.15.183:36764) 2024-04-25T14:36:46.547 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:36:46.544+0000 7f4bea59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bec1019a0 0x7f4bec198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:36:46.547 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:36:46.544+0000 7f4be1d9b700 1 -- 172.21.15.183:0/173247389 learned_addr learned my addr 172.21.15.183:0/173247389 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-04-25T14:36:46.547 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:36:46.545+0000 7f4be1d9b700 1 -- 172.21.15.183:0/173247389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bec1019a0 msgr2=0x7f4bec198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:36:46.547 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:36:46.545+0000 7f4be1d9b700 1 --2- 172.21.15.183:0/173247389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bec1019a0 0x7f4bec198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:36:46.547 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:36:46.545+0000 7f4be1d9b700 1 -- 172.21.15.183:0/173247389 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bec103540 msgr2=0x7f4bec1a01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:36:46.547 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:36:46.545+0000 7f4be1d9b700 1 --2- 172.21.15.183:0/173247389 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bec103540 0x7f4bec1a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:36:46.547 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:36:46.545+0000 7f4be1d9b700 1 -- 172.21.15.183:0/173247389 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bec1a0890 con 0x7f4bec102ba0 2024-04-25T14:36:46.547 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:36:46.545+0000 7f4be1d9b700 1 --2- 172.21.15.183:0/173247389 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bec102ba0 0x7f4bec1991d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4bd400d980 tx=0x7f4bd400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:36:46.547 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:36:46.545+0000 7f4be3fff700 1 -- 172.21.15.183:0/173247389 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd4017070 con 0x7f4bec102ba0 2024-04-25T14:36:46.547 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:36:46.545+0000 7f4be3fff700 1 -- 172.21.15.183:0/173247389 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bd4005020 con 0x7f4bec102ba0 2024-04-25T14:36:46.547 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:36:46.545+0000 7f4bf0d19700 1 -- 172.21.15.183:0/173247389 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bec1a0b80 con 0x7f4bec102ba0 2024-04-25T14:36:46.547 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:36:46.545+0000 7f4be3fff700 1 -- 172.21.15.183:0/173247389 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd4009b50 con 0x7f4bec102ba0 2024-04-25T14:36:46.548 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:36:46.545+0000 7f4bf0d19700 1 -- 172.21.15.183:0/173247389 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bec1a10f0 con 0x7f4bec102ba0 2024-04-25T14:36:46.548 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:36:46.546+0000 7f4bf0d19700 1 -- 172.21.15.183:0/173247389 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bec1a12c0 con 0x7f4bec102ba0 2024-04-25T14:36:46.548 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:36:46.547+0000 7f4be3fff700 1 -- 172.21.15.183:0/173247389 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4bd40051d0 con 0x7f4bec102ba0 2024-04-25T14:36:46.548 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:36:46.548+0000 7f4be3fff700 1 --2- 172.21.15.183:0/173247389 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f4bd80a3c40 0x7f4bd80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:36:46.549 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:36:46.548+0000 7f4be3fff700 1 -- 172.21.15.183:0/173247389 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6723+0+0 (secure 0 0 0) 0x7f4bd40cfd10 con 0x7f4bec102ba0 2024-04-25T14:36:46.549 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:36:46.548+0000 7f4bea59c700 1 --2- 172.21.15.183:0/173247389 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f4bd80a3c40 0x7f4bd80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:36:46.549 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:36:46.549+0000 7f4bea59c700 1 --2- 172.21.15.183:0/173247389 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f4bd80a3c40 0x7f4bd80a6100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4bec19a120 tx=0x7f4bdc009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:36:46.553 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:36:46.552+0000 7f4be3fff700 1 -- 172.21.15.183:0/173247389 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4bd4020020 con 0x7f4bec102ba0 2024-04-25T14:36:46.767 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:36:46.766+0000 7f4bf0d19700 1 -- 172.21.15.183:0/173247389 --> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f4bec0ff150 con 0x7f4bd80a3c40 2024-04-25T14:36:47.408 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:47 smithi183 ceph-mon[32584]: Detected new or changed devices on smithi150 2024-04-25T14:36:47.408 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:47 smithi183 ceph-mon[32584]: Adjusting osd_memory_target on smithi150 to 4272M 2024-04-25T14:36:47.408 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:47 smithi183 ceph-mon[32584]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T14:36:47.408 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:47 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:36:47.408 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:47 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:36:47.408 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:47 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:36:47.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:47 smithi150 ceph-mon[32504]: Detected new or changed devices on smithi150 2024-04-25T14:36:47.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:47 smithi150 ceph-mon[32504]: Adjusting osd_memory_target on smithi150 to 4272M 2024-04-25T14:36:47.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:47 smithi150 ceph-mon[32504]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T14:36:47.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:47 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:36:47.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:47 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:36:47.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:47 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:36:47.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:47 smithi053 ceph-mon[27773]: Detected new or changed devices on smithi150 2024-04-25T14:36:47.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:47 smithi053 ceph-mon[27773]: Adjusting osd_memory_target on smithi150 to 4272M 2024-04-25T14:36:47.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:47 smithi053 ceph-mon[27773]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T14:36:47.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:47 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:36:47.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:47 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:36:47.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:47 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:36:48.346 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:48 smithi183 ceph-mon[32584]: pgmap v263: 1 pgs: 1 remapped+peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:36:48.346 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:48 smithi183 ceph-mon[32584]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:36:48.346 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:48 smithi183 ceph-mon[32584]: osdmap e78: 11 total, 11 up, 11 in 2024-04-25T14:36:48.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:48 smithi150 ceph-mon[32504]: pgmap v263: 1 pgs: 1 remapped+peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:36:48.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:48 smithi150 ceph-mon[32504]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:36:48.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:48 smithi150 ceph-mon[32504]: osdmap e78: 11 total, 11 up, 11 in 2024-04-25T14:36:48.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:48 smithi053 ceph-mon[27773]: pgmap v263: 1 pgs: 1 remapped+peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:36:48.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:48 smithi053 ceph-mon[27773]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:36:48.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:48 smithi053 ceph-mon[27773]: osdmap e78: 11 total, 11 up, 11 in 2024-04-25T14:36:49.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:49 smithi053 ceph-mon[27773]: osdmap e79: 11 total, 11 up, 11 in 2024-04-25T14:36:49.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:49 smithi053 ceph-mon[27773]: pgmap v266: 1 pgs: 1 remapped+peering; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:36:49.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:49 smithi183 ceph-mon[32584]: osdmap e79: 11 total, 11 up, 11 in 2024-04-25T14:36:49.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:49 smithi183 ceph-mon[32584]: pgmap v266: 1 pgs: 1 remapped+peering; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:36:49.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:49 smithi150 ceph-mon[32504]: osdmap e79: 11 total, 11 up, 11 in 2024-04-25T14:36:49.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:49 smithi150 ceph-mon[32504]: pgmap v266: 1 pgs: 1 remapped+peering; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:36:50.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:50 smithi053 ceph-mon[27773]: from='client.? 172.21.15.183:0/679396444' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e819a68-ed56-477e-b9c0-81e8cc0bce11"}]: dispatch 2024-04-25T14:36:50.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:50 smithi053 ceph-mon[27773]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e819a68-ed56-477e-b9c0-81e8cc0bce11"}]: dispatch 2024-04-25T14:36:50.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:50 smithi053 ceph-mon[27773]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e819a68-ed56-477e-b9c0-81e8cc0bce11"}]': finished 2024-04-25T14:36:50.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:50 smithi053 ceph-mon[27773]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T14:36:50.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:50 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:36:50.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:50 smithi053 ceph-mon[27773]: from='client.? 172.21.15.183:0/148715219' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:36:50.612 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:50 smithi150 ceph-mon[32504]: from='client.? 172.21.15.183:0/679396444' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e819a68-ed56-477e-b9c0-81e8cc0bce11"}]: dispatch 2024-04-25T14:36:50.612 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:50 smithi150 ceph-mon[32504]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e819a68-ed56-477e-b9c0-81e8cc0bce11"}]: dispatch 2024-04-25T14:36:50.612 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:50 smithi150 ceph-mon[32504]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e819a68-ed56-477e-b9c0-81e8cc0bce11"}]': finished 2024-04-25T14:36:50.612 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:50 smithi150 ceph-mon[32504]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T14:36:50.612 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:50 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:36:50.612 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:50 smithi150 ceph-mon[32504]: from='client.? 172.21.15.183:0/148715219' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:36:50.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:50 smithi183 ceph-mon[32584]: from='client.? 172.21.15.183:0/679396444' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e819a68-ed56-477e-b9c0-81e8cc0bce11"}]: dispatch 2024-04-25T14:36:50.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:50 smithi183 ceph-mon[32584]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e819a68-ed56-477e-b9c0-81e8cc0bce11"}]: dispatch 2024-04-25T14:36:50.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:50 smithi183 ceph-mon[32584]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e819a68-ed56-477e-b9c0-81e8cc0bce11"}]': finished 2024-04-25T14:36:50.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:50 smithi183 ceph-mon[32584]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T14:36:50.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:50 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:36:50.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:50 smithi183 ceph-mon[32584]: from='client.? 172.21.15.183:0/148715219' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:36:51.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:51 smithi053 ceph-mon[27773]: pgmap v268: 1 pgs: 1 remapped+peering; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:36:51.631 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:51 smithi150 ceph-mon[32504]: pgmap v268: 1 pgs: 1 remapped+peering; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:36:51.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:51 smithi183 ceph-mon[32584]: pgmap v268: 1 pgs: 1 remapped+peering; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:36:53.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:53 smithi183 ceph-mon[32584]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T14:36:53.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:53 smithi150 ceph-mon[32504]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T14:36:53.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:53 smithi053 ceph-mon[27773]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T14:36:55.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:55 smithi183 ceph-mon[32584]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-04-25T14:36:55.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:55 smithi150 ceph-mon[32504]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-04-25T14:36:55.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:55 smithi053 ceph-mon[27773]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-04-25T14:36:57.920 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:57 smithi183 ceph-mon[32584]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T14:36:57.920 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:57 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T14:36:57.920 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:57 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:36:57.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:57 smithi150 ceph-mon[32504]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T14:36:57.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:57 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T14:36:57.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:57 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:36:58.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:57 smithi053 ceph-mon[27773]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T14:36:58.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:57 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T14:36:58.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:57 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:36:58.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:58 smithi183 ceph-mon[32584]: Deploying daemon osd.11 on smithi183 2024-04-25T14:36:59.154 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:58 smithi053 ceph-mon[27773]: Deploying daemon osd.11 on smithi183 2024-04-25T14:36:59.155 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:58 smithi150 ceph-mon[32504]: Deploying daemon osd.11 on smithi183 2024-04-25T14:36:59.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:36:59 smithi183 ceph-mon[32584]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T14:36:59.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:36:59 smithi150 ceph-mon[32504]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T14:37:00.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:36:59 smithi053 ceph-mon[27773]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T14:37:01.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:01 smithi183 ceph-mon[32584]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-04-25T14:37:01.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:01 smithi150 ceph-mon[32504]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-04-25T14:37:02.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:01 smithi053 ceph-mon[27773]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-04-25T14:37:04.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:03 smithi053 ceph-mon[27773]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T14:37:04.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:03 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:04.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:03 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:04.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:03 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:37:04.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:03 smithi183 ceph-mon[32584]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T14:37:04.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:03 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:04.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:03 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:04.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:03 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:37:04.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:03 smithi150 ceph-mon[32504]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T14:37:04.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:03 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:04.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:03 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:04.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:03 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:37:06.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:06 smithi183 ceph-mon[32584]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:37:06.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:06 smithi053 ceph-mon[27773]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:37:06.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:06 smithi150 ceph-mon[32504]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:37:06.956 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:06.956+0000 7f4be3fff700 1 -- 172.21.15.183:0/173247389 <== mgr.14152 v2:172.21.15.53:6800/4154933108 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f4bec0ff150 con 0x7f4bd80a3c40 2024-04-25T14:37:06.957 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 11 on host 'smithi183' 2024-04-25T14:37:06.959 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:06.959+0000 7f4bf0d19700 1 -- 172.21.15.183:0/173247389 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f4bd80a3c40 msgr2=0x7f4bd80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:06.959 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:06.959+0000 7f4bf0d19700 1 --2- 172.21.15.183:0/173247389 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f4bd80a3c40 0x7f4bd80a6100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4bec19a120 tx=0x7f4bdc009190 comp rx=0 tx=0).stop 2024-04-25T14:37:06.959 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:06.959+0000 7f4bf0d19700 1 -- 172.21.15.183:0/173247389 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bec102ba0 msgr2=0x7f4bec1991d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:06.959 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:06.959+0000 7f4bf0d19700 1 --2- 172.21.15.183:0/173247389 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bec102ba0 0x7f4bec1991d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4bd400d980 tx=0x7f4bd400dc90 comp rx=0 tx=0).stop 2024-04-25T14:37:06.959 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:06.959+0000 7f4bf0d19700 1 -- 172.21.15.183:0/173247389 shutdown_connections 2024-04-25T14:37:06.959 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:06.959+0000 7f4bf0d19700 1 --2- 172.21.15.183:0/173247389 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f4bd80a3c40 0x7f4bd80a6100 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:06.959 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:06.959+0000 7f4bf0d19700 1 --2- 172.21.15.183:0/173247389 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bec103540 0x7f4bec1a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:06.959 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:06.959+0000 7f4bf0d19700 1 --2- 172.21.15.183:0/173247389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bec1019a0 0x7f4bec198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:06.959 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:06.959+0000 7f4bf0d19700 1 --2- 172.21.15.183:0/173247389 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bec102ba0 0x7f4bec1991d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:06.959 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:06.959+0000 7f4bf0d19700 1 -- 172.21.15.183:0/173247389 >> 172.21.15.183:0/173247389 conn(0x7f4bec0fcf40 msgr2=0x7f4bec0fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:37:06.959 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:06.959+0000 7f4bf0d19700 1 -- 172.21.15.183:0/173247389 shutdown_connections 2024-04-25T14:37:06.959 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:06.959+0000 7f4bf0d19700 1 -- 172.21.15.183:0/173247389 wait complete. 2024-04-25T14:37:07.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:07 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:07.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:07 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:07.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:07 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:37:07.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:07 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:37:07.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:07 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:07.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:07 smithi183 ceph-mon[32584]: from='osd.11 [v2:172.21.15.183:6824/1957842686,v1:172.21.15.183:6825/1957842686]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T14:37:07.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:07 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:07.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:07 smithi183 ceph-mon[32584]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T14:37:07.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:07 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:07.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:07 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:37:07.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:07 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:07.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:07 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:07.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:07 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:37:07.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:07 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:37:07.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:07 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:07.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:07 smithi150 ceph-mon[32504]: from='osd.11 [v2:172.21.15.183:6824/1957842686,v1:172.21.15.183:6825/1957842686]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T14:37:07.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:07 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:07.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:07 smithi150 ceph-mon[32504]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T14:37:07.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:07 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:07.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:07 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:37:07.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:07 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:07.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:07 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:07.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:07 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:37:07.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:07 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:37:07.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:07 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:07.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:07 smithi053 ceph-mon[27773]: from='osd.11 [v2:172.21.15.183:6824/1957842686,v1:172.21.15.183:6825/1957842686]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T14:37:07.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:07 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:07.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:07 smithi053 ceph-mon[27773]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T14:37:07.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:07 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:07.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:07 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:37:07.625 DEBUG:teuthology.orchestra.run.smithi183:osd.11> sudo journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@osd.11.service 2024-04-25T14:37:07.627 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-25T14:37:07.628 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T14:37:08.092 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:37:08.417 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:08 smithi053 ceph-mon[27773]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:37:08.417 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:08 smithi053 ceph-mon[27773]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T14:37:08.417 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:08 smithi053 ceph-mon[27773]: from='osd.11 [v2:172.21.15.183:6824/1957842686,v1:172.21.15.183:6825/1957842686]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T14:37:08.417 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:08 smithi053 ceph-mon[27773]: osdmap e81: 12 total, 11 up, 12 in 2024-04-25T14:37:08.417 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:08 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:37:08.417 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:08 smithi053 ceph-mon[27773]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T14:37:08.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:08 smithi183 ceph-mon[32584]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:37:08.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:08 smithi183 ceph-mon[32584]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T14:37:08.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:08 smithi183 ceph-mon[32584]: from='osd.11 [v2:172.21.15.183:6824/1957842686,v1:172.21.15.183:6825/1957842686]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T14:37:08.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:08 smithi183 ceph-mon[32584]: osdmap e81: 12 total, 11 up, 12 in 2024-04-25T14:37:08.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:08 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:37:08.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:08 smithi183 ceph-mon[32584]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T14:37:08.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:08 smithi150 ceph-mon[32504]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:37:08.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:08 smithi150 ceph-mon[32504]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T14:37:08.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:08 smithi150 ceph-mon[32504]: from='osd.11 [v2:172.21.15.183:6824/1957842686,v1:172.21.15.183:6825/1957842686]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T14:37:08.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:08 smithi150 ceph-mon[32504]: osdmap e81: 12 total, 11 up, 12 in 2024-04-25T14:37:08.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:08 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:37:08.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:08 smithi150 ceph-mon[32504]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T14:37:10.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:09 smithi183 ceph-mon[32584]: purged_snaps scrub starts 2024-04-25T14:37:10.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:09 smithi183 ceph-mon[32584]: purged_snaps scrub ok 2024-04-25T14:37:10.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:09 smithi183 ceph-mon[32584]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:37:10.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:09 smithi183 ceph-mon[32584]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-25T14:37:10.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:09 smithi183 ceph-mon[32584]: osdmap e82: 12 total, 11 up, 12 in 2024-04-25T14:37:10.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:09 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:37:10.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:09 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:37:10.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:09 smithi183 ceph-mon[32584]: from='osd.11 ' entity='osd.11' 2024-04-25T14:37:10.205 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:37:09 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:37:09.815+0000 7f26c3bb2700 -1 osd.11 0 waiting for initial osdmap 2024-04-25T14:37:10.205 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:37:10 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:37:10.013+0000 7f26bb99d700 -1 osd.11 82 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:37:10.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:09 smithi150 ceph-mon[32504]: purged_snaps scrub starts 2024-04-25T14:37:10.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:09 smithi150 ceph-mon[32504]: purged_snaps scrub ok 2024-04-25T14:37:10.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:09 smithi150 ceph-mon[32504]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:37:10.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:09 smithi150 ceph-mon[32504]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-25T14:37:10.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:09 smithi150 ceph-mon[32504]: osdmap e82: 12 total, 11 up, 12 in 2024-04-25T14:37:10.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:09 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:37:10.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:09 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:37:10.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:09 smithi150 ceph-mon[32504]: from='osd.11 ' entity='osd.11' 2024-04-25T14:37:10.218 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:09 smithi053 ceph-mon[27773]: purged_snaps scrub starts 2024-04-25T14:37:10.218 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:09 smithi053 ceph-mon[27773]: purged_snaps scrub ok 2024-04-25T14:37:10.218 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:09 smithi053 ceph-mon[27773]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:37:10.218 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:09 smithi053 ceph-mon[27773]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-25T14:37:10.219 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:09 smithi053 ceph-mon[27773]: osdmap e82: 12 total, 11 up, 12 in 2024-04-25T14:37:10.219 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:09 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:37:10.219 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:09 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:37:10.219 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:09 smithi053 ceph-mon[27773]: from='osd.11 ' entity='osd.11' 2024-04-25T14:37:10.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.560+0000 7f37f3400700 1 -- 172.21.15.53:0/678537372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37ec103520 msgr2=0x7f37ec105910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:10.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.560+0000 7f37f3400700 1 --2- 172.21.15.53:0/678537372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37ec103520 0x7f37ec105910 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f37e00098f0 tx=0x7f37e0009c00 comp rx=0 tx=0).stop 2024-04-25T14:37:10.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.560+0000 7f37f3400700 1 -- 172.21.15.53:0/678537372 shutdown_connections 2024-04-25T14:37:10.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.560+0000 7f37f3400700 1 --2- 172.21.15.53:0/678537372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37ec108780 0x7f37ec10abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:10.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.560+0000 7f37f3400700 1 --2- 172.21.15.53:0/678537372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37ec103520 0x7f37ec105910 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:10.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.560+0000 7f37f3400700 1 --2- 172.21.15.53:0/678537372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37ec105e50 0x7f37ec108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:10.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.560+0000 7f37f3400700 1 -- 172.21.15.53:0/678537372 >> 172.21.15.53:0/678537372 conn(0x7f37ec0fcf30 msgr2=0x7f37ec0ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:37:10.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.560+0000 7f37f3400700 1 -- 172.21.15.53:0/678537372 shutdown_connections 2024-04-25T14:37:10.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.560+0000 7f37f3400700 1 -- 172.21.15.53:0/678537372 wait complete. 2024-04-25T14:37:10.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.561+0000 7f37f3400700 1 Processor -- start 2024-04-25T14:37:10.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.561+0000 7f37f3400700 1 -- start start 2024-04-25T14:37:10.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.562+0000 7f37f3400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37ec103520 0x7f37ec19aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:10.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.562+0000 7f37f3400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37ec105e50 0x7f37ec19b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:10.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.562+0000 7f37f3400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37ec108780 0x7f37ec1a2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:10.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.562+0000 7f37f3400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec10e560 con 0x7f37ec103520 2024-04-25T14:37:10.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.562+0000 7f37f3400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec10e3e0 con 0x7f37ec108780 2024-04-25T14:37:10.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.562+0000 7f37f3400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec10e6e0 con 0x7f37ec105e50 2024-04-25T14:37:10.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.562+0000 7f37f0a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37ec103520 0x7f37ec19aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:10.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.562+0000 7f37f0a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37ec103520 0x7f37ec19aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38432/0 (socket says 172.21.15.53:38432) 2024-04-25T14:37:10.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.562+0000 7f37f0a00700 1 -- 172.21.15.53:0/697105738 learned_addr learned my addr 172.21.15.53:0/697105738 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:37:10.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.562+0000 7f37ebe00700 1 --2- 172.21.15.53:0/697105738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37ec105e50 0x7f37ec19b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:10.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.562+0000 7f37f1400700 1 --2- 172.21.15.53:0/697105738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37ec108780 0x7f37ec1a2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:10.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.562+0000 7f37f0a00700 1 -- 172.21.15.53:0/697105738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37ec105e50 msgr2=0x7f37ec19b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:10.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.562+0000 7f37f0a00700 1 --2- 172.21.15.53:0/697105738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37ec105e50 0x7f37ec19b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:10.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.562+0000 7f37f0a00700 1 -- 172.21.15.53:0/697105738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37ec108780 msgr2=0x7f37ec1a2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:10.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.562+0000 7f37f0a00700 1 --2- 172.21.15.53:0/697105738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37ec108780 0x7f37ec1a2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:10.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.562+0000 7f37f0a00700 1 -- 172.21.15.53:0/697105738 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37ec1a2b10 con 0x7f37ec103520 2024-04-25T14:37:10.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.562+0000 7f37ebe00700 1 --2- 172.21.15.53:0/697105738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37ec105e50 0x7f37ec19b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:37:10.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.563+0000 7f37f0a00700 1 --2- 172.21.15.53:0/697105738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37ec103520 0x7f37ec19aeb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f37e0009810 tx=0x7f37e0005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:37:10.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.563+0000 7f37e9600700 1 -- 172.21.15.53:0/697105738 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e0025070 con 0x7f37ec103520 2024-04-25T14:37:10.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.563+0000 7f37f3400700 1 -- 172.21.15.53:0/697105738 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37ec1a2da0 con 0x7f37ec103520 2024-04-25T14:37:10.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.563+0000 7f37e9600700 1 -- 172.21.15.53:0/697105738 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37e0005670 con 0x7f37ec103520 2024-04-25T14:37:10.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.563+0000 7f37e9600700 1 -- 172.21.15.53:0/697105738 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e0003c10 con 0x7f37ec103520 2024-04-25T14:37:10.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.563+0000 7f37f3400700 1 -- 172.21.15.53:0/697105738 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37ec1a31a0 con 0x7f37ec103520 2024-04-25T14:37:10.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.565+0000 7f37f3400700 1 -- 172.21.15.53:0/697105738 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37ec069730 con 0x7f37ec103520 2024-04-25T14:37:10.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.567+0000 7f37e9600700 1 -- 172.21.15.53:0/697105738 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f37e0028070 con 0x7f37ec103520 2024-04-25T14:37:10.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.568+0000 7f37e9600700 1 --2- 172.21.15.53:0/697105738 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f37d40a3c40 0x7f37d40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:10.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.568+0000 7f37e9600700 1 -- 172.21.15.53:0/697105738 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6847+0+0 (secure 0 0 0) 0x7f37e00d0640 con 0x7f37ec103520 2024-04-25T14:37:10.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.568+0000 7f37ebe00700 1 --2- 172.21.15.53:0/697105738 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f37d40a3c40 0x7f37d40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:10.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.569+0000 7f37ebe00700 1 --2- 172.21.15.53:0/697105738 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f37d40a3c40 0x7f37d40a6100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f37d8011490 tx=0x7f37d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:37:10.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.573+0000 7f37e9600700 1 -- 172.21.15.53:0/697105738 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f37e0013030 con 0x7f37ec103520 2024-04-25T14:37:10.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.817+0000 7f37f3400700 1 -- 172.21.15.53:0/697105738 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f37ec063650 con 0x7f37ec103520 2024-04-25T14:37:10.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.818+0000 7f37e9600700 1 -- 172.21.15.53:0/697105738 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v82) ==== 74+0+133 (secure 0 0 0) 0x7f37e009d5d0 con 0x7f37ec103520 2024-04-25T14:37:10.819 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:37:10.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.820+0000 7f37f3400700 1 -- 172.21.15.53:0/697105738 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f37d40a3c40 msgr2=0x7f37d40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:10.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.820+0000 7f37f3400700 1 --2- 172.21.15.53:0/697105738 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f37d40a3c40 0x7f37d40a6100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f37d8011490 tx=0x7f37d8009260 comp rx=0 tx=0).stop 2024-04-25T14:37:10.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.820+0000 7f37f3400700 1 -- 172.21.15.53:0/697105738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37ec103520 msgr2=0x7f37ec19aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:10.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.820+0000 7f37f3400700 1 --2- 172.21.15.53:0/697105738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37ec103520 0x7f37ec19aeb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f37e0009810 tx=0x7f37e0005270 comp rx=0 tx=0).stop 2024-04-25T14:37:10.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.820+0000 7f37f3400700 1 -- 172.21.15.53:0/697105738 shutdown_connections 2024-04-25T14:37:10.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.820+0000 7f37f3400700 1 --2- 172.21.15.53:0/697105738 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f37d40a3c40 0x7f37d40a6100 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:10.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.820+0000 7f37f3400700 1 --2- 172.21.15.53:0/697105738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37ec108780 0x7f37ec1a2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:10.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.820+0000 7f37f3400700 1 --2- 172.21.15.53:0/697105738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37ec103520 0x7f37ec19aeb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:10.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.820+0000 7f37f3400700 1 --2- 172.21.15.53:0/697105738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37ec105e50 0x7f37ec19b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:10.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.820+0000 7f37f3400700 1 -- 172.21.15.53:0/697105738 >> 172.21.15.53:0/697105738 conn(0x7f37ec0fcf30 msgr2=0x7f37ec102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:37:10.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.821+0000 7f37f3400700 1 -- 172.21.15.53:0/697105738 shutdown_connections 2024-04-25T14:37:10.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:10.821+0000 7f37f3400700 1 -- 172.21.15.53:0/697105738 wait complete. 2024-04-25T14:37:11.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:10 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:37:11.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:10 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:11.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:10 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:11.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:10 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:37:11.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:10 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:37:11.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:10 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:37:11.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:10 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:37:11.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:10 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:11.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:10 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:37:11.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:10 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:37:11.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:10 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:11.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:10 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/697105738' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T14:37:11.205 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:10 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:37:11.205 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:10 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:11.205 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:10 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:11.205 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:10 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:37:11.205 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:10 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:37:11.205 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:10 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:37:11.206 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:10 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:37:11.206 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:10 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:11.206 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:10 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:37:11.206 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:10 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:37:11.206 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:10 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:11.206 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:10 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/697105738' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T14:37:11.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:10 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:37:11.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:10 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:11.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:10 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:11.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:10 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:37:11.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:10 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:37:11.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:10 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:37:11.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:10 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:37:11.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:10 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:11.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:10 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:37:11.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:10 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:37:11.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:10 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:11.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:10 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/697105738' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T14:37:11.722 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":82,"num_osds":12,"num_up_osds":11,"osd_up_since":1714055804,"num_in_osds":12,"osd_in_since":1714055809,"num_remapped_pgs":0} 2024-04-25T14:37:12.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:11 smithi183 ceph-mon[32584]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:37:12.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:11 smithi183 ceph-mon[32584]: Detected new or changed devices on smithi183 2024-04-25T14:37:12.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:11 smithi183 ceph-mon[32584]: Adjusting osd_memory_target on smithi183 to 4272M 2024-04-25T14:37:12.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:37:12.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:11 smithi183 ceph-mon[32584]: osd.11 [v2:172.21.15.183:6824/1957842686,v1:172.21.15.183:6825/1957842686] boot 2024-04-25T14:37:12.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:11 smithi183 ceph-mon[32584]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T14:37:12.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:11 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:37:12.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:11 smithi150 ceph-mon[32504]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:37:12.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:11 smithi150 ceph-mon[32504]: Detected new or changed devices on smithi183 2024-04-25T14:37:12.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:11 smithi150 ceph-mon[32504]: Adjusting osd_memory_target on smithi183 to 4272M 2024-04-25T14:37:12.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:11 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:37:12.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:11 smithi150 ceph-mon[32504]: osd.11 [v2:172.21.15.183:6824/1957842686,v1:172.21.15.183:6825/1957842686] boot 2024-04-25T14:37:12.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:11 smithi150 ceph-mon[32504]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T14:37:12.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:11 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:37:12.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:11 smithi053 ceph-mon[27773]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:37:12.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:11 smithi053 ceph-mon[27773]: Detected new or changed devices on smithi183 2024-04-25T14:37:12.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:11 smithi053 ceph-mon[27773]: Adjusting osd_memory_target on smithi183 to 4272M 2024-04-25T14:37:12.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:11 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:37:12.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:11 smithi053 ceph-mon[27773]: osd.11 [v2:172.21.15.183:6824/1957842686,v1:172.21.15.183:6825/1957842686] boot 2024-04-25T14:37:12.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:11 smithi053 ceph-mon[27773]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T14:37:12.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:11 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:37:12.723 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T14:37:13.175 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:37:13.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:12 smithi183 ceph-mon[32584]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T14:37:13.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:12 smithi150 ceph-mon[32504]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T14:37:13.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:12 smithi053 ceph-mon[27773]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T14:37:14.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:13 smithi183 ceph-mon[32584]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:14.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:13 smithi150 ceph-mon[32504]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:14.273 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:13 smithi053 ceph-mon[27773]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:15.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.552+0000 7f3fc9800700 1 -- 172.21.15.53:0/3158917206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fc4105e50 msgr2=0x7f3fc4108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:15.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.552+0000 7f3fc9800700 1 --2- 172.21.15.53:0/3158917206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fc4105e50 0x7f3fc4108240 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0009990 tx=0x7f3fb0009ca0 comp rx=0 tx=0).stop 2024-04-25T14:37:15.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.553+0000 7f3fc9800700 1 -- 172.21.15.53:0/3158917206 shutdown_connections 2024-04-25T14:37:15.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.553+0000 7f3fc9800700 1 --2- 172.21.15.53:0/3158917206 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fc4108780 0x7f3fc410abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:15.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.553+0000 7f3fc9800700 1 --2- 172.21.15.53:0/3158917206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fc4103520 0x7f3fc4105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:15.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.553+0000 7f3fc9800700 1 --2- 172.21.15.53:0/3158917206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fc4105e50 0x7f3fc4108240 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:15.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.553+0000 7f3fc9800700 1 -- 172.21.15.53:0/3158917206 >> 172.21.15.53:0/3158917206 conn(0x7f3fc40fcf30 msgr2=0x7f3fc40ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:37:15.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.553+0000 7f3fc9800700 1 -- 172.21.15.53:0/3158917206 shutdown_connections 2024-04-25T14:37:15.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.553+0000 7f3fc9800700 1 -- 172.21.15.53:0/3158917206 wait complete. 2024-04-25T14:37:15.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.554+0000 7f3fc9800700 1 Processor -- start 2024-04-25T14:37:15.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.554+0000 7f3fc9800700 1 -- start start 2024-04-25T14:37:15.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.554+0000 7f3fc9800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fc4103520 0x7f3fc4102e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:15.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.554+0000 7f3fc9800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fc4105e50 0x7f3fc41014d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:15.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.554+0000 7f3fc9800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fc4108780 0x7f3fc4101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:15.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.554+0000 7f3fc9800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc410e520 con 0x7f3fc4103520 2024-04-25T14:37:15.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.554+0000 7f3fc9800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc410e3a0 con 0x7f3fc4105e50 2024-04-25T14:37:15.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.554+0000 7f3fc9800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc410e6a0 con 0x7f3fc4108780 2024-04-25T14:37:15.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.555+0000 7f3fc2a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fc4103520 0x7f3fc4102e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:15.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.555+0000 7f3fc2a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fc4103520 0x7f3fc4102e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56948/0 (socket says 172.21.15.53:56948) 2024-04-25T14:37:15.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.555+0000 7f3fc2a00700 1 -- 172.21.15.53:0/1965818573 learned_addr learned my addr 172.21.15.53:0/1965818573 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:37:15.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.555+0000 7f3fc2000700 1 --2- 172.21.15.53:0/1965818573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fc4105e50 0x7f3fc41014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:15.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.555+0000 7f3fc3400700 1 --2- 172.21.15.53:0/1965818573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fc4108780 0x7f3fc4101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:15.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.555+0000 7f3fc2a00700 1 -- 172.21.15.53:0/1965818573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fc4108780 msgr2=0x7f3fc4101a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:15.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.555+0000 7f3fc2a00700 1 --2- 172.21.15.53:0/1965818573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fc4108780 0x7f3fc4101a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:15.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.555+0000 7f3fc2a00700 1 -- 172.21.15.53:0/1965818573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fc4105e50 msgr2=0x7f3fc41014d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:15.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.555+0000 7f3fc2a00700 1 --2- 172.21.15.53:0/1965818573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fc4105e50 0x7f3fc41014d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:15.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.555+0000 7f3fc2a00700 1 -- 172.21.15.53:0/1965818573 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fc41022c0 con 0x7f3fc4103520 2024-04-25T14:37:15.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.556+0000 7f3fc2a00700 1 --2- 172.21.15.53:0/1965818573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fc4103520 0x7f3fc4102e80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400e9a0 tx=0x7f3fb400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:37:15.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.556+0000 7f3fbb400700 1 -- 172.21.15.53:0/1965818573 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb400ca10 con 0x7f3fc4103520 2024-04-25T14:37:15.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.556+0000 7f3fbb400700 1 -- 172.21.15.53:0/1965818573 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fb400cbb0 con 0x7f3fc4103520 2024-04-25T14:37:15.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.556+0000 7f3fbb400700 1 -- 172.21.15.53:0/1965818573 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4018680 con 0x7f3fc4103520 2024-04-25T14:37:15.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.556+0000 7f3fc9800700 1 -- 172.21.15.53:0/1965818573 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fc41a2d60 con 0x7f3fc4103520 2024-04-25T14:37:15.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.556+0000 7f3fc9800700 1 -- 172.21.15.53:0/1965818573 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fc41a31c0 con 0x7f3fc4103520 2024-04-25T14:37:15.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.558+0000 7f3fc9800700 1 -- 172.21.15.53:0/1965818573 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fc4069730 con 0x7f3fc4103520 2024-04-25T14:37:15.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.560+0000 7f3fbb400700 1 -- 172.21.15.53:0/1965818573 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3fb4010430 con 0x7f3fc4103520 2024-04-25T14:37:15.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.561+0000 7f3fbb400700 1 --2- 172.21.15.53:0/1965818573 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f3fa80a3cc0 0x7f3fa80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:15.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.562+0000 7f3fbb400700 1 -- 172.21.15.53:0/1965818573 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f3fb40d00f0 con 0x7f3fc4103520 2024-04-25T14:37:15.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.562+0000 7f3fc2000700 1 --2- 172.21.15.53:0/1965818573 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f3fa80a3cc0 0x7f3fa80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:15.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.563+0000 7f3fc2000700 1 --2- 172.21.15.53:0/1965818573 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f3fa80a3cc0 0x7f3fa80a6180 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3fb000b2e0 tx=0x7f3fb0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:37:15.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.567+0000 7f3fbb400700 1 -- 172.21.15.53:0/1965818573 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3fb409ced0 con 0x7f3fc4103520 2024-04-25T14:37:15.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.801+0000 7f3fc9800700 1 -- 172.21.15.53:0/1965818573 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f3fc4063720 con 0x7f3fc4103520 2024-04-25T14:37:15.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.802+0000 7f3fbb400700 1 -- 172.21.15.53:0/1965818573 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v84) ==== 74+0+133 (secure 0 0 0) 0x7f3fb409c7d0 con 0x7f3fc4103520 2024-04-25T14:37:15.803 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:37:15.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.805+0000 7f3fc9800700 1 -- 172.21.15.53:0/1965818573 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f3fa80a3cc0 msgr2=0x7f3fa80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:15.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.805+0000 7f3fc9800700 1 --2- 172.21.15.53:0/1965818573 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f3fa80a3cc0 0x7f3fa80a6180 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3fb000b2e0 tx=0x7f3fb0005fb0 comp rx=0 tx=0).stop 2024-04-25T14:37:15.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.805+0000 7f3fc9800700 1 -- 172.21.15.53:0/1965818573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fc4103520 msgr2=0x7f3fc4102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:15.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.805+0000 7f3fc9800700 1 --2- 172.21.15.53:0/1965818573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fc4103520 0x7f3fc4102e80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400e9a0 tx=0x7f3fb400ecb0 comp rx=0 tx=0).stop 2024-04-25T14:37:15.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.805+0000 7f3fc9800700 1 -- 172.21.15.53:0/1965818573 shutdown_connections 2024-04-25T14:37:15.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.805+0000 7f3fc9800700 1 --2- 172.21.15.53:0/1965818573 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f3fa80a3cc0 0x7f3fa80a6180 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:15.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.805+0000 7f3fc9800700 1 --2- 172.21.15.53:0/1965818573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fc4108780 0x7f3fc4101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:15.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.805+0000 7f3fc9800700 1 --2- 172.21.15.53:0/1965818573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fc4103520 0x7f3fc4102e80 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:15.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.805+0000 7f3fc9800700 1 --2- 172.21.15.53:0/1965818573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fc4105e50 0x7f3fc41014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:15.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.805+0000 7f3fc9800700 1 -- 172.21.15.53:0/1965818573 >> 172.21.15.53:0/1965818573 conn(0x7f3fc40fcf30 msgr2=0x7f3fc4103e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:37:15.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.805+0000 7f3fc9800700 1 -- 172.21.15.53:0/1965818573 shutdown_connections 2024-04-25T14:37:15.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:15.805+0000 7f3fc9800700 1 -- 172.21.15.53:0/1965818573 wait complete. 2024-04-25T14:37:16.058 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:15 smithi053 ceph-mon[27773]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:16.058 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:15 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1965818573' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T14:37:16.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:15 smithi150 ceph-mon[32504]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:16.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:15 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/1965818573' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T14:37:16.454 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":84,"num_osds":12,"num_up_osds":12,"osd_up_since":1714055830,"num_in_osds":12,"osd_in_since":1714055809,"num_remapped_pgs":0} 2024-04-25T14:37:16.454 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T14:37:16.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:15 smithi183 ceph-mon[32584]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:16.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:15 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/1965818573' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T14:37:16.909 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:37:18.049 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:17 smithi053 ceph-mon[27773]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:18.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:17 smithi150 ceph-mon[32504]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:18.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:17 smithi183 ceph-mon[32584]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:18.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:18.919+0000 7f8913e00700 1 -- 172.21.15.53:0/843365237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f890c109c80 msgr2=0x7f890c10dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:18.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:18.919+0000 7f8913e00700 1 --2- 172.21.15.53:0/843365237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f890c109c80 0x7f890c10dcd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f88f80099c0 tx=0x7f88f8009cd0 comp rx=0 tx=0).stop 2024-04-25T14:37:18.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:18.920+0000 7f8913e00700 1 -- 172.21.15.53:0/843365237 shutdown_connections 2024-04-25T14:37:18.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:18.920+0000 7f8913e00700 1 --2- 172.21.15.53:0/843365237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f890c10e3b0 0x7f890c115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:18.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:18.920+0000 7f8913e00700 1 --2- 172.21.15.53:0/843365237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f890c1092d0 0x7f890c1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:18.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:18.920+0000 7f8913e00700 1 --2- 172.21.15.53:0/843365237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f890c109c80 0x7f890c10dcd0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:18.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:18.920+0000 7f8913e00700 1 -- 172.21.15.53:0/843365237 >> 172.21.15.53:0/843365237 conn(0x7f890c076210 msgr2=0x7f890c107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:37:18.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:18.920+0000 7f8913e00700 1 -- 172.21.15.53:0/843365237 shutdown_connections 2024-04-25T14:37:18.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:18.920+0000 7f8913e00700 1 -- 172.21.15.53:0/843365237 wait complete. 2024-04-25T14:37:18.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:18.921+0000 7f8913e00700 1 Processor -- start 2024-04-25T14:37:18.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:18.921+0000 7f8913e00700 1 -- start start 2024-04-25T14:37:18.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:18.921+0000 7f8913e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f890c1092d0 0x7f890c19f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:18.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:18.921+0000 7f8913e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f890c109c80 0x7f890c19fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:18.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:18.921+0000 7f8913e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f890c10e3b0 0x7f890c1a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:18.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:18.921+0000 7f8913e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f890c119220 con 0x7f890c109c80 2024-04-25T14:37:18.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:18.921+0000 7f8913e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f890c1190a0 con 0x7f890c10e3b0 2024-04-25T14:37:18.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:18.921+0000 7f8913e00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f890c1193a0 con 0x7f890c1092d0 2024-04-25T14:37:18.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:18.922+0000 7f8910a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f890c109c80 0x7f890c19fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:18.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:18.922+0000 7f8911400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f890c1092d0 0x7f890c19f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:18.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:18.922+0000 7f8910a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f890c109c80 0x7f890c19fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56964/0 (socket says 172.21.15.53:56964) 2024-04-25T14:37:18.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:18.922+0000 7f8910a00700 1 -- 172.21.15.53:0/2357601233 learned_addr learned my addr 172.21.15.53:0/2357601233 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:37:18.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:18.922+0000 7f8911e00700 1 --2- 172.21.15.53:0/2357601233 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f890c10e3b0 0x7f890c1a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:18.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:18.922+0000 7f8911400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f890c1092d0 0x7f890c19f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:45878/0 (socket says 172.21.15.53:45878) 2024-04-25T14:37:18.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:18.922+0000 7f8910a00700 1 -- 172.21.15.53:0/2357601233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f890c1092d0 msgr2=0x7f890c19f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:18.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:18.922+0000 7f8910a00700 1 --2- 172.21.15.53:0/2357601233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f890c1092d0 0x7f890c19f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:18.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:18.922+0000 7f8910a00700 1 -- 172.21.15.53:0/2357601233 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f890c10e3b0 msgr2=0x7f890c1a4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:18.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:18.922+0000 7f8910a00700 1 --2- 172.21.15.53:0/2357601233 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f890c10e3b0 0x7f890c1a4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:18.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:18.922+0000 7f8910a00700 1 -- 172.21.15.53:0/2357601233 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f890c1a4750 con 0x7f890c109c80 2024-04-25T14:37:18.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:18.923+0000 7f8910a00700 1 --2- 172.21.15.53:0/2357601233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f890c109c80 0x7f890c19fcd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f88f8006010 tx=0x7f88f80055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:37:18.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:18.923+0000 7f890a000700 1 -- 172.21.15.53:0/2357601233 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f8026070 con 0x7f890c109c80 2024-04-25T14:37:18.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:18.923+0000 7f890a000700 1 -- 172.21.15.53:0/2357601233 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88f800f460 con 0x7f890c109c80 2024-04-25T14:37:18.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:18.923+0000 7f890a000700 1 -- 172.21.15.53:0/2357601233 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f8009e60 con 0x7f890c109c80 2024-04-25T14:37:18.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:18.923+0000 7f8913e00700 1 -- 172.21.15.53:0/2357601233 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f890c1a49e0 con 0x7f890c109c80 2024-04-25T14:37:18.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:18.923+0000 7f8913e00700 1 -- 172.21.15.53:0/2357601233 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f890c1ac2c0 con 0x7f890c109c80 2024-04-25T14:37:18.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:18.924+0000 7f8913e00700 1 -- 172.21.15.53:0/2357601233 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f890c10a7b0 con 0x7f890c109c80 2024-04-25T14:37:18.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:18.926+0000 7f890a000700 1 -- 172.21.15.53:0/2357601233 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f88f8029070 con 0x7f890c109c80 2024-04-25T14:37:18.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:18.928+0000 7f890a000700 1 --2- 172.21.15.53:0/2357601233 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f88f40a3c40 0x7f88f40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:18.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:18.928+0000 7f890a000700 1 -- 172.21.15.53:0/2357601233 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f88f80d05d0 con 0x7f890c109c80 2024-04-25T14:37:18.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:18.928+0000 7f8911400700 1 --2- 172.21.15.53:0/2357601233 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f88f40a3c40 0x7f88f40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:18.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:18.929+0000 7f8911400700 1 --2- 172.21.15.53:0/2357601233 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f88f40a3c40 0x7f88f40a6100 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8900009c60 tx=0x7f8900009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:37:18.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:18.933+0000 7f890a000700 1 -- 172.21.15.53:0/2357601233 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f88f8013070 con 0x7f890c109c80 2024-04-25T14:37:19.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:19.167+0000 7f8913e00700 1 -- 172.21.15.53:0/2357601233 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f890c02df80 con 0x7f890c109c80 2024-04-25T14:37:19.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:19.169+0000 7f890a000700 1 -- 172.21.15.53:0/2357601233 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+18408 (secure 0 0 0) 0x7f88f809d4e0 con 0x7f890c109c80 2024-04-25T14:37:19.170 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:37:19.170 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":84,"fsid":"fd30e6ce-030f-11ef-bc93-c7b262605968","created":"2024-04-25T14:28:54.330326+0000","modified":"2024-04-25T14:37:11.928831+0000","last_up_change":"2024-04-25T14:37:10.921148+0000","last_in_change":"2024-04-25T14:36:49.566459+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:33:02.338115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aecfbaf0-39da-4749-a49b-c7205dabd4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6803","nonce":464176015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6805","nonce":464176015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6809","nonce":464176015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6807","nonce":464176015}]},"public_addr":"172.21.15.53:6803/464176015","cluster_addr":"172.21.15.53:6805/464176015","heartbeat_back_addr":"172.21.15.53:6809/464176015","heartbeat_front_addr":"172.21.15.53:6807/464176015","state":["exists","up"]},{"osd":1,"uuid":"2c4a21ae-c43e-4839-8a56-c3fc63aeb7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6801","nonce":1304621982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6803","nonce":1304621982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6807","nonce":1304621982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6805","nonce":1304621982}]},"public_addr":"172.21.15.150:6801/1304621982","cluster_addr":"172.21.15.150:6803/1304621982","heartbeat_back_addr":"172.21.15.150:6807/1304621982","heartbeat_front_addr":"172.21.15.150:6805/1304621982","state":["exists","up"]},{"osd":2,"uuid":"e8effd43-adc0-472a-9eed-a9273b06dc0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6801","nonce":574509311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6803","nonce":574509311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6807","nonce":574509311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6805","nonce":574509311}]},"public_addr":"172.21.15.183:6801/574509311","cluster_addr":"172.21.15.183:6803/574509311","heartbeat_back_addr":"172.21.15.183:6807/574509311","heartbeat_front_addr":"172.21.15.183:6805/574509311","state":["exists","up"]},{"osd":3,"uuid":"e7cfced4-9b40-40de-a303-75ea8d5df4a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6811","nonce":570672219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6813","nonce":570672219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6817","nonce":570672219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6815","nonce":570672219}]},"public_addr":"172.21.15.53:6811/570672219","cluster_addr":"172.21.15.53:6813/570672219","heartbeat_back_addr":"172.21.15.53:6817/570672219","heartbeat_front_addr":"172.21.15.53:6815/570672219","state":["exists","up"]},{"osd":4,"uuid":"44a9e070-2282-43f1-9f05-b1f8ff81e540","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6809","nonce":1526850103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6811","nonce":1526850103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6815","nonce":1526850103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6813","nonce":1526850103}]},"public_addr":"172.21.15.150:6809/1526850103","cluster_addr":"172.21.15.150:6811/1526850103","heartbeat_back_addr":"172.21.15.150:6815/1526850103","heartbeat_front_addr":"172.21.15.150:6813/1526850103","state":["exists","up"]},{"osd":5,"uuid":"2c5619d2-c0f9-47d6-bf7e-2f4afd1b09f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6809","nonce":773726576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6811","nonce":773726576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6815","nonce":773726576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6813","nonce":773726576}]},"public_addr":"172.21.15.183:6809/773726576","cluster_addr":"172.21.15.183:6811/773726576","heartbeat_back_addr":"172.21.15.183:6815/773726576","heartbeat_front_addr":"172.21.15.183:6813/773726576","state":["exists","up"]},{"osd":6,"uuid":"1cf74d94-c118-4738-b6db-040af458a8eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6819","nonce":355884319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6821","nonce":355884319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6825","nonce":355884319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6823","nonce":355884319}]},"public_addr":"172.21.15.53:6819/355884319","cluster_addr":"172.21.15.53:6821/355884319","heartbeat_back_addr":"172.21.15.53:6825/355884319","heartbeat_front_addr":"172.21.15.53:6823/355884319","state":["exists","up"]},{"osd":7,"uuid":"0ce97c04-cd41-4080-bbbc-be9cf5fadc02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6817","nonce":229022562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6819","nonce":229022562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6823","nonce":229022562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6821","nonce":229022562}]},"public_addr":"172.21.15.150:6817/229022562","cluster_addr":"172.21.15.150:6819/229022562","heartbeat_back_addr":"172.21.15.150:6823/229022562","heartbeat_front_addr":"172.21.15.150:6821/229022562","state":["exists","up"]},{"osd":8,"uuid":"5d57787d-2a7c-4514-bd01-357ced413f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6817","nonce":301003844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6819","nonce":301003844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6823","nonce":301003844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6821","nonce":301003844}]},"public_addr":"172.21.15.183:6817/301003844","cluster_addr":"172.21.15.183:6819/301003844","heartbeat_back_addr":"172.21.15.183:6823/301003844","heartbeat_front_addr":"172.21.15.183:6821/301003844","state":["exists","up"]},{"osd":9,"uuid":"e11dd8eb-c55a-4d70-8c58-457540b7a50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6827","nonce":3449260978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6829","nonce":3449260978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6833","nonce":3449260978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6831","nonce":3449260978}]},"public_addr":"172.21.15.53:6827/3449260978","cluster_addr":"172.21.15.53:6829/3449260978","heartbeat_back_addr":"172.21.15.53:6833/3449260978","heartbeat_front_addr":"172.21.15.53:6831/3449260978","state":["exists","up"]},{"osd":10,"uuid":"5de4803d-ad37-4a90-add5-0048f44fc4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6825","nonce":3435992308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6827","nonce":3435992308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6831","nonce":3435992308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6829","nonce":3435992308}]},"public_addr":"172.21.15.150:6825/3435992308","cluster_addr":"172.21.15.150:6827/3435992308","heartbeat_back_addr":"172.21.15.150:6831/3435992308","heartbeat_front_addr":"172.21.15.150:6829/3435992308","state":["exists","up"]},{"osd":11,"uuid":"0e819a68-ed56-477e-b9c0-81e8cc0bce11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6825","nonce":1957842686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6827","nonce":1957842686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6831","nonce":1957842686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6829","nonce":1957842686}]},"public_addr":"172.21.15.183:6825/1957842686","cluster_addr":"172.21.15.183:6827/1957842686","heartbeat_back_addr":"172.21.15.183:6831/1957842686","heartbeat_front_addr":"172.21.15.183:6829/1957842686","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:09.840629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:34.769537+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:58.112969+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:33:29.656790+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:33:55.181287+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:34:19.772645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:34:53.833921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:35:19.076733+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:35:43.163390+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:36:16.568340+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:36:42.492768+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:37:07.944639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6800/3887954889":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/1014501022":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/2576091441":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/1233527569":"2024-04-26T14:29:38.106242+0000","172.21.15.53:6801/3887954889":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/931889810":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/177196141":"2024-04-26T14:30:14.162383+0000","172.21.15.53:6800/1722497390":"2024-04-26T14:29:38.106242+0000","172.21.15.53:6801/1722497390":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/1927549198":"2024-04-26T14:30:14.162383+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:37:19.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:19.172+0000 7f8913e00700 1 -- 172.21.15.53:0/2357601233 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f88f40a3c40 msgr2=0x7f88f40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:19.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:19.172+0000 7f8913e00700 1 --2- 172.21.15.53:0/2357601233 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f88f40a3c40 0x7f88f40a6100 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8900009c60 tx=0x7f8900009260 comp rx=0 tx=0).stop 2024-04-25T14:37:19.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:19.172+0000 7f8913e00700 1 -- 172.21.15.53:0/2357601233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f890c109c80 msgr2=0x7f890c19fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:19.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:19.172+0000 7f8913e00700 1 --2- 172.21.15.53:0/2357601233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f890c109c80 0x7f890c19fcd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f88f8006010 tx=0x7f88f80055f0 comp rx=0 tx=0).stop 2024-04-25T14:37:19.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:19.172+0000 7f8913e00700 1 -- 172.21.15.53:0/2357601233 shutdown_connections 2024-04-25T14:37:19.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:19.172+0000 7f8913e00700 1 --2- 172.21.15.53:0/2357601233 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f88f40a3c40 0x7f88f40a6100 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:19.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:19.172+0000 7f8913e00700 1 --2- 172.21.15.53:0/2357601233 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f890c10e3b0 0x7f890c1a4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:19.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:19.172+0000 7f8913e00700 1 --2- 172.21.15.53:0/2357601233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f890c1092d0 0x7f890c19f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:19.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:19.172+0000 7f8913e00700 1 --2- 172.21.15.53:0/2357601233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f890c109c80 0x7f890c19fcd0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:19.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:19.172+0000 7f8913e00700 1 -- 172.21.15.53:0/2357601233 >> 172.21.15.53:0/2357601233 conn(0x7f890c076210 msgr2=0x7f890c10f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:37:19.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:19.172+0000 7f8913e00700 1 -- 172.21.15.53:0/2357601233 shutdown_connections 2024-04-25T14:37:19.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:19.172+0000 7f8913e00700 1 -- 172.21.15.53:0/2357601233 wait complete. 2024-04-25T14:37:19.933 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:33:02.338115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:37:19.934 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-25T14:37:20.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:19 smithi053 ceph-mon[27773]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:20.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:19 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/2357601233' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:37:20.391 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:37:20.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:19 smithi183 ceph-mon[32584]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:20.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:19 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/2357601233' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:37:20.463 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:19 smithi150 ceph-mon[32504]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:20.463 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:19 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/2357601233' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:37:22.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:21 smithi053 ceph-mon[27773]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:22.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:21 smithi183 ceph-mon[32584]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:22.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:21 smithi150 ceph-mon[32504]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:22.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.489+0000 7f825c200700 1 -- 172.21.15.53:0/464490392 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82541092d0 msgr2=0x7f82541096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:22.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.489+0000 7f825c200700 1 --2- 172.21.15.53:0/464490392 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82541092d0 0x7f82541096b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8248009fc0 tx=0x7f824800b330 comp rx=0 tx=0).stop 2024-04-25T14:37:22.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.489+0000 7f825c200700 1 -- 172.21.15.53:0/464490392 shutdown_connections 2024-04-25T14:37:22.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.489+0000 7f825c200700 1 --2- 172.21.15.53:0/464490392 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f825410e3b0 0x7f8254115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:22.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.489+0000 7f825c200700 1 --2- 172.21.15.53:0/464490392 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82541092d0 0x7f82541096b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:22.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.489+0000 7f825c200700 1 --2- 172.21.15.53:0/464490392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8254109c80 0x7f825410dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:22.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.489+0000 7f825c200700 1 -- 172.21.15.53:0/464490392 >> 172.21.15.53:0/464490392 conn(0x7f8254076210 msgr2=0x7f8254107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:37:22.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.489+0000 7f825c200700 1 -- 172.21.15.53:0/464490392 shutdown_connections 2024-04-25T14:37:22.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.489+0000 7f825c200700 1 -- 172.21.15.53:0/464490392 wait complete. 2024-04-25T14:37:22.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.491+0000 7f825c200700 1 Processor -- start 2024-04-25T14:37:22.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.491+0000 7f825c200700 1 -- start start 2024-04-25T14:37:22.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.492+0000 7f825c200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82541092d0 0x7f82541a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:22.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.492+0000 7f825c200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8254109c80 0x7f82541a5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:22.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.492+0000 7f825c200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f825410e3b0 0x7f825419f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:22.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.492+0000 7f825c200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8254119290 con 0x7f8254109c80 2024-04-25T14:37:22.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.492+0000 7f825c200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8254119110 con 0x7f825410e3b0 2024-04-25T14:37:22.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.492+0000 7f825c200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8254119410 con 0x7f82541092d0 2024-04-25T14:37:22.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.492+0000 7f8259800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82541092d0 0x7f82541a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:22.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.492+0000 7f8258e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8254109c80 0x7f82541a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:22.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.492+0000 7f8259800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82541092d0 0x7f82541a53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:36940/0 (socket says 172.21.15.53:36940) 2024-04-25T14:37:22.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.492+0000 7f8259800700 1 -- 172.21.15.53:0/1701766202 learned_addr learned my addr 172.21.15.53:0/1701766202 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:37:22.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.492+0000 7f825a200700 1 --2- 172.21.15.53:0/1701766202 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f825410e3b0 0x7f825419f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:22.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.493+0000 7f8259800700 1 -- 172.21.15.53:0/1701766202 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f825410e3b0 msgr2=0x7f825419f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:22.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.493+0000 7f8259800700 1 --2- 172.21.15.53:0/1701766202 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f825410e3b0 0x7f825419f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:22.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.493+0000 7f8259800700 1 -- 172.21.15.53:0/1701766202 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8254109c80 msgr2=0x7f82541a5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:22.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.493+0000 7f8259800700 1 --2- 172.21.15.53:0/1701766202 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8254109c80 0x7f82541a5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:22.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.493+0000 7f8259800700 1 -- 172.21.15.53:0/1701766202 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f825419fd10 con 0x7f82541092d0 2024-04-25T14:37:22.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.493+0000 7f8259800700 1 --2- 172.21.15.53:0/1701766202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82541092d0 0x7f82541a53f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f824800ccb0 tx=0x7f8248004f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:37:22.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.493+0000 7f8252000700 1 -- 172.21.15.53:0/1701766202 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8248021070 con 0x7f82541092d0 2024-04-25T14:37:22.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.493+0000 7f8252000700 1 -- 172.21.15.53:0/1701766202 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8248007ac0 con 0x7f82541092d0 2024-04-25T14:37:22.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.493+0000 7f825c200700 1 -- 172.21.15.53:0/1701766202 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f825419ffa0 con 0x7f82541092d0 2024-04-25T14:37:22.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.494+0000 7f8252000700 1 -- 172.21.15.53:0/1701766202 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82480052a0 con 0x7f82541092d0 2024-04-25T14:37:22.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.494+0000 7f825c200700 1 -- 172.21.15.53:0/1701766202 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82541ac2c0 con 0x7f82541092d0 2024-04-25T14:37:22.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.495+0000 7f825c200700 1 -- 172.21.15.53:0/1701766202 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8254069730 con 0x7f82541092d0 2024-04-25T14:37:22.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.497+0000 7f8252000700 1 -- 172.21.15.53:0/1701766202 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8248024070 con 0x7f82541092d0 2024-04-25T14:37:22.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.498+0000 7f8252000700 1 --2- 172.21.15.53:0/1701766202 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f823c0a3cc0 0x7f823c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:22.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.498+0000 7f8252000700 1 -- 172.21.15.53:0/1701766202 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f824800f020 con 0x7f82541092d0 2024-04-25T14:37:22.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.499+0000 7f8258e00700 1 --2- 172.21.15.53:0/1701766202 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f823c0a3cc0 0x7f823c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:22.500 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.499+0000 7f8258e00700 1 --2- 172.21.15.53:0/1701766202 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f823c0a3cc0 0x7f823c0a6180 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8254078260 tx=0x7f8240009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:37:22.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.503+0000 7f8252000700 1 -- 172.21.15.53:0/1701766202 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f824802b020 con 0x7f82541092d0 2024-04-25T14:37:22.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.736+0000 7f825c200700 1 -- 172.21.15.53:0/1701766202 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f82541a0ae0 con 0x7f82541092d0 2024-04-25T14:37:22.738 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.737+0000 7f8252000700 1 -- 172.21.15.53:0/1701766202 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7f824809e300 con 0x7f82541092d0 2024-04-25T14:37:22.738 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2024-04-25T14:37:22.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.740+0000 7f825c200700 1 -- 172.21.15.53:0/1701766202 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f823c0a3cc0 msgr2=0x7f823c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:22.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.740+0000 7f825c200700 1 --2- 172.21.15.53:0/1701766202 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f823c0a3cc0 0x7f823c0a6180 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8254078260 tx=0x7f8240009260 comp rx=0 tx=0).stop 2024-04-25T14:37:22.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.740+0000 7f825c200700 1 -- 172.21.15.53:0/1701766202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82541092d0 msgr2=0x7f82541a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:22.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.740+0000 7f825c200700 1 --2- 172.21.15.53:0/1701766202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82541092d0 0x7f82541a53f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f824800ccb0 tx=0x7f8248004f90 comp rx=0 tx=0).stop 2024-04-25T14:37:22.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.740+0000 7f825c200700 1 -- 172.21.15.53:0/1701766202 shutdown_connections 2024-04-25T14:37:22.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.740+0000 7f825c200700 1 --2- 172.21.15.53:0/1701766202 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f823c0a3cc0 0x7f823c0a6180 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:22.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.740+0000 7f825c200700 1 --2- 172.21.15.53:0/1701766202 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f825410e3b0 0x7f825419f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:22.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.740+0000 7f825c200700 1 --2- 172.21.15.53:0/1701766202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82541092d0 0x7f82541a53f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:22.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.740+0000 7f825c200700 1 --2- 172.21.15.53:0/1701766202 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8254109c80 0x7f82541a5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:22.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.740+0000 7f825c200700 1 -- 172.21.15.53:0/1701766202 >> 172.21.15.53:0/1701766202 conn(0x7f8254076210 msgr2=0x7f825410cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:37:22.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.741+0000 7f825c200700 1 -- 172.21.15.53:0/1701766202 shutdown_connections 2024-04-25T14:37:22.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:22.741+0000 7f825c200700 1 -- 172.21.15.53:0/1701766202 wait complete. 2024-04-25T14:37:23.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:22 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1701766202' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:37:23.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:22 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/1701766202' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:37:23.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:22 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/1701766202' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:37:23.497 INFO:tasks.cephadm:Adding mds.a on smithi053 2024-04-25T14:37:23.497 INFO:tasks.cephadm:Adding mds.d on smithi053 2024-04-25T14:37:23.498 INFO:tasks.cephadm:Adding mds.g on smithi053 2024-04-25T14:37:23.498 INFO:tasks.cephadm:Adding mds.j on smithi053 2024-04-25T14:37:23.498 INFO:tasks.cephadm:Adding mds.b on smithi150 2024-04-25T14:37:23.498 INFO:tasks.cephadm:Adding mds.e on smithi150 2024-04-25T14:37:23.498 INFO:tasks.cephadm:Adding mds.h on smithi150 2024-04-25T14:37:23.498 INFO:tasks.cephadm:Adding mds.k on smithi150 2024-04-25T14:37:23.498 INFO:tasks.cephadm:Adding mds.c on smithi183 2024-04-25T14:37:23.498 INFO:tasks.cephadm:Adding mds.f on smithi183 2024-04-25T14:37:23.498 INFO:tasks.cephadm:Adding mds.i on smithi183 2024-04-25T14:37:23.498 INFO:tasks.cephadm:Adding mds.l on smithi183 2024-04-25T14:37:23.498 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi053=a;smithi053=d;smithi053=g;smithi053=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l' 2024-04-25T14:37:23.747 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:37:24.177 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:23 smithi183 ceph-mon[32584]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:24.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:23 smithi053 ceph-mon[27773]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:24.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:23 smithi150 ceph-mon[32504]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:25.329 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.328+0000 7feeef078700 1 -- 172.21.15.183:0/467894379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feee8104860 msgr2=0x7feee810b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:25.329 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.328+0000 7feeef078700 1 --2- 172.21.15.183:0/467894379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feee8104860 0x7feee810b110 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7feed0009990 tx=0x7feed0009ca0 comp rx=0 tx=0).stop 2024-04-25T14:37:25.329 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.328+0000 7feeef078700 1 -- 172.21.15.183:0/467894379 shutdown_connections 2024-04-25T14:37:25.329 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.328+0000 7feeef078700 1 --2- 172.21.15.183:0/467894379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feee8104860 0x7feee810b110 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:25.329 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.328+0000 7feeef078700 1 --2- 172.21.15.183:0/467894379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feee8100e40 0x7feee8101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:25.329 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.328+0000 7feeef078700 1 --2- 172.21.15.183:0/467894379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feee8103ec0 0x7feee8104320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:25.329 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.328+0000 7feeef078700 1 -- 172.21.15.183:0/467894379 >> 172.21.15.183:0/467894379 conn(0x7feee80fc7a0 msgr2=0x7feee80febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:37:25.329 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.328+0000 7feeef078700 1 -- 172.21.15.183:0/467894379 shutdown_connections 2024-04-25T14:37:25.329 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.328+0000 7feeef078700 1 -- 172.21.15.183:0/467894379 wait complete. 2024-04-25T14:37:25.332 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.329+0000 7feeef078700 1 Processor -- start 2024-04-25T14:37:25.332 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.329+0000 7feeef078700 1 -- start start 2024-04-25T14:37:25.332 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.329+0000 7feeef078700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feee8100e40 0x7feee8103820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:25.332 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.329+0000 7feeef078700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feee8103ec0 0x7feee8101e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:25.332 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.329+0000 7feeef078700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feee8104860 0x7feee81023b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:25.332 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.329+0000 7feeef078700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7feee810e6a0 con 0x7feee8100e40 2024-04-25T14:37:25.332 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.329+0000 7feeef078700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7feee810e520 con 0x7feee8104860 2024-04-25T14:37:25.332 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.329+0000 7feeef078700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feee810e820 con 0x7feee8103ec0 2024-04-25T14:37:25.332 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.329+0000 7feee7fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feee8103ec0 0x7feee8101e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:25.332 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.329+0000 7feee7fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feee8103ec0 0x7feee8101e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:35528/0 (socket says 172.21.15.183:35528) 2024-04-25T14:37:25.332 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.329+0000 7feee7fff700 1 -- 172.21.15.183:0/2618885046 learned_addr learned my addr 172.21.15.183:0/2618885046 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-04-25T14:37:25.332 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.329+0000 7feeed615700 1 --2- 172.21.15.183:0/2618885046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feee8104860 0x7feee81023b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:25.333 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.329+0000 7feee7fff700 1 -- 172.21.15.183:0/2618885046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feee8104860 msgr2=0x7feee81023b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:25.333 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.329+0000 7feee7fff700 1 --2- 172.21.15.183:0/2618885046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feee8104860 0x7feee81023b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:25.333 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.329+0000 7feee7fff700 1 -- 172.21.15.183:0/2618885046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feee8100e40 msgr2=0x7feee8103820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:25.333 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.329+0000 7feee7fff700 1 --2- 172.21.15.183:0/2618885046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feee8100e40 0x7feee8103820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:25.333 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.329+0000 7feee7fff700 1 -- 172.21.15.183:0/2618885046 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feee8102c60 con 0x7feee8103ec0 2024-04-25T14:37:25.333 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.330+0000 7feee7fff700 1 --2- 172.21.15.183:0/2618885046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feee8103ec0 0x7feee8101e70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7feed800c970 tx=0x7feed800cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:37:25.333 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.330+0000 7feee5ffb700 1 -- 172.21.15.183:0/2618885046 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed8007aa0 con 0x7feee8103ec0 2024-04-25T14:37:25.333 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.330+0000 7feee5ffb700 1 -- 172.21.15.183:0/2618885046 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feed8004ce0 con 0x7feee8103ec0 2024-04-25T14:37:25.333 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.330+0000 7feeef078700 1 -- 172.21.15.183:0/2618885046 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feee81a2e10 con 0x7feee8103ec0 2024-04-25T14:37:25.333 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.330+0000 7feee5ffb700 1 -- 172.21.15.183:0/2618885046 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed8005510 con 0x7feee8103ec0 2024-04-25T14:37:25.333 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.330+0000 7feeef078700 1 -- 172.21.15.183:0/2618885046 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feee81a3270 con 0x7feee8103ec0 2024-04-25T14:37:25.333 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.331+0000 7feeef078700 1 -- 172.21.15.183:0/2618885046 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feee8069730 con 0x7feee8103ec0 2024-04-25T14:37:25.334 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.332+0000 7feee5ffb700 1 -- 172.21.15.183:0/2618885046 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7feed8007c40 con 0x7feee8103ec0 2024-04-25T14:37:25.334 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.332+0000 7feee5ffb700 1 --2- 172.21.15.183:0/2618885046 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7feed40a3cc0 0x7feed40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:25.334 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.332+0000 7feee5ffb700 1 -- 172.21.15.183:0/2618885046 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7feed80cfc60 con 0x7feee8103ec0 2024-04-25T14:37:25.334 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.333+0000 7feeece14700 1 --2- 172.21.15.183:0/2618885046 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7feed40a3cc0 0x7feed40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:25.335 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.334+0000 7feeece14700 1 --2- 172.21.15.183:0/2618885046 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7feed40a3cc0 0x7feed40a6180 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7feee806aeb0 tx=0x7feee0009250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:37:25.337 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.337+0000 7feee5ffb700 1 -- 172.21.15.183:0/2618885046 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feed809cb70 con 0x7feee8103ec0 2024-04-25T14:37:25.545 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.545+0000 7feeef078700 1 -- 172.21.15.183:0/2618885046 --> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi053=a;smithi053=d;smithi053=g;smithi053=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l", "target": ["mon-mgr", ""]}) -- 0x7feee8062f80 con 0x7feed40a3cc0 2024-04-25T14:37:25.562 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.561+0000 7feee5ffb700 1 -- 172.21.15.183:0/2618885046 <== mgr.14152 v2:172.21.15.53:6800/4154933108 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7feee8062f80 con 0x7feed40a3cc0 2024-04-25T14:37:25.562 INFO:teuthology.orchestra.run.smithi183.stdout:Scheduled mds.all update... 2024-04-25T14:37:25.564 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.563+0000 7feeef078700 1 -- 172.21.15.183:0/2618885046 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7feed40a3cc0 msgr2=0x7feed40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:25.564 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.563+0000 7feeef078700 1 --2- 172.21.15.183:0/2618885046 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7feed40a3cc0 0x7feed40a6180 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7feee806aeb0 tx=0x7feee0009250 comp rx=0 tx=0).stop 2024-04-25T14:37:25.565 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.563+0000 7feeef078700 1 -- 172.21.15.183:0/2618885046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feee8103ec0 msgr2=0x7feee8101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:25.565 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.563+0000 7feeef078700 1 --2- 172.21.15.183:0/2618885046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feee8103ec0 0x7feee8101e70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7feed800c970 tx=0x7feed800cc80 comp rx=0 tx=0).stop 2024-04-25T14:37:25.565 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.563+0000 7feeef078700 1 -- 172.21.15.183:0/2618885046 shutdown_connections 2024-04-25T14:37:25.565 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.563+0000 7feeef078700 1 --2- 172.21.15.183:0/2618885046 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7feed40a3cc0 0x7feed40a6180 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:25.565 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.563+0000 7feeef078700 1 --2- 172.21.15.183:0/2618885046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feee8104860 0x7feee81023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:25.565 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.563+0000 7feeef078700 1 --2- 172.21.15.183:0/2618885046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feee8100e40 0x7feee8103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:25.565 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.563+0000 7feeef078700 1 --2- 172.21.15.183:0/2618885046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feee8103ec0 0x7feee8101e70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:25.565 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.563+0000 7feeef078700 1 -- 172.21.15.183:0/2618885046 >> 172.21.15.183:0/2618885046 conn(0x7feee80fc7a0 msgr2=0x7feee8107160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:37:25.565 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.563+0000 7feeef078700 1 -- 172.21.15.183:0/2618885046 shutdown_connections 2024-04-25T14:37:25.565 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T14:37:25.564+0000 7feeef078700 1 -- 172.21.15.183:0/2618885046 wait complete. 2024-04-25T14:37:26.166 DEBUG:teuthology.orchestra.run.smithi053:mds.a> sudo journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mds.a.service 2024-04-25T14:37:26.175 DEBUG:teuthology.orchestra.run.smithi053:mds.d> sudo journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mds.d.service 2024-04-25T14:37:26.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:25 smithi183 ceph-mon[32584]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:26.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:25 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:26.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:25 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:37:26.220 DEBUG:teuthology.orchestra.run.smithi053:mds.g> sudo journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mds.g.service 2024-04-25T14:37:26.235 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:25 smithi053 ceph-mon[27773]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:26.235 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:25 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:26.235 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:25 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:37:26.240 DEBUG:teuthology.orchestra.run.smithi053:mds.j> sudo journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mds.j.service 2024-04-25T14:37:26.286 DEBUG:teuthology.orchestra.run.smithi150:mds.b> sudo journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mds.b.service 2024-04-25T14:37:26.289 DEBUG:teuthology.orchestra.run.smithi150:mds.e> sudo journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mds.e.service 2024-04-25T14:37:26.308 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:25 smithi150 ceph-mon[32504]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:26.308 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:25 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:26.308 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:25 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:37:26.309 DEBUG:teuthology.orchestra.run.smithi150:mds.h> sudo journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mds.h.service 2024-04-25T14:37:26.352 DEBUG:teuthology.orchestra.run.smithi150:mds.k> sudo journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mds.k.service 2024-04-25T14:37:26.395 DEBUG:teuthology.orchestra.run.smithi183:mds.c> sudo journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mds.c.service 2024-04-25T14:37:26.398 DEBUG:teuthology.orchestra.run.smithi183:mds.f> sudo journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mds.f.service 2024-04-25T14:37:26.441 DEBUG:teuthology.orchestra.run.smithi183:mds.i> sudo journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mds.i.service 2024-04-25T14:37:26.484 DEBUG:teuthology.orchestra.run.smithi183:mds.l> sudo journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mds.l.service 2024-04-25T14:37:26.527 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-25T14:37:26.528 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:37:26.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.741+0000 7fc33ec00640 1 -- 172.21.15.53:0/2762413986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc338107ab0 msgr2=0x7fc338109f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:26.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.741+0000 7fc33ec00640 1 --2- 172.21.15.53:0/2762413986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc338107ab0 0x7fc338109f00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc330009900 tx=0x7fc33002a3f0 comp rx=0 tx=0).stop 2024-04-25T14:37:26.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.741+0000 7fc33ec00640 1 -- 172.21.15.53:0/2762413986 shutdown_connections 2024-04-25T14:37:26.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.741+0000 7fc33ec00640 1 --2- 172.21.15.53:0/2762413986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc338107ab0 0x7fc338109f00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:26.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.741+0000 7fc33ec00640 1 --2- 172.21.15.53:0/2762413986 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc338105180 0x7fc338107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:26.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.741+0000 7fc33ec00640 1 --2- 172.21.15.53:0/2762413986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc338102850 0x7fc338104c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:26.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.741+0000 7fc33ec00640 1 -- 172.21.15.53:0/2762413986 >> 172.21.15.53:0/2762413986 conn(0x7fc3380fc220 msgr2=0x7fc3380fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:37:26.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.741+0000 7fc33ec00640 1 -- 172.21.15.53:0/2762413986 shutdown_connections 2024-04-25T14:37:26.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.741+0000 7fc33ec00640 1 -- 172.21.15.53:0/2762413986 wait complete. 2024-04-25T14:37:26.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.743+0000 7fc33ec00640 1 Processor -- start 2024-04-25T14:37:26.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.743+0000 7fc33ec00640 1 -- start start 2024-04-25T14:37:26.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.743+0000 7fc33ec00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc338102850 0x7fc338102120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:26.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.743+0000 7fc33ec00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc338105180 0x7fc338100800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:26.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.743+0000 7fc33ec00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc338107ab0 0x7fc338100d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:26.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.743+0000 7fc33ec00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc33810d950 con 0x7fc338102850 2024-04-25T14:37:26.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.743+0000 7fc33ec00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc33810d7d0 con 0x7fc338107ab0 2024-04-25T14:37:26.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.743+0000 7fc33ec00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc33810dad0 con 0x7fc338105180 2024-04-25T14:37:26.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.743+0000 7fc33d800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc338102850 0x7fc338102120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:26.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.743+0000 7fc33ce00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc338105180 0x7fc338100800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:26.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.743+0000 7fc33ce00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc338105180 0x7fc338100800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:36966/0 (socket says 172.21.15.53:36966) 2024-04-25T14:37:26.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.743+0000 7fc33ce00640 1 -- 172.21.15.53:0/1322715635 learned_addr learned my addr 172.21.15.53:0/1322715635 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:37:26.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.744+0000 7fc33e200640 1 --2- 172.21.15.53:0/1322715635 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc338107ab0 0x7fc338100d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:26.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.744+0000 7fc33ce00640 1 -- 172.21.15.53:0/1322715635 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc338107ab0 msgr2=0x7fc338100d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:26.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.744+0000 7fc33ce00640 1 --2- 172.21.15.53:0/1322715635 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc338107ab0 0x7fc338100d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:26.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.744+0000 7fc33ce00640 1 -- 172.21.15.53:0/1322715635 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc338102850 msgr2=0x7fc338102120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:26.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.744+0000 7fc33ce00640 1 --2- 172.21.15.53:0/1322715635 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc338102850 0x7fc338102120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:26.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.744+0000 7fc33ce00640 1 -- 172.21.15.53:0/1322715635 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc338101540 con 0x7fc338105180 2024-04-25T14:37:26.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.744+0000 7fc33ce00640 1 --2- 172.21.15.53:0/1322715635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc338105180 0x7fc338100800 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc32400e9c0 tx=0x7fc32400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:37:26.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.744+0000 7fc336000640 1 -- 172.21.15.53:0/1322715635 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32400cd80 con 0x7fc338105180 2024-04-25T14:37:26.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.745+0000 7fc336000640 1 -- 172.21.15.53:0/1322715635 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3240054d0 con 0x7fc338105180 2024-04-25T14:37:26.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.745+0000 7fc33ec00640 1 -- 172.21.15.53:0/1322715635 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3381a2120 con 0x7fc338105180 2024-04-25T14:37:26.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.745+0000 7fc336000640 1 -- 172.21.15.53:0/1322715635 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc324010430 con 0x7fc338105180 2024-04-25T14:37:26.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.745+0000 7fc33ec00640 1 -- 172.21.15.53:0/1322715635 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3381a2690 con 0x7fc338105180 2024-04-25T14:37:26.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.746+0000 7fc33ec00640 1 -- 172.21.15.53:0/1322715635 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3380fdd10 con 0x7fc338105180 2024-04-25T14:37:26.749 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.749+0000 7fc336000640 1 -- 172.21.15.53:0/1322715635 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc324004fd0 con 0x7fc338105180 2024-04-25T14:37:26.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.750+0000 7fc336000640 1 --2- 172.21.15.53:0/1322715635 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fc3200a3c30 0x7fc3200a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:26.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.750+0000 7fc336000640 1 -- 172.21.15.53:0/1322715635 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fc3240d0340 con 0x7fc338105180 2024-04-25T14:37:26.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.750+0000 7fc33d800640 1 --2- 172.21.15.53:0/1322715635 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fc3200a3c30 0x7fc3200a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:26.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.751+0000 7fc33d800640 1 --2- 172.21.15.53:0/1322715635 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fc3200a3c30 0x7fc3200a60f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc32c005fd0 tx=0x7fc32c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:37:26.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.754+0000 7fc336000640 1 -- 172.21.15.53:0/1322715635 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc32409d120 con 0x7fc338105180 2024-04-25T14:37:26.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.970+0000 7fc33ec00640 1 -- 172.21.15.53:0/1322715635 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc338062b50 con 0x7fc338105180 2024-04-25T14:37:26.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.971+0000 7fc336000640 1 -- 172.21.15.53:0/1322715635 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+18408 (secure 0 0 0) 0x7fc32409ca20 con 0x7fc338105180 2024-04-25T14:37:26.973 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:37:26.973 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":84,"fsid":"fd30e6ce-030f-11ef-bc93-c7b262605968","created":"2024-04-25T14:28:54.330326+0000","modified":"2024-04-25T14:37:11.928831+0000","last_up_change":"2024-04-25T14:37:10.921148+0000","last_in_change":"2024-04-25T14:36:49.566459+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:33:02.338115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aecfbaf0-39da-4749-a49b-c7205dabd4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6803","nonce":464176015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6805","nonce":464176015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6809","nonce":464176015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6807","nonce":464176015}]},"public_addr":"172.21.15.53:6803/464176015","cluster_addr":"172.21.15.53:6805/464176015","heartbeat_back_addr":"172.21.15.53:6809/464176015","heartbeat_front_addr":"172.21.15.53:6807/464176015","state":["exists","up"]},{"osd":1,"uuid":"2c4a21ae-c43e-4839-8a56-c3fc63aeb7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6801","nonce":1304621982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6803","nonce":1304621982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6807","nonce":1304621982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6805","nonce":1304621982}]},"public_addr":"172.21.15.150:6801/1304621982","cluster_addr":"172.21.15.150:6803/1304621982","heartbeat_back_addr":"172.21.15.150:6807/1304621982","heartbeat_front_addr":"172.21.15.150:6805/1304621982","state":["exists","up"]},{"osd":2,"uuid":"e8effd43-adc0-472a-9eed-a9273b06dc0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6801","nonce":574509311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6803","nonce":574509311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6807","nonce":574509311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6805","nonce":574509311}]},"public_addr":"172.21.15.183:6801/574509311","cluster_addr":"172.21.15.183:6803/574509311","heartbeat_back_addr":"172.21.15.183:6807/574509311","heartbeat_front_addr":"172.21.15.183:6805/574509311","state":["exists","up"]},{"osd":3,"uuid":"e7cfced4-9b40-40de-a303-75ea8d5df4a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6811","nonce":570672219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6813","nonce":570672219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6817","nonce":570672219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6815","nonce":570672219}]},"public_addr":"172.21.15.53:6811/570672219","cluster_addr":"172.21.15.53:6813/570672219","heartbeat_back_addr":"172.21.15.53:6817/570672219","heartbeat_front_addr":"172.21.15.53:6815/570672219","state":["exists","up"]},{"osd":4,"uuid":"44a9e070-2282-43f1-9f05-b1f8ff81e540","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6809","nonce":1526850103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6811","nonce":1526850103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6815","nonce":1526850103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6813","nonce":1526850103}]},"public_addr":"172.21.15.150:6809/1526850103","cluster_addr":"172.21.15.150:6811/1526850103","heartbeat_back_addr":"172.21.15.150:6815/1526850103","heartbeat_front_addr":"172.21.15.150:6813/1526850103","state":["exists","up"]},{"osd":5,"uuid":"2c5619d2-c0f9-47d6-bf7e-2f4afd1b09f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6809","nonce":773726576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6811","nonce":773726576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6815","nonce":773726576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6813","nonce":773726576}]},"public_addr":"172.21.15.183:6809/773726576","cluster_addr":"172.21.15.183:6811/773726576","heartbeat_back_addr":"172.21.15.183:6815/773726576","heartbeat_front_addr":"172.21.15.183:6813/773726576","state":["exists","up"]},{"osd":6,"uuid":"1cf74d94-c118-4738-b6db-040af458a8eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6819","nonce":355884319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6821","nonce":355884319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6825","nonce":355884319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6823","nonce":355884319}]},"public_addr":"172.21.15.53:6819/355884319","cluster_addr":"172.21.15.53:6821/355884319","heartbeat_back_addr":"172.21.15.53:6825/355884319","heartbeat_front_addr":"172.21.15.53:6823/355884319","state":["exists","up"]},{"osd":7,"uuid":"0ce97c04-cd41-4080-bbbc-be9cf5fadc02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6817","nonce":229022562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6819","nonce":229022562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6823","nonce":229022562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6821","nonce":229022562}]},"public_addr":"172.21.15.150:6817/229022562","cluster_addr":"172.21.15.150:6819/229022562","heartbeat_back_addr":"172.21.15.150:6823/229022562","heartbeat_front_addr":"172.21.15.150:6821/229022562","state":["exists","up"]},{"osd":8,"uuid":"5d57787d-2a7c-4514-bd01-357ced413f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6817","nonce":301003844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6819","nonce":301003844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6823","nonce":301003844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6821","nonce":301003844}]},"public_addr":"172.21.15.183:6817/301003844","cluster_addr":"172.21.15.183:6819/301003844","heartbeat_back_addr":"172.21.15.183:6823/301003844","heartbeat_front_addr":"172.21.15.183:6821/301003844","state":["exists","up"]},{"osd":9,"uuid":"e11dd8eb-c55a-4d70-8c58-457540b7a50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6827","nonce":3449260978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6829","nonce":3449260978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6833","nonce":3449260978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6831","nonce":3449260978}]},"public_addr":"172.21.15.53:6827/3449260978","cluster_addr":"172.21.15.53:6829/3449260978","heartbeat_back_addr":"172.21.15.53:6833/3449260978","heartbeat_front_addr":"172.21.15.53:6831/3449260978","state":["exists","up"]},{"osd":10,"uuid":"5de4803d-ad37-4a90-add5-0048f44fc4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6825","nonce":3435992308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6827","nonce":3435992308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6831","nonce":3435992308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6829","nonce":3435992308}]},"public_addr":"172.21.15.150:6825/3435992308","cluster_addr":"172.21.15.150:6827/3435992308","heartbeat_back_addr":"172.21.15.150:6831/3435992308","heartbeat_front_addr":"172.21.15.150:6829/3435992308","state":["exists","up"]},{"osd":11,"uuid":"0e819a68-ed56-477e-b9c0-81e8cc0bce11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6825","nonce":1957842686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6827","nonce":1957842686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6831","nonce":1957842686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6829","nonce":1957842686}]},"public_addr":"172.21.15.183:6825/1957842686","cluster_addr":"172.21.15.183:6827/1957842686","heartbeat_back_addr":"172.21.15.183:6831/1957842686","heartbeat_front_addr":"172.21.15.183:6829/1957842686","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:09.840629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:34.769537+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:58.112969+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:33:29.656790+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:33:55.181287+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:34:19.772645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:34:53.833921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:35:19.076733+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:35:43.163390+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:36:16.568340+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:36:42.492768+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:37:07.944639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6800/3887954889":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/1014501022":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/2576091441":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/1233527569":"2024-04-26T14:29:38.106242+0000","172.21.15.53:6801/3887954889":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/931889810":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/177196141":"2024-04-26T14:30:14.162383+0000","172.21.15.53:6800/1722497390":"2024-04-26T14:29:38.106242+0000","172.21.15.53:6801/1722497390":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/1927549198":"2024-04-26T14:30:14.162383+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:37:26.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.974+0000 7fc33ec00640 1 -- 172.21.15.53:0/1322715635 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fc3200a3c30 msgr2=0x7fc3200a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:26.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.974+0000 7fc33ec00640 1 --2- 172.21.15.53:0/1322715635 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fc3200a3c30 0x7fc3200a60f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc32c005fd0 tx=0x7fc32c009500 comp rx=0 tx=0).stop 2024-04-25T14:37:26.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.974+0000 7fc33ec00640 1 -- 172.21.15.53:0/1322715635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc338105180 msgr2=0x7fc338100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:26.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.974+0000 7fc33ec00640 1 --2- 172.21.15.53:0/1322715635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc338105180 0x7fc338100800 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc32400e9c0 tx=0x7fc32400ee80 comp rx=0 tx=0).stop 2024-04-25T14:37:26.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.974+0000 7fc33ec00640 1 -- 172.21.15.53:0/1322715635 shutdown_connections 2024-04-25T14:37:26.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.974+0000 7fc33ec00640 1 --2- 172.21.15.53:0/1322715635 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fc3200a3c30 0x7fc3200a60f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:26.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.974+0000 7fc33ec00640 1 --2- 172.21.15.53:0/1322715635 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc338107ab0 0x7fc338100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:26.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.974+0000 7fc33ec00640 1 --2- 172.21.15.53:0/1322715635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc338105180 0x7fc338100800 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:26.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.974+0000 7fc33ec00640 1 --2- 172.21.15.53:0/1322715635 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc338102850 0x7fc338102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:26.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.974+0000 7fc33ec00640 1 -- 172.21.15.53:0/1322715635 >> 172.21.15.53:0/1322715635 conn(0x7fc3380fc220 msgr2=0x7fc3381031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:37:26.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.974+0000 7fc33ec00640 1 -- 172.21.15.53:0/1322715635 shutdown_connections 2024-04-25T14:37:26.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:26.974+0000 7fc33ec00640 1 -- 172.21.15.53:0/1322715635 wait complete. 2024-04-25T14:37:26.996 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:33:02.338115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:37:26.997 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:37:27.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.195+0000 7f1cbc000640 1 -- 172.21.15.53:0/3833990417 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cb41053a0 msgr2=0x7f1cb4105820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:27.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.195+0000 7f1cbc000640 1 --2- 172.21.15.53:0/3833990417 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cb41053a0 0x7f1cb4105820 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0009900 tx=0x7f1ca002a3f0 comp rx=0 tx=0).stop 2024-04-25T14:37:27.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.196+0000 7f1cbc000640 1 -- 172.21.15.53:0/3833990417 shutdown_connections 2024-04-25T14:37:27.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.196+0000 7f1cbc000640 1 --2- 172.21.15.53:0/3833990417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cb4105d60 0x7f1cb410c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:27.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.196+0000 7f1cbc000640 1 --2- 172.21.15.53:0/3833990417 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cb41053a0 0x7f1cb4105820 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:27.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.196+0000 7f1cbc000640 1 --2- 172.21.15.53:0/3833990417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cb41041b0 0x7f1cb41045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:27.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.196+0000 7f1cbc000640 1 -- 172.21.15.53:0/3833990417 >> 172.21.15.53:0/3833990417 conn(0x7f1cb40ff740 msgr2=0x7f1cb4101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:37:27.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.196+0000 7f1cbc000640 1 -- 172.21.15.53:0/3833990417 shutdown_connections 2024-04-25T14:37:27.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.196+0000 7f1cbc000640 1 -- 172.21.15.53:0/3833990417 wait complete. 2024-04-25T14:37:27.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.197+0000 7f1cbc000640 1 Processor -- start 2024-04-25T14:37:27.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.197+0000 7f1cbc000640 1 -- start start 2024-04-25T14:37:27.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.197+0000 7f1cbc000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cb41041b0 0x7f1cb419a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:27.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.197+0000 7f1cbc000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cb41053a0 0x7f1cb419a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:27.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.198+0000 7f1cbc000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cb4105d60 0x7f1cb41a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:27.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.198+0000 7f1cbc000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb410fd40 con 0x7f1cb4105d60 2024-04-25T14:37:27.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.198+0000 7f1cbc000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb410fbc0 con 0x7f1cb41041b0 2024-04-25T14:37:27.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.198+0000 7f1cbc000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb410fec0 con 0x7f1cb41053a0 2024-04-25T14:37:27.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.198+0000 7f1cb8c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cb41053a0 0x7f1cb419a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:27.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.198+0000 7f1cb8c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cb41053a0 0x7f1cb419a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:36992/0 (socket says 172.21.15.53:36992) 2024-04-25T14:37:27.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.198+0000 7f1cb8c00640 1 -- 172.21.15.53:0/2663458791 learned_addr learned my addr 172.21.15.53:0/2663458791 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:37:27.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.198+0000 7f1cba000640 1 --2- 172.21.15.53:0/2663458791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cb4105d60 0x7f1cb41a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:27.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.198+0000 7f1cb9600640 1 --2- 172.21.15.53:0/2663458791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cb41041b0 0x7f1cb419a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:27.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.198+0000 7f1cb8c00640 1 -- 172.21.15.53:0/2663458791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cb41041b0 msgr2=0x7f1cb419a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:27.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.198+0000 7f1cb8c00640 1 --2- 172.21.15.53:0/2663458791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cb41041b0 0x7f1cb419a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:27.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.198+0000 7f1cb8c00640 1 -- 172.21.15.53:0/2663458791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cb4105d60 msgr2=0x7f1cb41a1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:27.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.198+0000 7f1cb8c00640 1 --2- 172.21.15.53:0/2663458791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cb4105d60 0x7f1cb41a1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:27.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.198+0000 7f1cb8c00640 1 -- 172.21.15.53:0/2663458791 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb41a1fe0 con 0x7f1cb41053a0 2024-04-25T14:37:27.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.199+0000 7f1cb8c00640 1 --2- 172.21.15.53:0/2663458791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cb41053a0 0x7f1cb419a900 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0005ab0 tx=0x7f1ca0003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:37:27.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.199+0000 7f1cb2000640 1 -- 172.21.15.53:0/2663458791 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca003f070 con 0x7f1cb41053a0 2024-04-25T14:37:27.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.199+0000 7f1cbc000640 1 -- 172.21.15.53:0/2663458791 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cb41a2270 con 0x7f1cb41053a0 2024-04-25T14:37:27.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.199+0000 7f1cb2000640 1 -- 172.21.15.53:0/2663458791 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ca0031c50 con 0x7f1cb41053a0 2024-04-25T14:37:27.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.199+0000 7f1cb2000640 1 -- 172.21.15.53:0/2663458791 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0035030 con 0x7f1cb41053a0 2024-04-25T14:37:27.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.199+0000 7f1cbc000640 1 -- 172.21.15.53:0/2663458791 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cb41a27b0 con 0x7f1cb41053a0 2024-04-25T14:37:27.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.200+0000 7f1cbc000640 1 -- 172.21.15.53:0/2663458791 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cb410b5a0 con 0x7f1cb41053a0 2024-04-25T14:37:27.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.202+0000 7f1cb2000640 1 -- 172.21.15.53:0/2663458791 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1ca0042070 con 0x7f1cb41053a0 2024-04-25T14:37:27.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.203+0000 7f1cb2000640 1 --2- 172.21.15.53:0/2663458791 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f1c9c0a3c30 0x7f1c9c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:27.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.204+0000 7f1cb2000640 1 -- 172.21.15.53:0/2663458791 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f1ca00f0720 con 0x7f1cb41053a0 2024-04-25T14:37:27.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.204+0000 7f1cb9600640 1 --2- 172.21.15.53:0/2663458791 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f1c9c0a3c30 0x7f1c9c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:27.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.205+0000 7f1cb9600640 1 --2- 172.21.15.53:0/2663458791 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f1c9c0a3c30 0x7f1c9c0a60f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4105200 tx=0x7f1ca8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:37:27.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.212+0000 7f1cb2000640 1 -- 172.21.15.53:0/2663458791 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1ca00bceb0 con 0x7f1cb41053a0 2024-04-25T14:37:27.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:26 smithi053 ceph-mon[27773]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi053=a;smithi053=d;smithi053=g;smithi053=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:37:27.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:26 smithi053 ceph-mon[27773]: Saving service mds.all spec with placement smithi053=a;smithi053=d;smithi053=g;smithi053=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l;count:12 2024-04-25T14:37:27.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:26 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:37:27.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:26 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:37:27.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:26 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:27.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:26 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:37:27.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:26 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:37:27.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:26 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:37:27.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:26 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1322715635' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:37:27.383 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:26 smithi183 ceph-mon[32584]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi053=a;smithi053=d;smithi053=g;smithi053=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:37:27.384 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:26 smithi183 ceph-mon[32584]: Saving service mds.all spec with placement smithi053=a;smithi053=d;smithi053=g;smithi053=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l;count:12 2024-04-25T14:37:27.384 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:26 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:37:27.384 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:26 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:37:27.384 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:26 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:27.384 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:26 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:37:27.384 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:26 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:37:27.384 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:26 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:37:27.384 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:26 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/1322715635' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:37:27.421 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.420+0000 7f1cbc000640 1 -- 172.21.15.53:0/2663458791 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1cb4062b50 con 0x7f1cb41053a0 2024-04-25T14:37:27.422 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.421+0000 7f1cb2000640 1 -- 172.21.15.53:0/2663458791 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7f1ca00c11b0 con 0x7f1cb41053a0 2024-04-25T14:37:27.422 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2024-04-25T14:37:27.424 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.423+0000 7f1cbc000640 1 -- 172.21.15.53:0/2663458791 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f1c9c0a3c30 msgr2=0x7f1c9c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:27.424 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.423+0000 7f1cbc000640 1 --2- 172.21.15.53:0/2663458791 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f1c9c0a3c30 0x7f1c9c0a60f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4105200 tx=0x7f1ca8005f90 comp rx=0 tx=0).stop 2024-04-25T14:37:27.424 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.423+0000 7f1cbc000640 1 -- 172.21.15.53:0/2663458791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cb41053a0 msgr2=0x7f1cb419a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:27.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.423+0000 7f1cbc000640 1 --2- 172.21.15.53:0/2663458791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cb41053a0 0x7f1cb419a900 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0005ab0 tx=0x7f1ca0003b90 comp rx=0 tx=0).stop 2024-04-25T14:37:27.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.423+0000 7f1cbc000640 1 -- 172.21.15.53:0/2663458791 shutdown_connections 2024-04-25T14:37:27.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.424+0000 7f1cbc000640 1 --2- 172.21.15.53:0/2663458791 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f1c9c0a3c30 0x7f1c9c0a60f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:27.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.424+0000 7f1cbc000640 1 --2- 172.21.15.53:0/2663458791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cb4105d60 0x7f1cb41a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:27.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.424+0000 7f1cbc000640 1 --2- 172.21.15.53:0/2663458791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cb41053a0 0x7f1cb419a900 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:27.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.424+0000 7f1cbc000640 1 --2- 172.21.15.53:0/2663458791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cb41041b0 0x7f1cb419a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:27.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.424+0000 7f1cbc000640 1 -- 172.21.15.53:0/2663458791 >> 172.21.15.53:0/2663458791 conn(0x7f1cb40ff740 msgr2=0x7f1cb4108770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:37:27.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.424+0000 7f1cbc000640 1 -- 172.21.15.53:0/2663458791 shutdown_connections 2024-04-25T14:37:27.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.424+0000 7f1cbc000640 1 -- 172.21.15.53:0/2663458791 wait complete. 2024-04-25T14:37:27.445 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:37:27.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:26 smithi150 ceph-mon[32504]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi053=a;smithi053=d;smithi053=g;smithi053=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:37:27.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:26 smithi150 ceph-mon[32504]: Saving service mds.all spec with placement smithi053=a;smithi053=d;smithi053=g;smithi053=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l;count:12 2024-04-25T14:37:27.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:26 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:37:27.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:26 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:37:27.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:26 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:27.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:26 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:37:27.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:26 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:37:27.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:26 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:37:27.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:26 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/1322715635' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:37:27.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.654+0000 7f9851600640 1 -- 172.21.15.53:0/2301971041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f984c1097a0 msgr2=0x7f984c109c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:27.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.654+0000 7f9851600640 1 --2- 172.21.15.53:0/2301971041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f984c1097a0 0x7f984c109c20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9838009860 tx=0x7f983802a340 comp rx=0 tx=0).stop 2024-04-25T14:37:27.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.654+0000 7f9851600640 1 -- 172.21.15.53:0/2301971041 shutdown_connections 2024-04-25T14:37:27.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.654+0000 7f9851600640 1 --2- 172.21.15.53:0/2301971041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f984c10a160 0x7f984c1109f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:27.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.654+0000 7f9851600640 1 --2- 172.21.15.53:0/2301971041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f984c1097a0 0x7f984c109c20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:27.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.654+0000 7f9851600640 1 --2- 172.21.15.53:0/2301971041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f984c1085b0 0x7f984c1089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:27.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.654+0000 7f9851600640 1 -- 172.21.15.53:0/2301971041 >> 172.21.15.53:0/2301971041 conn(0x7f984c077570 msgr2=0x7f984c079990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:37:27.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.654+0000 7f9851600640 1 -- 172.21.15.53:0/2301971041 shutdown_connections 2024-04-25T14:37:27.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.654+0000 7f9851600640 1 -- 172.21.15.53:0/2301971041 wait complete. 2024-04-25T14:37:27.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.656+0000 7f9851600640 1 Processor -- start 2024-04-25T14:37:27.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.656+0000 7f9851600640 1 -- start start 2024-04-25T14:37:27.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.656+0000 7f9851600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f984c1085b0 0x7f984c19e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:27.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.656+0000 7f9851600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f984c1097a0 0x7f984c19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:27.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.656+0000 7f9851600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f984c10a160 0x7f984c1a5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:27.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.656+0000 7f9851600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f984c1140c0 con 0x7f984c10a160 2024-04-25T14:37:27.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.656+0000 7f9851600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f984c113f40 con 0x7f984c1085b0 2024-04-25T14:37:27.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.656+0000 7f9851600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f984c114240 con 0x7f984c1097a0 2024-04-25T14:37:27.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.657+0000 7f984b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f984c10a160 0x7f984c1a5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:27.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.657+0000 7f984a000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f984c1097a0 0x7f984c19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:27.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.657+0000 7f984b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f984c10a160 0x7f984c1a5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34154/0 (socket says 172.21.15.53:34154) 2024-04-25T14:37:27.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.657+0000 7f984aa00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f984c1085b0 0x7f984c19e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:27.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.657+0000 7f984b400640 1 -- 172.21.15.53:0/3831886535 learned_addr learned my addr 172.21.15.53:0/3831886535 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:37:27.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.657+0000 7f984a000640 1 -- 172.21.15.53:0/3831886535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f984c1085b0 msgr2=0x7f984c19e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:27.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.657+0000 7f984a000640 1 --2- 172.21.15.53:0/3831886535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f984c1085b0 0x7f984c19e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:27.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.657+0000 7f984a000640 1 -- 172.21.15.53:0/3831886535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f984c10a160 msgr2=0x7f984c1a5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:27.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.657+0000 7f984a000640 1 --2- 172.21.15.53:0/3831886535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f984c10a160 0x7f984c1a5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:27.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.657+0000 7f984a000640 1 -- 172.21.15.53:0/3831886535 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f984c1a6410 con 0x7f984c1097a0 2024-04-25T14:37:27.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.658+0000 7f984a000640 1 --2- 172.21.15.53:0/3831886535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f984c1097a0 0x7f984c19ed30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f983802a860 tx=0x7f9838032dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:37:27.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.658+0000 7f9843400640 1 -- 172.21.15.53:0/3831886535 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838037040 con 0x7f984c1097a0 2024-04-25T14:37:27.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.658+0000 7f9851600640 1 -- 172.21.15.53:0/3831886535 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f984c1a66a0 con 0x7f984c1097a0 2024-04-25T14:37:27.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.658+0000 7f9843400640 1 -- 172.21.15.53:0/3831886535 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9838030bd0 con 0x7f984c1097a0 2024-04-25T14:37:27.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.658+0000 7f9843400640 1 -- 172.21.15.53:0/3831886535 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838003850 con 0x7f984c1097a0 2024-04-25T14:37:27.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.658+0000 7f9851600640 1 -- 172.21.15.53:0/3831886535 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f984c1a6b30 con 0x7f984c1097a0 2024-04-25T14:37:27.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.660+0000 7f9851600640 1 -- 172.21.15.53:0/3831886535 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f984c10f9a0 con 0x7f984c1097a0 2024-04-25T14:37:27.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.662+0000 7f9843400640 1 -- 172.21.15.53:0/3831886535 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9838043020 con 0x7f984c1097a0 2024-04-25T14:37:27.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.663+0000 7f9843400640 1 --2- 172.21.15.53:0/3831886535 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f98300a3bb0 0x7f98300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:27.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.663+0000 7f9843400640 1 -- 172.21.15.53:0/3831886535 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f98380ef100 con 0x7f984c1097a0 2024-04-25T14:37:27.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.664+0000 7f984aa00640 1 --2- 172.21.15.53:0/3831886535 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f98300a3bb0 0x7f98300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:27.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.665+0000 7f984aa00640 1 --2- 172.21.15.53:0/3831886535 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f98300a3bb0 0x7f98300a6070 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f984c109600 tx=0x7f983c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:37:27.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.668+0000 7f9843400640 1 -- 172.21.15.53:0/3831886535 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9838030d80 con 0x7f984c1097a0 2024-04-25T14:37:27.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.914+0000 7f9851600640 1 -- 172.21.15.53:0/3831886535 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f984c062b50 con 0x7f984c1097a0 2024-04-25T14:37:27.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.915+0000 7f9843400640 1 -- 172.21.15.53:0/3831886535 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f98380bbee0 con 0x7f984c1097a0 2024-04-25T14:37:27.916 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:37:27.916 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-25T14:37:27.916 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 1 2024-04-25T14:37:27.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.917+0000 7f9851600640 1 -- 172.21.15.53:0/3831886535 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f98300a3bb0 msgr2=0x7f98300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:27.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.917+0000 7f9851600640 1 --2- 172.21.15.53:0/3831886535 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f98300a3bb0 0x7f98300a6070 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f984c109600 tx=0x7f983c005f90 comp rx=0 tx=0).stop 2024-04-25T14:37:27.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.917+0000 7f9851600640 1 -- 172.21.15.53:0/3831886535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f984c1097a0 msgr2=0x7f984c19ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:27.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.917+0000 7f9851600640 1 --2- 172.21.15.53:0/3831886535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f984c1097a0 0x7f984c19ed30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f983802a860 tx=0x7f9838032dc0 comp rx=0 tx=0).stop 2024-04-25T14:37:27.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.918+0000 7f9851600640 1 -- 172.21.15.53:0/3831886535 shutdown_connections 2024-04-25T14:37:27.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.918+0000 7f9851600640 1 --2- 172.21.15.53:0/3831886535 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f98300a3bb0 0x7f98300a6070 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:27.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.918+0000 7f9851600640 1 --2- 172.21.15.53:0/3831886535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f984c10a160 0x7f984c1a5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:27.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.918+0000 7f9851600640 1 --2- 172.21.15.53:0/3831886535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f984c1097a0 0x7f984c19ed30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:27.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.918+0000 7f9851600640 1 --2- 172.21.15.53:0/3831886535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f984c1085b0 0x7f984c19e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:27.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.918+0000 7f9851600640 1 -- 172.21.15.53:0/3831886535 >> 172.21.15.53:0/3831886535 conn(0x7f984c077570 msgr2=0x7f984c10cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:37:27.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.918+0000 7f9851600640 1 -- 172.21.15.53:0/3831886535 shutdown_connections 2024-04-25T14:37:27.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:27.918+0000 7f9851600640 1 -- 172.21.15.53:0/3831886535 wait complete. 2024-04-25T14:37:28.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:27 smithi183 ceph-mon[32584]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:28.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:27 smithi183 ceph-mon[32584]: Deploying daemon mds.l on smithi183 2024-04-25T14:37:28.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:27 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/2663458791' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:37:28.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:27 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/3831886535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:37:28.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:27 smithi053 ceph-mon[27773]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:28.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:27 smithi053 ceph-mon[27773]: Deploying daemon mds.l on smithi183 2024-04-25T14:37:28.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:27 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/2663458791' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:37:28.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:27 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/3831886535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:37:28.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:27 smithi150 ceph-mon[32504]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:28.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:27 smithi150 ceph-mon[32504]: Deploying daemon mds.l on smithi183 2024-04-25T14:37:28.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:27 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/2663458791' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:37:28.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:27 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/3831886535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:37:29.204 INFO:journalctl@ceph.mds.l.smithi183.stdout:Apr 25 14:37:29 smithi183 systemd[1]: Starting Ceph mds.l for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:37:29.488 INFO:journalctl@ceph.mds.l.smithi183.stdout:Apr 25 14:37:29 smithi183 podman[61955]: 2024-04-25 14:37:29.169915314 +0000 UTC m=+0.017856159 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T14:37:29.489 INFO:journalctl@ceph.mds.l.smithi183.stdout:Apr 25 14:37:29 smithi183 podman[61955]: 2024-04-25 14:37:29.380497359 +0000 UTC m=+0.228438208 container create de96912deee8eefe4d8f0c22e5acf13bcc68dcbc55725187b7056270b884392d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-l, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-25T14:37:29.939 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:37:29.954 INFO:journalctl@ceph.mds.l.smithi183.stdout:Apr 25 14:37:29 smithi183 podman[61955]: 2024-04-25 14:37:29.599658956 +0000 UTC m=+0.447599802 container init de96912deee8eefe4d8f0c22e5acf13bcc68dcbc55725187b7056270b884392d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240416) 2024-04-25T14:37:29.954 INFO:journalctl@ceph.mds.l.smithi183.stdout:Apr 25 14:37:29 smithi183 podman[61955]: 2024-04-25 14:37:29.602145204 +0000 UTC m=+0.450086053 container start de96912deee8eefe4d8f0c22e5acf13bcc68dcbc55725187b7056270b884392d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-l, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-25T14:37:29.954 INFO:journalctl@ceph.mds.l.smithi183.stdout:Apr 25 14:37:29 smithi183 ceph-mds[61974]: -- 172.21.15.183:0/868934760 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561a86d89a40 con 0x561a86d5cc00 2024-04-25T14:37:29.954 INFO:journalctl@ceph.mds.l.smithi183.stdout:Apr 25 14:37:29 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-l[61970]: starting mds.l at 2024-04-25T14:37:29.954 INFO:journalctl@ceph.mds.l.smithi183.stdout:Apr 25 14:37:29 smithi183 bash[61955]: de96912deee8eefe4d8f0c22e5acf13bcc68dcbc55725187b7056270b884392d 2024-04-25T14:37:29.954 INFO:journalctl@ceph.mds.l.smithi183.stdout:Apr 25 14:37:29 smithi183 systemd[1]: Started Ceph mds.l for fd30e6ce-030f-11ef-bc93-c7b262605968. 2024-04-25T14:37:30.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.165+0000 7f98a5200640 1 -- 172.21.15.53:0/118580230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98a0103190 msgr2=0x7f98a0103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:30.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.165+0000 7f98a5200640 1 --2- 172.21.15.53:0/118580230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98a0103190 0x7f98a0103610 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f988c009900 tx=0x7f988c02a3f0 comp rx=0 tx=0).stop 2024-04-25T14:37:30.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.166+0000 7f98a5200640 1 -- 172.21.15.53:0/118580230 shutdown_connections 2024-04-25T14:37:30.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.166+0000 7f98a5200640 1 --2- 172.21.15.53:0/118580230 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98a0103b50 0x7f98a010a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:30.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.166+0000 7f98a5200640 1 --2- 172.21.15.53:0/118580230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98a0103190 0x7f98a0103610 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:30.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.166+0000 7f98a5200640 1 --2- 172.21.15.53:0/118580230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98a0101fa0 0x7f98a01023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:30.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.166+0000 7f98a5200640 1 -- 172.21.15.53:0/118580230 >> 172.21.15.53:0/118580230 conn(0x7f98a00fd530 msgr2=0x7f98a00ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:37:30.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.166+0000 7f98a5200640 1 -- 172.21.15.53:0/118580230 shutdown_connections 2024-04-25T14:37:30.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.166+0000 7f98a5200640 1 -- 172.21.15.53:0/118580230 wait complete. 2024-04-25T14:37:30.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.167+0000 7f98a5200640 1 Processor -- start 2024-04-25T14:37:30.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.167+0000 7f98a5200640 1 -- start start 2024-04-25T14:37:30.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.167+0000 7f98a5200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98a0101fa0 0x7f98a0075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:30.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.167+0000 7f98a5200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98a0103190 0x7f98a0074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:30.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.167+0000 7f98a5200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98a0103b50 0x7f98a0074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:30.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.167+0000 7f98a5200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f98a010dae0 con 0x7f98a0103190 2024-04-25T14:37:30.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.167+0000 7f98a5200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f98a010d960 con 0x7f98a0101fa0 2024-04-25T14:37:30.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.167+0000 7f98a5200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f98a010dc60 con 0x7f98a0103b50 2024-04-25T14:37:30.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.167+0000 7f989de00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98a0101fa0 0x7f98a0075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:30.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.167+0000 7f989de00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98a0101fa0 0x7f98a0075d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.53:40672/0 (socket says 172.21.15.53:40672) 2024-04-25T14:37:30.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.167+0000 7f989de00640 1 -- 172.21.15.53:0/2926982439 learned_addr learned my addr 172.21.15.53:0/2926982439 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:37:30.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.167+0000 7f989d400640 1 --2- 172.21.15.53:0/2926982439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98a0103190 0x7f98a0074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:30.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.167+0000 7f989e800640 1 --2- 172.21.15.53:0/2926982439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98a0103b50 0x7f98a0074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:30.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.168+0000 7f989de00640 1 -- 172.21.15.53:0/2926982439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98a0103b50 msgr2=0x7f98a0074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:30.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.168+0000 7f989de00640 1 --2- 172.21.15.53:0/2926982439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98a0103b50 0x7f98a0074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:30.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.168+0000 7f989de00640 1 -- 172.21.15.53:0/2926982439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98a0103190 msgr2=0x7f98a0074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:30.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.168+0000 7f989de00640 1 --2- 172.21.15.53:0/2926982439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98a0103190 0x7f98a0074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:30.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.168+0000 7f989de00640 1 -- 172.21.15.53:0/2926982439 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a0075110 con 0x7f98a0101fa0 2024-04-25T14:37:30.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.168+0000 7f989e800640 1 --2- 172.21.15.53:0/2926982439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98a0103b50 0x7f98a0074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:37:30.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.168+0000 7f989de00640 1 --2- 172.21.15.53:0/2926982439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98a0101fa0 0x7f98a0075d00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f989000e9c0 tx=0x7f989000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:37:30.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.168+0000 7f9896a00640 1 -- 172.21.15.53:0/2926982439 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989000cde0 con 0x7f98a0101fa0 2024-04-25T14:37:30.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.168+0000 7f98a5200640 1 -- 172.21.15.53:0/2926982439 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a01a22d0 con 0x7f98a0101fa0 2024-04-25T14:37:30.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.168+0000 7f9896a00640 1 -- 172.21.15.53:0/2926982439 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98900108e0 con 0x7f98a0101fa0 2024-04-25T14:37:30.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.168+0000 7f9896a00640 1 -- 172.21.15.53:0/2926982439 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98900188d0 con 0x7f98a0101fa0 2024-04-25T14:37:30.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.169+0000 7f98a5200640 1 -- 172.21.15.53:0/2926982439 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a01a2840 con 0x7f98a0101fa0 2024-04-25T14:37:30.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.170+0000 7f98a5200640 1 -- 172.21.15.53:0/2926982439 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98a0109310 con 0x7f98a0101fa0 2024-04-25T14:37:30.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.172+0000 7f9896a00640 1 -- 172.21.15.53:0/2926982439 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9890010a90 con 0x7f98a0101fa0 2024-04-25T14:37:30.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.173+0000 7f9896a00640 1 --2- 172.21.15.53:0/2926982439 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f98840a3c30 0x7f98840a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:30.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.173+0000 7f9896a00640 1 -- 172.21.15.53:0/2926982439 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f98900d06e0 con 0x7f98a0101fa0 2024-04-25T14:37:30.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.174+0000 7f989d400640 1 --2- 172.21.15.53:0/2926982439 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f98840a3c30 0x7f98840a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:30.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.174+0000 7f989d400640 1 --2- 172.21.15.53:0/2926982439 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f98840a3c30 0x7f98840a60f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f988c000c00 tx=0x7f988c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:37:30.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.180+0000 7f9896a00640 1 -- 172.21.15.53:0/2926982439 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f989009d4c0 con 0x7f98a0101fa0 2024-04-25T14:37:30.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:29 smithi053 ceph-mon[27773]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:30.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:29 smithi053 ceph-mon[27773]: mds.? [v2:172.21.15.183:6832/94143643,v1:172.21.15.183:6833/94143643] up:boot 2024-04-25T14:37:30.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:29 smithi053 ceph-mon[27773]: fsmap 1 up:standby 2024-04-25T14:37:30.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:29 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T14:37:30.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:29 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:30.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:29 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:30.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:29 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:30.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:29 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:37:30.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:29 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:37:30.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:29 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:37:30.431 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.430+0000 7f98a5200640 1 -- 172.21.15.53:0/2926982439 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98a002d830 con 0x7f98a0101fa0 2024-04-25T14:37:30.431 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.431+0000 7f9896a00640 1 -- 172.21.15.53:0/2926982439 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1390 (secure 0 0 0) 0x7f989009cdc0 con 0x7f98a0101fa0 2024-04-25T14:37:30.432 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:37:30.432 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/94143643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":94143643},{"type":"v1","addr":"172.21.15.183:6833","nonce":94143643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T14:37:30.432 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 2 2024-04-25T14:37:30.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.433+0000 7f98a5200640 1 -- 172.21.15.53:0/2926982439 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f98840a3c30 msgr2=0x7f98840a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:30.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.433+0000 7f98a5200640 1 --2- 172.21.15.53:0/2926982439 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f98840a3c30 0x7f98840a60f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f988c000c00 tx=0x7f988c0314a0 comp rx=0 tx=0).stop 2024-04-25T14:37:30.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.433+0000 7f98a5200640 1 -- 172.21.15.53:0/2926982439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98a0101fa0 msgr2=0x7f98a0075d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:30.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.433+0000 7f98a5200640 1 --2- 172.21.15.53:0/2926982439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98a0101fa0 0x7f98a0075d00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f989000e9c0 tx=0x7f989000ee80 comp rx=0 tx=0).stop 2024-04-25T14:37:30.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.433+0000 7f98a5200640 1 -- 172.21.15.53:0/2926982439 shutdown_connections 2024-04-25T14:37:30.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.433+0000 7f98a5200640 1 --2- 172.21.15.53:0/2926982439 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f98840a3c30 0x7f98840a60f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:30.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.433+0000 7f98a5200640 1 --2- 172.21.15.53:0/2926982439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98a0103b50 0x7f98a0074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:30.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.433+0000 7f98a5200640 1 --2- 172.21.15.53:0/2926982439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98a0103190 0x7f98a0074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:30.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.433+0000 7f98a5200640 1 --2- 172.21.15.53:0/2926982439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98a0101fa0 0x7f98a0075d00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:30.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.433+0000 7f98a5200640 1 -- 172.21.15.53:0/2926982439 >> 172.21.15.53:0/2926982439 conn(0x7f98a00fd530 msgr2=0x7f98a01064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:37:30.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.434+0000 7f98a5200640 1 -- 172.21.15.53:0/2926982439 shutdown_connections 2024-04-25T14:37:30.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:30.434+0000 7f98a5200640 1 -- 172.21.15.53:0/2926982439 wait complete. 2024-04-25T14:37:30.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:29 smithi183 ceph-mon[32584]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:30.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:29 smithi183 ceph-mon[32584]: mds.? [v2:172.21.15.183:6832/94143643,v1:172.21.15.183:6833/94143643] up:boot 2024-04-25T14:37:30.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:29 smithi183 ceph-mon[32584]: fsmap 1 up:standby 2024-04-25T14:37:30.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:29 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T14:37:30.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:29 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:30.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:29 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:30.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:29 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:30.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:29 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:37:30.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:29 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:37:30.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:29 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:37:30.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:29 smithi150 ceph-mon[32504]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:30.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:29 smithi150 ceph-mon[32504]: mds.? [v2:172.21.15.183:6832/94143643,v1:172.21.15.183:6833/94143643] up:boot 2024-04-25T14:37:30.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:29 smithi150 ceph-mon[32504]: fsmap 1 up:standby 2024-04-25T14:37:30.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:29 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T14:37:30.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:29 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:30.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:29 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:30.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:29 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:30.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:29 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:37:30.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:29 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:37:30.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:29 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:37:31.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:31 smithi053 ceph-mon[27773]: Deploying daemon mds.k on smithi150 2024-04-25T14:37:31.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:31 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/2926982439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:37:31.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:31 smithi183 ceph-mon[32584]: Deploying daemon mds.k on smithi150 2024-04-25T14:37:31.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:31 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/2926982439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:37:31.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:31 smithi150 ceph-mon[32504]: Deploying daemon mds.k on smithi150 2024-04-25T14:37:31.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:31 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/2926982439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:37:32.009 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 25 14:37:31 smithi150 systemd[1]: Starting Ceph mds.k for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:37:32.259 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:32 smithi150 ceph-mon[32504]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:32.259 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 25 14:37:32 smithi150 podman[61947]: 2024-04-25 14:37:32.049495999 +0000 UTC m=+0.018278843 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T14:37:32.259 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 25 14:37:32 smithi150 podman[61947]: 2024-04-25 14:37:32.162993546 +0000 UTC m=+0.131776428 container create 21b3427b8c012213dd9767aab26b693e6550bc90159c24f403b9132dc5046a3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-k, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:37:32.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:32 smithi053 ceph-mon[27773]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:32.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:32 smithi183 ceph-mon[32584]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:32.456 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:37:32.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.679+0000 7f62a6000640 1 -- 172.21.15.53:0/860030639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62a0101fa0 msgr2=0x7f62a01023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:32.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.679+0000 7f62a6000640 1 --2- 172.21.15.53:0/860030639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62a0101fa0 0x7f62a01023c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6290009ca0 tx=0x7f629002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:37:32.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.680+0000 7f62a6000640 1 -- 172.21.15.53:0/860030639 shutdown_connections 2024-04-25T14:37:32.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.680+0000 7f62a6000640 1 --2- 172.21.15.53:0/860030639 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62a0103b50 0x7f62a010a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:32.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.680+0000 7f62a6000640 1 --2- 172.21.15.53:0/860030639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62a0103190 0x7f62a0103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:32.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.680+0000 7f62a6000640 1 --2- 172.21.15.53:0/860030639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62a0101fa0 0x7f62a01023c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:32.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.680+0000 7f62a6000640 1 -- 172.21.15.53:0/860030639 >> 172.21.15.53:0/860030639 conn(0x7f62a00fd530 msgr2=0x7f62a00ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:37:32.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.680+0000 7f62a6000640 1 -- 172.21.15.53:0/860030639 shutdown_connections 2024-04-25T14:37:32.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.680+0000 7f62a6000640 1 -- 172.21.15.53:0/860030639 wait complete. 2024-04-25T14:37:32.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.681+0000 7f62a6000640 1 Processor -- start 2024-04-25T14:37:32.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.681+0000 7f62a6000640 1 -- start start 2024-04-25T14:37:32.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.681+0000 7f62a6000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62a0101fa0 0x7f62a019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:32.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.681+0000 7f62a6000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62a0103190 0x7f62a019a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:32.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.681+0000 7f62a6000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62a0103b50 0x7f62a01a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:32.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.681+0000 7f62a6000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f62a010db30 con 0x7f62a0103190 2024-04-25T14:37:32.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.681+0000 7f62a6000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f62a010d9b0 con 0x7f62a0101fa0 2024-04-25T14:37:32.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.681+0000 7f62a6000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f62a010dcb0 con 0x7f62a0103b50 2024-04-25T14:37:32.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.681+0000 7f6296a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62a0103190 0x7f62a019a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:32.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.681+0000 7f629fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62a0103b50 0x7f62a01a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:32.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.681+0000 7f629f400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62a0101fa0 0x7f62a019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:32.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.681+0000 7f629fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62a0103b50 0x7f62a01a19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:52508/0 (socket says 172.21.15.53:52508) 2024-04-25T14:37:32.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.681+0000 7f629fe00640 1 -- 172.21.15.53:0/3659051262 learned_addr learned my addr 172.21.15.53:0/3659051262 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:37:32.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.681+0000 7f6296a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62a0103190 0x7f62a019a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56458/0 (socket says 172.21.15.53:56458) 2024-04-25T14:37:32.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.681+0000 7f629fe00640 1 -- 172.21.15.53:0/3659051262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62a0101fa0 msgr2=0x7f62a019a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:32.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.681+0000 7f629fe00640 1 --2- 172.21.15.53:0/3659051262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62a0101fa0 0x7f62a019a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:32.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.681+0000 7f629fe00640 1 -- 172.21.15.53:0/3659051262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62a0103190 msgr2=0x7f62a019a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:32.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.681+0000 7f629fe00640 1 --2- 172.21.15.53:0/3659051262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62a0103190 0x7f62a019a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:32.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.681+0000 7f629fe00640 1 -- 172.21.15.53:0/3659051262 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a01a2010 con 0x7f62a0103b50 2024-04-25T14:37:32.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.681+0000 7f629f400640 1 --2- 172.21.15.53:0/3659051262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62a0101fa0 0x7f62a019a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:37:32.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.681+0000 7f6296a00640 1 --2- 172.21.15.53:0/3659051262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62a0103190 0x7f62a019a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:37:32.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.683+0000 7f629fe00640 1 --2- 172.21.15.53:0/3659051262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62a0103b50 0x7f62a01a19a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f629800e970 tx=0x7f629800ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:37:32.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.683+0000 7f629cc00640 1 -- 172.21.15.53:0/3659051262 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629800ccc0 con 0x7f62a0103b50 2024-04-25T14:37:32.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.683+0000 7f629cc00640 1 -- 172.21.15.53:0/3659051262 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f629800ce60 con 0x7f62a0103b50 2024-04-25T14:37:32.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.683+0000 7f62a6000640 1 -- 172.21.15.53:0/3659051262 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62a01a2300 con 0x7f62a0103b50 2024-04-25T14:37:32.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.683+0000 7f629cc00640 1 -- 172.21.15.53:0/3659051262 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6298010430 con 0x7f62a0103b50 2024-04-25T14:37:32.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.683+0000 7f62a6000640 1 -- 172.21.15.53:0/3659051262 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62a01a28a0 con 0x7f62a0103b50 2024-04-25T14:37:32.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.684+0000 7f62a6000640 1 -- 172.21.15.53:0/3659051262 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62a01a2b60 con 0x7f62a0103b50 2024-04-25T14:37:32.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.686+0000 7f629cc00640 1 -- 172.21.15.53:0/3659051262 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6298004f90 con 0x7f62a0103b50 2024-04-25T14:37:32.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.687+0000 7f629cc00640 1 --2- 172.21.15.53:0/3659051262 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f62840a3c30 0x7f62840a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:32.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.688+0000 7f629cc00640 1 -- 172.21.15.53:0/3659051262 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f62980d0350 con 0x7f62a0103b50 2024-04-25T14:37:32.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.688+0000 7f629f400640 1 --2- 172.21.15.53:0/3659051262 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f62840a3c30 0x7f62840a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:32.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.689+0000 7f629f400640 1 --2- 172.21.15.53:0/3659051262 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f62840a3c30 0x7f62840a60f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6290000f80 tx=0x7f6290004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:37:32.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.692+0000 7f629cc00640 1 -- 172.21.15.53:0/3659051262 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f629809d130 con 0x7f62a0103b50 2024-04-25T14:37:32.711 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 25 14:37:32 smithi150 podman[61947]: 2024-04-25 14:37:32.382072134 +0000 UTC m=+0.350854965 container init 21b3427b8c012213dd9767aab26b693e6550bc90159c24f403b9132dc5046a3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-k, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0) 2024-04-25T14:37:32.711 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 25 14:37:32 smithi150 podman[61947]: 2024-04-25 14:37:32.384809431 +0000 UTC m=+0.353592251 container start 21b3427b8c012213dd9767aab26b693e6550bc90159c24f403b9132dc5046a3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-k, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-25T14:37:32.711 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 25 14:37:32 smithi150 ceph-mds[61966]: -- 172.21.15.150:0/1915697946 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5640f244ba40 con 0x5640f241f000 2024-04-25T14:37:32.711 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 25 14:37:32 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-k[61962]: starting mds.k at 2024-04-25T14:37:32.711 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 25 14:37:32 smithi150 bash[61947]: 21b3427b8c012213dd9767aab26b693e6550bc90159c24f403b9132dc5046a3c 2024-04-25T14:37:32.711 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 25 14:37:32 smithi150 systemd[1]: Started Ceph mds.k for fd30e6ce-030f-11ef-bc93-c7b262605968. 2024-04-25T14:37:32.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.945+0000 7f62a6000640 1 -- 172.21.15.53:0/3659051262 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62a01a2d30 con 0x7f62a0103b50 2024-04-25T14:37:32.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.945+0000 7f629cc00640 1 -- 172.21.15.53:0/3659051262 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1390 (secure 0 0 0) 0x7f629809ca30 con 0x7f62a0103b50 2024-04-25T14:37:32.946 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:37:32.946 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/94143643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":94143643},{"type":"v1","addr":"172.21.15.183:6833","nonce":94143643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T14:37:32.946 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 2 2024-04-25T14:37:32.949 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.948+0000 7f62a6000640 1 -- 172.21.15.53:0/3659051262 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f62840a3c30 msgr2=0x7f62840a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:32.949 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.948+0000 7f62a6000640 1 --2- 172.21.15.53:0/3659051262 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f62840a3c30 0x7f62840a60f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6290000f80 tx=0x7f6290004a40 comp rx=0 tx=0).stop 2024-04-25T14:37:32.949 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.949+0000 7f62a6000640 1 -- 172.21.15.53:0/3659051262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62a0103b50 msgr2=0x7f62a01a19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:32.949 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.949+0000 7f62a6000640 1 --2- 172.21.15.53:0/3659051262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62a0103b50 0x7f62a01a19a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f629800e970 tx=0x7f629800ed70 comp rx=0 tx=0).stop 2024-04-25T14:37:32.949 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.949+0000 7f62a6000640 1 -- 172.21.15.53:0/3659051262 shutdown_connections 2024-04-25T14:37:32.949 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.949+0000 7f62a6000640 1 --2- 172.21.15.53:0/3659051262 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f62840a3c30 0x7f62840a60f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:32.949 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.949+0000 7f62a6000640 1 --2- 172.21.15.53:0/3659051262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62a0103b50 0x7f62a01a19a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:32.949 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.949+0000 7f62a6000640 1 --2- 172.21.15.53:0/3659051262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62a0103190 0x7f62a019a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:32.949 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.949+0000 7f62a6000640 1 --2- 172.21.15.53:0/3659051262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62a0101fa0 0x7f62a019a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:32.949 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.949+0000 7f62a6000640 1 -- 172.21.15.53:0/3659051262 >> 172.21.15.53:0/3659051262 conn(0x7f62a00fd530 msgr2=0x7f62a0074180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:37:32.949 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.949+0000 7f62a6000640 1 -- 172.21.15.53:0/3659051262 shutdown_connections 2024-04-25T14:37:32.949 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:32.949+0000 7f62a6000640 1 -- 172.21.15.53:0/3659051262 wait complete. 2024-04-25T14:37:33.808 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:33 smithi150 ceph-mon[32504]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:33.809 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:33 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:33.809 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:33 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:33.809 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:33 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:33.809 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:33 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:37:33.809 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:33 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:37:33.809 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:33 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:37:33.809 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:33 smithi150 ceph-mon[32504]: Deploying daemon mds.j on smithi053 2024-04-25T14:37:33.809 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:33 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/3659051262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:37:33.809 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:33 smithi150 ceph-mon[32504]: mds.? [v2:172.21.15.150:6832/2966992472,v1:172.21.15.150:6833/2966992472] up:boot 2024-04-25T14:37:33.809 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:33 smithi150 ceph-mon[32504]: fsmap 2 up:standby 2024-04-25T14:37:33.809 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:33 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T14:37:33.837 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:33 smithi053 ceph-mon[27773]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:33.837 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:33 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:33.837 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:33 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:33.837 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:33 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:33.837 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:33 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:37:33.837 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:33 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:37:33.837 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:33 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:37:33.837 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:33 smithi053 ceph-mon[27773]: Deploying daemon mds.j on smithi053 2024-04-25T14:37:33.837 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:33 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/3659051262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:37:33.837 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:33 smithi053 ceph-mon[27773]: mds.? [v2:172.21.15.150:6832/2966992472,v1:172.21.15.150:6833/2966992472] up:boot 2024-04-25T14:37:33.837 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:33 smithi053 ceph-mon[27773]: fsmap 2 up:standby 2024-04-25T14:37:33.838 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:33 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T14:37:33.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:33 smithi183 ceph-mon[32584]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:33.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:33 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:33.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:33 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:33.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:33 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:33.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:33 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:37:33.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:33 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:37:33.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:33 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:37:33.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:33 smithi183 ceph-mon[32584]: Deploying daemon mds.j on smithi053 2024-04-25T14:37:33.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:33 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/3659051262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:37:33.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:33 smithi183 ceph-mon[32584]: mds.? [v2:172.21.15.150:6832/2966992472,v1:172.21.15.150:6833/2966992472] up:boot 2024-04-25T14:37:33.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:33 smithi183 ceph-mon[32584]: fsmap 2 up:standby 2024-04-25T14:37:33.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:33 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T14:37:34.969 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:37:35.399 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.398+0000 7fcbb3e00640 1 -- 172.21.15.53:0/3458533909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcbb41059a0 msgr2=0x7fcbb4105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:35.399 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.398+0000 7fcbb3e00640 1 --2- 172.21.15.53:0/3458533909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcbb41059a0 0x7fcbb4105e20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcbac009f00 tx=0x7fcbac02ab20 comp rx=0 tx=0).stop 2024-04-25T14:37:35.399 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.399+0000 7fcbb3e00640 1 -- 172.21.15.53:0/3458533909 shutdown_connections 2024-04-25T14:37:35.399 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.399+0000 7fcbb3e00640 1 --2- 172.21.15.53:0/3458533909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcbb41059a0 0x7fcbb4105e20 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:35.399 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.399+0000 7fcbb3e00640 1 --2- 172.21.15.53:0/3458533909 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbb4102ff0 0x7fcbb4105460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:35.399 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.399+0000 7fcbb3e00640 1 --2- 172.21.15.53:0/3458533909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbb4100680 0x7fcbb4102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:35.399 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.399+0000 7fcbb3e00640 1 -- 172.21.15.53:0/3458533909 >> 172.21.15.53:0/3458533909 conn(0x7fcbb40fc200 msgr2=0x7fcbb40fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:37:35.399 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.399+0000 7fcbb3e00640 1 -- 172.21.15.53:0/3458533909 shutdown_connections 2024-04-25T14:37:35.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.400+0000 7fcbb3e00640 1 -- 172.21.15.53:0/3458533909 wait complete. 2024-04-25T14:37:35.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.401+0000 7fcbb3e00640 1 Processor -- start 2024-04-25T14:37:35.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.401+0000 7fcbb3e00640 1 -- start start 2024-04-25T14:37:35.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.401+0000 7fcbb3e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbb4100680 0x7fcbb419e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:35.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.401+0000 7fcbb3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbb4102ff0 0x7fcbb419eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:35.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.401+0000 7fcbb3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcbb41059a0 0x7fcbb41a5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:35.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.401+0000 7fcbb3e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb4113ef0 con 0x7fcbb41059a0 2024-04-25T14:37:35.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.401+0000 7fcbb3e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb4113d70 con 0x7fcbb4100680 2024-04-25T14:37:35.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.401+0000 7fcbb3e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb4114070 con 0x7fcbb4102ff0 2024-04-25T14:37:35.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.402+0000 7fcbb2a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbb4100680 0x7fcbb419e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:35.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.402+0000 7fcbb2a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbb4100680 0x7fcbb419e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.53:55832/0 (socket says 172.21.15.53:55832) 2024-04-25T14:37:35.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.402+0000 7fcbb2a00640 1 -- 172.21.15.53:0/1213604997 learned_addr learned my addr 172.21.15.53:0/1213604997 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:37:35.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.402+0000 7fcbb3400640 1 --2- 172.21.15.53:0/1213604997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcbb41059a0 0x7fcbb41a5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:35.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.402+0000 7fcbb2000640 1 --2- 172.21.15.53:0/1213604997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbb4102ff0 0x7fcbb419eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:35.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.402+0000 7fcbb2a00640 1 -- 172.21.15.53:0/1213604997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbb4102ff0 msgr2=0x7fcbb419eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:35.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.402+0000 7fcbb2a00640 1 --2- 172.21.15.53:0/1213604997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbb4102ff0 0x7fcbb419eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:35.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.402+0000 7fcbb2a00640 1 -- 172.21.15.53:0/1213604997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcbb41059a0 msgr2=0x7fcbb41a5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:35.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.402+0000 7fcbb2a00640 1 --2- 172.21.15.53:0/1213604997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcbb41059a0 0x7fcbb41a5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:35.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.402+0000 7fcbb2a00640 1 -- 172.21.15.53:0/1213604997 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb41a61a0 con 0x7fcbb4100680 2024-04-25T14:37:35.403 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.403+0000 7fcbb2a00640 1 --2- 172.21.15.53:0/1213604997 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbb4100680 0x7fcbb419e580 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcba000d950 tx=0x7fcba000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:37:35.403 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.403+0000 7fcbb8a00640 1 -- 172.21.15.53:0/1213604997 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba0017070 con 0x7fcbb4100680 2024-04-25T14:37:35.403 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.403+0000 7fcbb3e00640 1 -- 172.21.15.53:0/1213604997 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbb41a6490 con 0x7fcbb4100680 2024-04-25T14:37:35.404 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.403+0000 7fcbb8a00640 1 -- 172.21.15.53:0/1213604997 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcba0009c30 con 0x7fcbb4100680 2024-04-25T14:37:35.404 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.403+0000 7fcbb3e00640 1 -- 172.21.15.53:0/1213604997 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbb41a6a30 con 0x7fcbb4100680 2024-04-25T14:37:35.404 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.403+0000 7fcbb8a00640 1 -- 172.21.15.53:0/1213604997 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba0020060 con 0x7fcbb4100680 2024-04-25T14:37:35.404 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.405+0000 7fcbb3e00640 1 -- 172.21.15.53:0/1213604997 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbb40fdcf0 con 0x7fcbb4100680 2024-04-25T14:37:35.407 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.407+0000 7fcbb8a00640 1 -- 172.21.15.53:0/1213604997 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fcba0010460 con 0x7fcbb4100680 2024-04-25T14:37:35.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.410+0000 7fcbb8a00640 1 --2- 172.21.15.53:0/1213604997 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fcb980a3c30 0x7fcb980a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:35.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.410+0000 7fcbb8a00640 1 -- 172.21.15.53:0/1213604997 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fcba00cf710 con 0x7fcbb4100680 2024-04-25T14:37:35.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.410+0000 7fcbb2000640 1 --2- 172.21.15.53:0/1213604997 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fcb980a3c30 0x7fcb980a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:35.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.411+0000 7fcbb2000640 1 --2- 172.21.15.53:0/1213604997 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fcb980a3c30 0x7fcb980a60f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcbb419fa90 tx=0x7fcba400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:37:35.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.413+0000 7fcbb8a00640 1 -- 172.21.15.53:0/1213604997 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcba009bea0 con 0x7fcbb4100680 2024-04-25T14:37:35.621 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:35 smithi053 ceph-mon[27773]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:35.622 INFO:journalctl@ceph.mds.j.smithi053.stdout:Apr 25 14:37:35 smithi053 systemd[1]: Starting Ceph mds.j for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:37:35.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.670+0000 7fcbb3e00640 1 -- 172.21.15.53:0/1213604997 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcbb402d5c0 con 0x7fcbb4100680 2024-04-25T14:37:35.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.671+0000 7fcbb8a00640 1 -- 172.21.15.53:0/1213604997 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2218 (secure 0 0 0) 0x7fcba00a01a0 con 0x7fcbb4100680 2024-04-25T14:37:35.671 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:37:35.671 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2966992472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2966992472},{"type":"v1","addr":"172.21.15.150:6833","nonce":2966992472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/94143643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":94143643},{"type":"v1","addr":"172.21.15.183:6833","nonce":94143643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T14:37:35.672 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 3 2024-04-25T14:37:35.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.674+0000 7fcbb3e00640 1 -- 172.21.15.53:0/1213604997 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fcb980a3c30 msgr2=0x7fcb980a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:35.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.674+0000 7fcbb3e00640 1 --2- 172.21.15.53:0/1213604997 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fcb980a3c30 0x7fcb980a60f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcbb419fa90 tx=0x7fcba400b390 comp rx=0 tx=0).stop 2024-04-25T14:37:35.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.674+0000 7fcbb3e00640 1 -- 172.21.15.53:0/1213604997 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbb4100680 msgr2=0x7fcbb419e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:35.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.674+0000 7fcbb3e00640 1 --2- 172.21.15.53:0/1213604997 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbb4100680 0x7fcbb419e580 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcba000d950 tx=0x7fcba000de10 comp rx=0 tx=0).stop 2024-04-25T14:37:35.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.674+0000 7fcbb3e00640 1 -- 172.21.15.53:0/1213604997 shutdown_connections 2024-04-25T14:37:35.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.674+0000 7fcbb3e00640 1 --2- 172.21.15.53:0/1213604997 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fcb980a3c30 0x7fcb980a60f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:35.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.674+0000 7fcbb3e00640 1 --2- 172.21.15.53:0/1213604997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcbb41059a0 0x7fcbb41a5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:35.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.674+0000 7fcbb3e00640 1 --2- 172.21.15.53:0/1213604997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbb4102ff0 0x7fcbb419eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:35.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.674+0000 7fcbb3e00640 1 --2- 172.21.15.53:0/1213604997 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbb4100680 0x7fcbb419e580 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:35.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.674+0000 7fcbb3e00640 1 -- 172.21.15.53:0/1213604997 >> 172.21.15.53:0/1213604997 conn(0x7fcbb40fc200 msgr2=0x7fcbb4101000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:37:35.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.674+0000 7fcbb3e00640 1 -- 172.21.15.53:0/1213604997 shutdown_connections 2024-04-25T14:37:35.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:35.674+0000 7fcbb3e00640 1 -- 172.21.15.53:0/1213604997 wait complete. 2024-04-25T14:37:35.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:35 smithi183 ceph-mon[32584]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:35.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:35 smithi150 ceph-mon[32504]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:36.284 INFO:journalctl@ceph.mds.j.smithi053.stdout:Apr 25 14:37:35 smithi053 podman[67989]: 2024-04-25 14:37:35.90406271 +0000 UTC m=+0.037568170 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T14:37:36.284 INFO:journalctl@ceph.mds.j.smithi053.stdout:Apr 25 14:37:36 smithi053 podman[67989]: 2024-04-25 14:37:36.001307626 +0000 UTC m=+0.134813145 container create 5baff6f1ec70e2890ea55bb2314c42a865b9c558dcc3ec1dd494c9a135e21c3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-j, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-25T14:37:36.557 INFO:journalctl@ceph.mds.j.smithi053.stdout:Apr 25 14:37:36 smithi053 podman[67989]: 2024-04-25 14:37:36.305131278 +0000 UTC m=+0.438636768 container init 5baff6f1ec70e2890ea55bb2314c42a865b9c558dcc3ec1dd494c9a135e21c3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-j, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True) 2024-04-25T14:37:36.558 INFO:journalctl@ceph.mds.j.smithi053.stdout:Apr 25 14:37:36 smithi053 podman[67989]: 2024-04-25 14:37:36.316198812 +0000 UTC m=+0.449704288 container start 5baff6f1ec70e2890ea55bb2314c42a865b9c558dcc3ec1dd494c9a135e21c3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-j, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:37:36.558 INFO:journalctl@ceph.mds.j.smithi053.stdout:Apr 25 14:37:36 smithi053 ceph-mds[68008]: -- 172.21.15.53:0/852962124 <== mon.0 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f99afe9a40 con 0x55f99afbc800 2024-04-25T14:37:36.558 INFO:journalctl@ceph.mds.j.smithi053.stdout:Apr 25 14:37:36 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-j[68004]: starting mds.j at 2024-04-25T14:37:36.558 INFO:journalctl@ceph.mds.j.smithi053.stdout:Apr 25 14:37:36 smithi053 bash[67989]: 5baff6f1ec70e2890ea55bb2314c42a865b9c558dcc3ec1dd494c9a135e21c3f 2024-04-25T14:37:36.558 INFO:journalctl@ceph.mds.j.smithi053.stdout:Apr 25 14:37:36 smithi053 systemd[1]: Started Ceph mds.j for fd30e6ce-030f-11ef-bc93-c7b262605968. 2024-04-25T14:37:36.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:36 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/1213604997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:37:36.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:36 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:36.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:36 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:36.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:36 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:36.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:36 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:37:36.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:36 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:37:36.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:36 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:37:36.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:36 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/1213604997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:37:36.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:36 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:36.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:36 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:36.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:36 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:36.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:36 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:37:36.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:36 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:37:36.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:36 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:37:37.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:36 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1213604997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:37:37.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:36 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:37.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:36 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:37.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:36 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:37.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:36 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:37:37.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:36 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:37:37.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:36 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:37:37.694 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:37:37.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:37 smithi053 ceph-mon[27773]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:37.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:37 smithi053 ceph-mon[27773]: Deploying daemon mds.d on smithi053 2024-04-25T14:37:37.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:37 smithi053 ceph-mon[27773]: mds.? [v2:172.21.15.53:6834/506682943,v1:172.21.15.53:6835/506682943] up:boot 2024-04-25T14:37:37.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:37 smithi053 ceph-mon[27773]: fsmap 3 up:standby 2024-04-25T14:37:37.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:37 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T14:37:37.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:37.899+0000 7fac48400640 1 -- 172.21.15.53:0/2337235635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac401053a0 msgr2=0x7fac40105820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:37.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:37.899+0000 7fac48400640 1 --2- 172.21.15.53:0/2337235635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac401053a0 0x7fac40105820 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fac2c009900 tx=0x7fac2c02a3f0 comp rx=0 tx=0).stop 2024-04-25T14:37:37.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:37.899+0000 7fac48400640 1 -- 172.21.15.53:0/2337235635 shutdown_connections 2024-04-25T14:37:37.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:37.899+0000 7fac48400640 1 --2- 172.21.15.53:0/2337235635 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac40105d60 0x7fac4010c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:37.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:37.899+0000 7fac48400640 1 --2- 172.21.15.53:0/2337235635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac401053a0 0x7fac40105820 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:37.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:37.899+0000 7fac48400640 1 --2- 172.21.15.53:0/2337235635 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac401041b0 0x7fac401045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:37.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:37.899+0000 7fac48400640 1 -- 172.21.15.53:0/2337235635 >> 172.21.15.53:0/2337235635 conn(0x7fac400ff740 msgr2=0x7fac40101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:37:37.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:37.899+0000 7fac48400640 1 -- 172.21.15.53:0/2337235635 shutdown_connections 2024-04-25T14:37:37.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:37.899+0000 7fac48400640 1 -- 172.21.15.53:0/2337235635 wait complete. 2024-04-25T14:37:37.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:37.900+0000 7fac48400640 1 Processor -- start 2024-04-25T14:37:37.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:37.900+0000 7fac48400640 1 -- start start 2024-04-25T14:37:37.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:37.900+0000 7fac48400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac401041b0 0x7fac4019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:37.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:37.901+0000 7fac48400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac401053a0 0x7fac4019a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:37.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:37.901+0000 7fac48400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac40105d60 0x7fac401a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:37.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:37.901+0000 7fac48400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fac4010fd40 con 0x7fac401053a0 2024-04-25T14:37:37.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:37.901+0000 7fac48400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fac4010fbc0 con 0x7fac40105d60 2024-04-25T14:37:37.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:37.901+0000 7fac48400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fac4010fec0 con 0x7fac401041b0 2024-04-25T14:37:37.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:37.901+0000 7fac45000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac401053a0 0x7fac4019a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:37.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:37.901+0000 7fac45a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac401041b0 0x7fac4019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:37.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:37.901+0000 7fac45000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac401053a0 0x7fac4019a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56506/0 (socket says 172.21.15.53:56506) 2024-04-25T14:37:37.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:37.901+0000 7fac45000640 1 -- 172.21.15.53:0/3165036269 learned_addr learned my addr 172.21.15.53:0/3165036269 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:37:37.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:37.901+0000 7fac46400640 1 --2- 172.21.15.53:0/3165036269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac40105d60 0x7fac401a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:37.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:37.901+0000 7fac45a00640 1 -- 172.21.15.53:0/3165036269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac40105d60 msgr2=0x7fac401a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:37.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:37.901+0000 7fac45a00640 1 --2- 172.21.15.53:0/3165036269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac40105d60 0x7fac401a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:37.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:37.901+0000 7fac45a00640 1 -- 172.21.15.53:0/3165036269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac401053a0 msgr2=0x7fac4019a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:37.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:37.901+0000 7fac45a00640 1 --2- 172.21.15.53:0/3165036269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac401053a0 0x7fac4019a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:37.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:37.901+0000 7fac45a00640 1 -- 172.21.15.53:0/3165036269 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac401a2020 con 0x7fac401041b0 2024-04-25T14:37:37.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:37.901+0000 7fac45000640 1 --2- 172.21.15.53:0/3165036269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac401053a0 0x7fac4019a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:37:37.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:37.902+0000 7fac45a00640 1 --2- 172.21.15.53:0/3165036269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac401041b0 0x7fac4019a400 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fac34005fd0 tx=0x7fac3400eed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:37:37.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:37.902+0000 7fac3e000640 1 -- 172.21.15.53:0/3165036269 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3400cdb0 con 0x7fac401041b0 2024-04-25T14:37:37.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:37.902+0000 7fac3e000640 1 -- 172.21.15.53:0/3165036269 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac34003e20 con 0x7fac401041b0 2024-04-25T14:37:37.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:37.902+0000 7fac48400640 1 -- 172.21.15.53:0/3165036269 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac401a2310 con 0x7fac401041b0 2024-04-25T14:37:37.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:37.902+0000 7fac3e000640 1 -- 172.21.15.53:0/3165036269 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac34010690 con 0x7fac401041b0 2024-04-25T14:37:37.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:37.902+0000 7fac48400640 1 -- 172.21.15.53:0/3165036269 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac401a2860 con 0x7fac401041b0 2024-04-25T14:37:37.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:37.903+0000 7fac48400640 1 -- 172.21.15.53:0/3165036269 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac4010b550 con 0x7fac401041b0 2024-04-25T14:37:37.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:37.906+0000 7fac3e000640 1 -- 172.21.15.53:0/3165036269 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fac34004260 con 0x7fac401041b0 2024-04-25T14:37:37.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:37.908+0000 7fac3e000640 1 --2- 172.21.15.53:0/3165036269 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fac280a3c30 0x7fac280a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:37.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:37.908+0000 7fac3e000640 1 -- 172.21.15.53:0/3165036269 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fac340d06f0 con 0x7fac401041b0 2024-04-25T14:37:37.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:37.908+0000 7fac45000640 1 --2- 172.21.15.53:0/3165036269 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fac280a3c30 0x7fac280a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:37.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:37.909+0000 7fac45000640 1 --2- 172.21.15.53:0/3165036269 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fac280a3c30 0x7fac280a60f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fac4019b910 tx=0x7fac2c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:37:37.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:37.912+0000 7fac3e000640 1 -- 172.21.15.53:0/3165036269 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fac3409d550 con 0x7fac401041b0 2024-04-25T14:37:37.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:37 smithi183 ceph-mon[32584]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:37.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:37 smithi183 ceph-mon[32584]: Deploying daemon mds.d on smithi053 2024-04-25T14:37:37.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:37 smithi183 ceph-mon[32584]: mds.? [v2:172.21.15.53:6834/506682943,v1:172.21.15.53:6835/506682943] up:boot 2024-04-25T14:37:37.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:37 smithi183 ceph-mon[32584]: fsmap 3 up:standby 2024-04-25T14:37:37.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:37 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T14:37:37.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:37 smithi150 ceph-mon[32504]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:37.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:37 smithi150 ceph-mon[32504]: Deploying daemon mds.d on smithi053 2024-04-25T14:37:37.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:37 smithi150 ceph-mon[32504]: mds.? [v2:172.21.15.53:6834/506682943,v1:172.21.15.53:6835/506682943] up:boot 2024-04-25T14:37:37.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:37 smithi150 ceph-mon[32504]: fsmap 3 up:standby 2024-04-25T14:37:37.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:37 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T14:37:38.166 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:38.166+0000 7fac48400640 1 -- 172.21.15.53:0/3165036269 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac4002d5c0 con 0x7fac401041b0 2024-04-25T14:37:38.166 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:38.167+0000 7fac3e000640 1 -- 172.21.15.53:0/3165036269 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3040 (secure 0 0 0) 0x7fac3409ce50 con 0x7fac401041b0 2024-04-25T14:37:38.167 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:37:38.167 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2966992472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2966992472},{"type":"v1","addr":"172.21.15.150:6833","nonce":2966992472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/94143643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":94143643},{"type":"v1","addr":"172.21.15.183:6833","nonce":94143643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24470,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/506682943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":506682943},{"type":"v1","addr":"172.21.15.53:6835","nonce":506682943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[]} 2024-04-25T14:37:38.167 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 4 2024-04-25T14:37:38.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:38.169+0000 7fac48400640 1 -- 172.21.15.53:0/3165036269 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fac280a3c30 msgr2=0x7fac280a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:38.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:38.169+0000 7fac48400640 1 --2- 172.21.15.53:0/3165036269 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fac280a3c30 0x7fac280a60f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fac4019b910 tx=0x7fac2c005a20 comp rx=0 tx=0).stop 2024-04-25T14:37:38.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:38.169+0000 7fac48400640 1 -- 172.21.15.53:0/3165036269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac401041b0 msgr2=0x7fac4019a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:38.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:38.169+0000 7fac48400640 1 --2- 172.21.15.53:0/3165036269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac401041b0 0x7fac4019a400 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fac34005fd0 tx=0x7fac3400eed0 comp rx=0 tx=0).stop 2024-04-25T14:37:38.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:38.169+0000 7fac48400640 1 -- 172.21.15.53:0/3165036269 shutdown_connections 2024-04-25T14:37:38.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:38.169+0000 7fac48400640 1 --2- 172.21.15.53:0/3165036269 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fac280a3c30 0x7fac280a60f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:38.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:38.169+0000 7fac48400640 1 --2- 172.21.15.53:0/3165036269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac40105d60 0x7fac401a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:38.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:38.169+0000 7fac48400640 1 --2- 172.21.15.53:0/3165036269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac401053a0 0x7fac4019a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:38.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:38.169+0000 7fac48400640 1 --2- 172.21.15.53:0/3165036269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac401041b0 0x7fac4019a400 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:38.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:38.169+0000 7fac48400640 1 -- 172.21.15.53:0/3165036269 >> 172.21.15.53:0/3165036269 conn(0x7fac400ff740 msgr2=0x7fac40108720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:37:38.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:38.170+0000 7fac48400640 1 -- 172.21.15.53:0/3165036269 shutdown_connections 2024-04-25T14:37:38.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:38.170+0000 7fac48400640 1 -- 172.21.15.53:0/3165036269 wait complete. 2024-04-25T14:37:38.832 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:38 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/3165036269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:37:38.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:38 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/3165036269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:37:38.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:38 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/3165036269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:37:39.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:39 smithi053 ceph-mon[27773]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:39.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:39 smithi183 ceph-mon[32584]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:39.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:39 smithi150 ceph-mon[32504]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:40.136 INFO:journalctl@ceph.mds.d.smithi053.stdout:Apr 25 14:37:39 smithi053 podman[68326]: 2024-04-25 14:37:39.787844931 +0000 UTC m=+0.037557239 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T14:37:40.136 INFO:journalctl@ceph.mds.d.smithi053.stdout:Apr 25 14:37:40 smithi053 podman[68326]: 2024-04-25 14:37:40.13503192 +0000 UTC m=+0.384744253 container init a35a0e9c6b5135a100009a093b2a1efc541886c775b2ff5c3ffc3aee4a8f5850 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-d, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-25T14:37:40.190 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:37:40.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.388+0000 7f84f4800640 1 -- 172.21.15.53:0/1079802992 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84ec1053c0 msgr2=0x7f84ec105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:40.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.388+0000 7f84f4800640 1 --2- 172.21.15.53:0/1079802992 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84ec1053c0 0x7f84ec105840 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f84dc009900 tx=0x7f84dc02a3f0 comp rx=0 tx=0).stop 2024-04-25T14:37:40.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.388+0000 7f84f4800640 1 -- 172.21.15.53:0/1079802992 shutdown_connections 2024-04-25T14:37:40.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.388+0000 7f84f4800640 1 --2- 172.21.15.53:0/1079802992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84ec105d80 0x7f84ec10c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:40.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.388+0000 7f84f4800640 1 --2- 172.21.15.53:0/1079802992 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84ec1053c0 0x7f84ec105840 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:40.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.388+0000 7f84f4800640 1 --2- 172.21.15.53:0/1079802992 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84ec1041d0 0x7f84ec1045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:40.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.388+0000 7f84f4800640 1 -- 172.21.15.53:0/1079802992 >> 172.21.15.53:0/1079802992 conn(0x7f84ec0ff740 msgr2=0x7f84ec101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:37:40.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.388+0000 7f84f4800640 1 -- 172.21.15.53:0/1079802992 shutdown_connections 2024-04-25T14:37:40.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.388+0000 7f84f4800640 1 -- 172.21.15.53:0/1079802992 wait complete. 2024-04-25T14:37:40.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.389+0000 7f84f4800640 1 Processor -- start 2024-04-25T14:37:40.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.389+0000 7f84f4800640 1 -- start start 2024-04-25T14:37:40.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.389+0000 7f84f4800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84ec1041d0 0x7f84ec19a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:40.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.390+0000 7f84f4800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84ec1053c0 0x7f84ec19a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:40.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.390+0000 7f84f4800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84ec105d80 0x7f84ec1a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:40.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.390+0000 7f84f4800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f84ec10fd60 con 0x7f84ec1041d0 2024-04-25T14:37:40.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.390+0000 7f84f4800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f84ec10fbe0 con 0x7f84ec1053c0 2024-04-25T14:37:40.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.390+0000 7f84f4800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f84ec10fee0 con 0x7f84ec105d80 2024-04-25T14:37:40.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.390+0000 7f84f1e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84ec1041d0 0x7f84ec19a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:40.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.390+0000 7f84f1e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84ec1041d0 0x7f84ec19a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56558/0 (socket says 172.21.15.53:56558) 2024-04-25T14:37:40.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.390+0000 7f84f1e00640 1 -- 172.21.15.53:0/3982404699 learned_addr learned my addr 172.21.15.53:0/3982404699 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:37:40.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.390+0000 7f84f1400640 1 --2- 172.21.15.53:0/3982404699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84ec1053c0 0x7f84ec19a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:40.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.390+0000 7f84f2800640 1 --2- 172.21.15.53:0/3982404699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84ec105d80 0x7f84ec1a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:40.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.390+0000 7f84f1e00640 1 -- 172.21.15.53:0/3982404699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84ec105d80 msgr2=0x7f84ec1a19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:40.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.390+0000 7f84f1e00640 1 --2- 172.21.15.53:0/3982404699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84ec105d80 0x7f84ec1a19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:40.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.390+0000 7f84f1e00640 1 -- 172.21.15.53:0/3982404699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84ec1053c0 msgr2=0x7f84ec19a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:40.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.390+0000 7f84f1e00640 1 --2- 172.21.15.53:0/3982404699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84ec1053c0 0x7f84ec19a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:40.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.390+0000 7f84f1e00640 1 -- 172.21.15.53:0/3982404699 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84ec1a2030 con 0x7f84ec1041d0 2024-04-25T14:37:40.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.390+0000 7f84f1400640 1 --2- 172.21.15.53:0/3982404699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84ec1053c0 0x7f84ec19a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:37:40.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.391+0000 7f84f1e00640 1 --2- 172.21.15.53:0/3982404699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84ec1041d0 0x7f84ec19a410 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f84e400eb70 tx=0x7f84e400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:37:40.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.391+0000 7f84e2a00640 1 -- 172.21.15.53:0/3982404699 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e4003970 con 0x7f84ec1041d0 2024-04-25T14:37:40.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.391+0000 7f84f4800640 1 -- 172.21.15.53:0/3982404699 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84ec1a2320 con 0x7f84ec1041d0 2024-04-25T14:37:40.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.391+0000 7f84e2a00640 1 -- 172.21.15.53:0/3982404699 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84e4004020 con 0x7f84ec1041d0 2024-04-25T14:37:40.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.391+0000 7f84e2a00640 1 -- 172.21.15.53:0/3982404699 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e4010690 con 0x7f84ec1041d0 2024-04-25T14:37:40.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.391+0000 7f84f4800640 1 -- 172.21.15.53:0/3982404699 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84ec1a2840 con 0x7f84ec1041d0 2024-04-25T14:37:40.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.392+0000 7f84f4800640 1 -- 172.21.15.53:0/3982404699 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84ec10b570 con 0x7f84ec1041d0 2024-04-25T14:37:40.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.394+0000 7f84e2a00640 1 -- 172.21.15.53:0/3982404699 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f84e40044d0 con 0x7f84ec1041d0 2024-04-25T14:37:40.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.396+0000 7f84e2a00640 1 --2- 172.21.15.53:0/3982404699 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f84d80a3bb0 0x7f84d80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:40.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.396+0000 7f84e2a00640 1 -- 172.21.15.53:0/3982404699 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f84e40d06e0 con 0x7f84ec1041d0 2024-04-25T14:37:40.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.396+0000 7f84f1400640 1 --2- 172.21.15.53:0/3982404699 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f84d80a3bb0 0x7f84d80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:40.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.397+0000 7f84f1400640 1 --2- 172.21.15.53:0/3982404699 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f84d80a3bb0 0x7f84d80a6070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f84dc031500 tx=0x7f84dc005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:37:40.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.401+0000 7f84e2a00640 1 -- 172.21.15.53:0/3982404699 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f84e409d4c0 con 0x7f84ec1041d0 2024-04-25T14:37:40.533 INFO:journalctl@ceph.mds.d.smithi053.stdout:Apr 25 14:37:40 smithi053 podman[68326]: 2024-04-25 14:37:40.146395251 +0000 UTC m=+0.396107548 container start a35a0e9c6b5135a100009a093b2a1efc541886c775b2ff5c3ffc3aee4a8f5850 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T14:37:40.533 INFO:journalctl@ceph.mds.d.smithi053.stdout:Apr 25 14:37:40 smithi053 ceph-mds[68344]: -- 172.21.15.53:0/269786422 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55be19ab5a40 con 0x55be19a88c00 2024-04-25T14:37:40.533 INFO:journalctl@ceph.mds.d.smithi053.stdout:Apr 25 14:37:40 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-d[68340]: starting mds.d at 2024-04-25T14:37:40.533 INFO:journalctl@ceph.mds.d.smithi053.stdout:Apr 25 14:37:40 smithi053 bash[68326]: a35a0e9c6b5135a100009a093b2a1efc541886c775b2ff5c3ffc3aee4a8f5850 2024-04-25T14:37:40.533 INFO:journalctl@ceph.mds.d.smithi053.stdout:Apr 25 14:37:40 smithi053 systemd[1]: Started Ceph mds.d for fd30e6ce-030f-11ef-bc93-c7b262605968. 2024-04-25T14:37:40.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.652+0000 7f84f4800640 1 -- 172.21.15.53:0/3982404699 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84ec02d5c0 con 0x7f84ec1041d0 2024-04-25T14:37:40.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.653+0000 7f84e2a00640 1 -- 172.21.15.53:0/3982404699 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3865 (secure 0 0 0) 0x7f84ec02d5c0 con 0x7f84ec1041d0 2024-04-25T14:37:40.653 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:37:40.653 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2966992472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2966992472},{"type":"v1","addr":"172.21.15.150:6833","nonce":2966992472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/94143643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":94143643},{"type":"v1","addr":"172.21.15.183:6833","nonce":94143643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24470,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/506682943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":506682943},{"type":"v1","addr":"172.21.15.53:6835","nonce":506682943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24502,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3786232682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3786232682},{"type":"v1","addr":"172.21.15.53:6837","nonce":3786232682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[]} 2024-04-25T14:37:40.653 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 5 2024-04-25T14:37:40.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.655+0000 7f84f4800640 1 -- 172.21.15.53:0/3982404699 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f84d80a3bb0 msgr2=0x7f84d80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:40.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.655+0000 7f84f4800640 1 --2- 172.21.15.53:0/3982404699 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f84d80a3bb0 0x7f84d80a6070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f84dc031500 tx=0x7f84dc005a20 comp rx=0 tx=0).stop 2024-04-25T14:37:40.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.655+0000 7f84f4800640 1 -- 172.21.15.53:0/3982404699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84ec1041d0 msgr2=0x7f84ec19a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:40.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.655+0000 7f84f4800640 1 --2- 172.21.15.53:0/3982404699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84ec1041d0 0x7f84ec19a410 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f84e400eb70 tx=0x7f84e400ef70 comp rx=0 tx=0).stop 2024-04-25T14:37:40.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.656+0000 7f84f4800640 1 -- 172.21.15.53:0/3982404699 shutdown_connections 2024-04-25T14:37:40.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.656+0000 7f84f4800640 1 --2- 172.21.15.53:0/3982404699 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f84d80a3bb0 0x7f84d80a6070 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:40.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.656+0000 7f84f4800640 1 --2- 172.21.15.53:0/3982404699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84ec105d80 0x7f84ec1a19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:40.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.656+0000 7f84f4800640 1 --2- 172.21.15.53:0/3982404699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84ec1053c0 0x7f84ec19a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:40.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.656+0000 7f84f4800640 1 --2- 172.21.15.53:0/3982404699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84ec1041d0 0x7f84ec19a410 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:40.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.656+0000 7f84f4800640 1 -- 172.21.15.53:0/3982404699 >> 172.21.15.53:0/3982404699 conn(0x7f84ec0ff740 msgr2=0x7f84ec108740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:37:40.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.656+0000 7f84f4800640 1 -- 172.21.15.53:0/3982404699 shutdown_connections 2024-04-25T14:37:40.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:40.656+0000 7f84f4800640 1 -- 172.21.15.53:0/3982404699 wait complete. 2024-04-25T14:37:41.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:41 smithi183 ceph-mon[32584]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:41.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:41 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:41.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:41 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:41.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:41 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:41.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:41 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:37:41.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:41 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:37:41.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:41 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:37:41.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:41 smithi183 ceph-mon[32584]: Deploying daemon mds.e on smithi150 2024-04-25T14:37:41.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:41 smithi183 ceph-mon[32584]: mds.? [v2:172.21.15.53:6836/3786232682,v1:172.21.15.53:6837/3786232682] up:boot 2024-04-25T14:37:41.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:41 smithi183 ceph-mon[32584]: fsmap 4 up:standby 2024-04-25T14:37:41.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:41 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T14:37:41.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:41 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/3982404699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:37:41.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:41 smithi150 ceph-mon[32504]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:41.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:41 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:41.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:41 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:41.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:41 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:41.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:41 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:37:41.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:41 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:37:41.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:41 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:37:41.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:41 smithi150 ceph-mon[32504]: Deploying daemon mds.e on smithi150 2024-04-25T14:37:41.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:41 smithi150 ceph-mon[32504]: mds.? [v2:172.21.15.53:6836/3786232682,v1:172.21.15.53:6837/3786232682] up:boot 2024-04-25T14:37:41.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:41 smithi150 ceph-mon[32504]: fsmap 4 up:standby 2024-04-25T14:37:41.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:41 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T14:37:41.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:41 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/3982404699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:37:41.711 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:41 smithi053 ceph-mon[27773]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:41.711 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:41 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:41.711 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:41 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:41.711 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:41 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:41.711 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:41 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:37:41.711 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:41 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:37:41.712 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:41 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:37:41.712 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:41 smithi053 ceph-mon[27773]: Deploying daemon mds.e on smithi150 2024-04-25T14:37:41.712 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:41 smithi053 ceph-mon[27773]: mds.? [v2:172.21.15.53:6836/3786232682,v1:172.21.15.53:6837/3786232682] up:boot 2024-04-25T14:37:41.712 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:41 smithi053 ceph-mon[27773]: fsmap 4 up:standby 2024-04-25T14:37:41.712 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:41 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T14:37:41.712 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:41 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/3982404699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:37:42.675 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:37:42.710 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 25 14:37:42 smithi150 systemd[1]: Starting Ceph mds.e for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:37:42.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:42.872+0000 7ff005800640 1 -- 172.21.15.53:0/3280933709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff000107ab0 msgr2=0x7ff000109f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:42.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:42.872+0000 7ff005800640 1 --2- 172.21.15.53:0/3280933709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff000107ab0 0x7ff000109f00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7feff8009ca0 tx=0x7feff802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:37:42.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:42.872+0000 7ff005800640 1 -- 172.21.15.53:0/3280933709 shutdown_connections 2024-04-25T14:37:42.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:42.872+0000 7ff005800640 1 --2- 172.21.15.53:0/3280933709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff000107ab0 0x7ff000109f00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:42.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:42.872+0000 7ff005800640 1 --2- 172.21.15.53:0/3280933709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff000105180 0x7ff000107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:42.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:42.872+0000 7ff005800640 1 --2- 172.21.15.53:0/3280933709 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff000102850 0x7ff000104c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:42.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:42.872+0000 7ff005800640 1 -- 172.21.15.53:0/3280933709 >> 172.21.15.53:0/3280933709 conn(0x7ff0000fc220 msgr2=0x7ff0000fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:37:42.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:42.872+0000 7ff005800640 1 -- 172.21.15.53:0/3280933709 shutdown_connections 2024-04-25T14:37:42.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:42.872+0000 7ff005800640 1 -- 172.21.15.53:0/3280933709 wait complete. 2024-04-25T14:37:42.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:42.873+0000 7ff005800640 1 Processor -- start 2024-04-25T14:37:42.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:42.873+0000 7ff005800640 1 -- start start 2024-04-25T14:37:42.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:42.873+0000 7ff005800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff000102850 0x7ff00019a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:42.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:42.873+0000 7ff005800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff000105180 0x7ff00019a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:42.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:42.873+0000 7ff005800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff000107ab0 0x7ff0001a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:42.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:42.873+0000 7ff005800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff00010d950 con 0x7ff000102850 2024-04-25T14:37:42.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:42.873+0000 7ff005800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff00010d7d0 con 0x7ff000107ab0 2024-04-25T14:37:42.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:42.873+0000 7ff005800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff00010dad0 con 0x7ff000105180 2024-04-25T14:37:42.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:42.874+0000 7fefffe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff000102850 0x7ff00019a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:42.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:42.874+0000 7fefffe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff000102850 0x7ff00019a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38806/0 (socket says 172.21.15.53:38806) 2024-04-25T14:37:42.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:42.874+0000 7fefffe00640 1 -- 172.21.15.53:0/2178093943 learned_addr learned my addr 172.21.15.53:0/2178093943 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:37:42.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:42.874+0000 7fefff400640 1 --2- 172.21.15.53:0/2178093943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff000105180 0x7ff00019a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:42.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:42.874+0000 7ff004e00640 1 --2- 172.21.15.53:0/2178093943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff000107ab0 0x7ff0001a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:42.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:42.874+0000 7fefffe00640 1 -- 172.21.15.53:0/2178093943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff000105180 msgr2=0x7ff00019a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:42.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:42.874+0000 7fefffe00640 1 --2- 172.21.15.53:0/2178093943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff000105180 0x7ff00019a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:42.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:42.874+0000 7fefffe00640 1 -- 172.21.15.53:0/2178093943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff000107ab0 msgr2=0x7ff0001a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:42.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:42.874+0000 7fefffe00640 1 --2- 172.21.15.53:0/2178093943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff000107ab0 0x7ff0001a1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:42.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:42.874+0000 7fefffe00640 1 -- 172.21.15.53:0/2178093943 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0001a1e00 con 0x7ff000102850 2024-04-25T14:37:42.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:42.874+0000 7fefffe00640 1 --2- 172.21.15.53:0/2178093943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff000102850 0x7ff00019a1e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7feff000ea60 tx=0x7feff000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:37:42.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:42.874+0000 7feffcc00640 1 -- 172.21.15.53:0/2178093943 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff000cde0 con 0x7ff000102850 2024-04-25T14:37:42.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:42.875+0000 7feffcc00640 1 -- 172.21.15.53:0/2178093943 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feff00035f0 con 0x7ff000102850 2024-04-25T14:37:42.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:42.875+0000 7ff005800640 1 -- 172.21.15.53:0/2178093943 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0001a20f0 con 0x7ff000102850 2024-04-25T14:37:42.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:42.875+0000 7feffcc00640 1 -- 172.21.15.53:0/2178093943 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff0010690 con 0x7ff000102850 2024-04-25T14:37:42.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:42.875+0000 7ff005800640 1 -- 172.21.15.53:0/2178093943 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0001a2640 con 0x7ff000102850 2024-04-25T14:37:42.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:42.876+0000 7ff005800640 1 -- 172.21.15.53:0/2178093943 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0001095a0 con 0x7ff000102850 2024-04-25T14:37:42.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:42.878+0000 7feffcc00640 1 -- 172.21.15.53:0/2178093943 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7feff00037a0 con 0x7ff000102850 2024-04-25T14:37:42.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:42.879+0000 7feffcc00640 1 --2- 172.21.15.53:0/2178093943 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fefe40a3bb0 0x7fefe40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:42.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:42.879+0000 7feffcc00640 1 -- 172.21.15.53:0/2178093943 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7feff00d0340 con 0x7ff000102850 2024-04-25T14:37:42.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:42.879+0000 7fefff400640 1 --2- 172.21.15.53:0/2178093943 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fefe40a3bb0 0x7fefe40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:42.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:42.880+0000 7fefff400640 1 --2- 172.21.15.53:0/2178093943 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fefe40a3bb0 0x7fefe40a6070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff00019b6f0 tx=0x7fefec00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:37:42.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:42.884+0000 7feffcc00640 1 -- 172.21.15.53:0/2178093943 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feff009d120 con 0x7ff000102850 2024-04-25T14:37:43.099 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 25 14:37:42 smithi150 podman[62263]: 2024-04-25 14:37:42.665728001 +0000 UTC m=+0.018081875 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T14:37:43.099 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 25 14:37:42 smithi150 podman[62263]: 2024-04-25 14:37:42.788587712 +0000 UTC m=+0.140941631 container create b270a6af510a8e330d1804de028d11a270fe4b8e18f35332c793cfd4a69e2f83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-e, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:37:43.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:43.131+0000 7ff005800640 1 -- 172.21.15.53:0/2178093943 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff00002d5c0 con 0x7ff000102850 2024-04-25T14:37:43.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:43.132+0000 7feffcc00640 1 -- 172.21.15.53:0/2178093943 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3865 (secure 0 0 0) 0x7feff009ca20 con 0x7ff000102850 2024-04-25T14:37:43.132 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:37:43.132 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2966992472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2966992472},{"type":"v1","addr":"172.21.15.150:6833","nonce":2966992472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/94143643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":94143643},{"type":"v1","addr":"172.21.15.183:6833","nonce":94143643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24470,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/506682943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":506682943},{"type":"v1","addr":"172.21.15.53:6835","nonce":506682943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24502,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3786232682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3786232682},{"type":"v1","addr":"172.21.15.53:6837","nonce":3786232682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[]} 2024-04-25T14:37:43.132 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 5 2024-04-25T14:37:43.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:43.135+0000 7ff005800640 1 -- 172.21.15.53:0/2178093943 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fefe40a3bb0 msgr2=0x7fefe40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:43.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:43.135+0000 7ff005800640 1 --2- 172.21.15.53:0/2178093943 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fefe40a3bb0 0x7fefe40a6070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff00019b6f0 tx=0x7fefec00b390 comp rx=0 tx=0).stop 2024-04-25T14:37:43.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:43.135+0000 7ff005800640 1 -- 172.21.15.53:0/2178093943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff000102850 msgr2=0x7ff00019a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:43.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:43.135+0000 7ff005800640 1 --2- 172.21.15.53:0/2178093943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff000102850 0x7ff00019a1e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7feff000ea60 tx=0x7feff000ef20 comp rx=0 tx=0).stop 2024-04-25T14:37:43.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:43.135+0000 7ff005800640 1 -- 172.21.15.53:0/2178093943 shutdown_connections 2024-04-25T14:37:43.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:43.135+0000 7ff005800640 1 --2- 172.21.15.53:0/2178093943 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fefe40a3bb0 0x7fefe40a6070 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:43.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:43.135+0000 7ff005800640 1 --2- 172.21.15.53:0/2178093943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff000107ab0 0x7ff0001a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:43.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:43.135+0000 7ff005800640 1 --2- 172.21.15.53:0/2178093943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff000105180 0x7ff00019a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:43.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:43.135+0000 7ff005800640 1 --2- 172.21.15.53:0/2178093943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff000102850 0x7ff00019a1e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:43.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:43.135+0000 7ff005800640 1 -- 172.21.15.53:0/2178093943 >> 172.21.15.53:0/2178093943 conn(0x7ff0000fc220 msgr2=0x7ff000101fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:37:43.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:43.135+0000 7ff005800640 1 -- 172.21.15.53:0/2178093943 shutdown_connections 2024-04-25T14:37:43.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:43.135+0000 7ff005800640 1 -- 172.21.15.53:0/2178093943 wait complete. 2024-04-25T14:37:43.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:43 smithi150 ceph-mon[32504]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:43.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:43 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/2178093943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:37:43.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:43 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:43.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:43 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:43.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:43 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:43.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:43 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:37:43.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:43 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:37:43.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:43 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:37:43.461 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 25 14:37:43 smithi150 podman[62263]: 2024-04-25 14:37:43.099380578 +0000 UTC m=+0.451734439 container init b270a6af510a8e330d1804de028d11a270fe4b8e18f35332c793cfd4a69e2f83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-e, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-25T14:37:43.461 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 25 14:37:43 smithi150 podman[62263]: 2024-04-25 14:37:43.10266752 +0000 UTC m=+0.455021384 container start b270a6af510a8e330d1804de028d11a270fe4b8e18f35332c793cfd4a69e2f83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-e, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T14:37:43.461 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 25 14:37:43 smithi150 ceph-mds[62283]: -- 172.21.15.150:0/2109294177 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a3ea49ba40 con 0x55a3ea46f000 2024-04-25T14:37:43.461 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 25 14:37:43 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-e[62279]: starting mds.e at 2024-04-25T14:37:43.461 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 25 14:37:43 smithi150 bash[62263]: b270a6af510a8e330d1804de028d11a270fe4b8e18f35332c793cfd4a69e2f83 2024-04-25T14:37:43.461 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 25 14:37:43 smithi150 systemd[1]: Started Ceph mds.e for fd30e6ce-030f-11ef-bc93-c7b262605968. 2024-04-25T14:37:43.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:43 smithi053 ceph-mon[27773]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:43.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:43 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/2178093943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:37:43.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:43 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:43.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:43 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:43.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:43 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:43.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:43 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:37:43.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:43 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:37:43.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:43 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:37:43.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:43 smithi183 ceph-mon[32584]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:43.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:43 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/2178093943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:37:43.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:43 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:43.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:43 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:43.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:43 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:43.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:43 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:37:43.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:43 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:37:43.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:43 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:37:44.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:44 smithi183 ceph-mon[32584]: Deploying daemon mds.f on smithi183 2024-04-25T14:37:44.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:44 smithi183 ceph-mon[32584]: mds.? [v2:172.21.15.150:6834/1369488844,v1:172.21.15.150:6835/1369488844] up:boot 2024-04-25T14:37:44.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:44 smithi183 ceph-mon[32584]: fsmap 5 up:standby 2024-04-25T14:37:44.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:44 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T14:37:44.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:44 smithi150 ceph-mon[32504]: Deploying daemon mds.f on smithi183 2024-04-25T14:37:44.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:44 smithi150 ceph-mon[32504]: mds.? [v2:172.21.15.150:6834/1369488844,v1:172.21.15.150:6835/1369488844] up:boot 2024-04-25T14:37:44.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:44 smithi150 ceph-mon[32504]: fsmap 5 up:standby 2024-04-25T14:37:44.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:44 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T14:37:44.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:44 smithi053 ceph-mon[27773]: Deploying daemon mds.f on smithi183 2024-04-25T14:37:44.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:44 smithi053 ceph-mon[27773]: mds.? [v2:172.21.15.150:6834/1369488844,v1:172.21.15.150:6835/1369488844] up:boot 2024-04-25T14:37:44.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:44 smithi053 ceph-mon[27773]: fsmap 5 up:standby 2024-04-25T14:37:44.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:44 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T14:37:45.153 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:37:45.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.379+0000 7f1f1b000640 1 -- 172.21.15.53:0/3430687702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f14109f80 msgr2=0x7f1f14110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:45.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.379+0000 7f1f1b000640 1 --2- 172.21.15.53:0/3430687702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f14109f80 0x7f1f14110810 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c009ca0 tx=0x7f1f0c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:37:45.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.380+0000 7f1f1b000640 1 -- 172.21.15.53:0/3430687702 shutdown_connections 2024-04-25T14:37:45.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.380+0000 7f1f1b000640 1 --2- 172.21.15.53:0/3430687702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f14109f80 0x7f1f14110810 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:45.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.380+0000 7f1f1b000640 1 --2- 172.21.15.53:0/3430687702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f141095c0 0x7f1f14109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:45.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.380+0000 7f1f1b000640 1 --2- 172.21.15.53:0/3430687702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f141083d0 0x7f1f141087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:45.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.380+0000 7f1f1b000640 1 -- 172.21.15.53:0/3430687702 >> 172.21.15.53:0/3430687702 conn(0x7f1f14077390 msgr2=0x7f1f140797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:37:45.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.380+0000 7f1f1b000640 1 -- 172.21.15.53:0/3430687702 shutdown_connections 2024-04-25T14:37:45.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.380+0000 7f1f1b000640 1 -- 172.21.15.53:0/3430687702 wait complete. 2024-04-25T14:37:45.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.381+0000 7f1f1b000640 1 Processor -- start 2024-04-25T14:37:45.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.381+0000 7f1f1b000640 1 -- start start 2024-04-25T14:37:45.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.381+0000 7f1f1b000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f141083d0 0x7f1f1419e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:45.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.381+0000 7f1f1b000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f141095c0 0x7f1f1419eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:45.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.381+0000 7f1f1b000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f14109f80 0x7f1f141a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:45.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.381+0000 7f1f1b000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1f14113f10 con 0x7f1f14109f80 2024-04-25T14:37:45.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.381+0000 7f1f1b000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1f14113d90 con 0x7f1f141083d0 2024-04-25T14:37:45.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.381+0000 7f1f1b000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1f14114090 con 0x7f1f141095c0 2024-04-25T14:37:45.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.382+0000 7f1f19200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f141095c0 0x7f1f1419eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:45.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.382+0000 7f1f19200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f141095c0 0x7f1f1419eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:38760/0 (socket says 172.21.15.53:38760) 2024-04-25T14:37:45.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.382+0000 7f1f19200640 1 -- 172.21.15.53:0/3955178757 learned_addr learned my addr 172.21.15.53:0/3955178757 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:37:45.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.382+0000 7f1f1a600640 1 --2- 172.21.15.53:0/3955178757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f14109f80 0x7f1f141a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:45.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.382+0000 7f1f19c00640 1 --2- 172.21.15.53:0/3955178757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f141083d0 0x7f1f1419e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:45.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.382+0000 7f1f19200640 1 -- 172.21.15.53:0/3955178757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f141083d0 msgr2=0x7f1f1419e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:45.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.382+0000 7f1f19200640 1 --2- 172.21.15.53:0/3955178757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f141083d0 0x7f1f1419e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:45.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.382+0000 7f1f19200640 1 -- 172.21.15.53:0/3955178757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f14109f80 msgr2=0x7f1f141a5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:45.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.382+0000 7f1f19200640 1 --2- 172.21.15.53:0/3955178757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f14109f80 0x7f1f141a5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:45.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.382+0000 7f1f19200640 1 -- 172.21.15.53:0/3955178757 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f141a61d0 con 0x7f1f141095c0 2024-04-25T14:37:45.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.382+0000 7f1f19200640 1 --2- 172.21.15.53:0/3955178757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f141095c0 0x7f1f1419eaf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1f0000e9f0 tx=0x7f1f0000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:37:45.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.383+0000 7f1f12000640 1 -- 172.21.15.53:0/3955178757 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0000cd80 con 0x7f1f141095c0 2024-04-25T14:37:45.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.383+0000 7f1f12000640 1 -- 172.21.15.53:0/3955178757 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f000054d0 con 0x7f1f141095c0 2024-04-25T14:37:45.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.383+0000 7f1f12000640 1 -- 172.21.15.53:0/3955178757 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f00010690 con 0x7f1f141095c0 2024-04-25T14:37:45.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.383+0000 7f1f1b000640 1 -- 172.21.15.53:0/3955178757 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f141a64c0 con 0x7f1f141095c0 2024-04-25T14:37:45.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.383+0000 7f1f1b000640 1 -- 172.21.15.53:0/3955178757 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f141a69c0 con 0x7f1f141095c0 2024-04-25T14:37:45.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.384+0000 7f1f1b000640 1 -- 172.21.15.53:0/3955178757 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f1410f7c0 con 0x7f1f141095c0 2024-04-25T14:37:45.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.387+0000 7f1f12000640 1 -- 172.21.15.53:0/3955178757 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1f00004e90 con 0x7f1f141095c0 2024-04-25T14:37:45.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.388+0000 7f1f12000640 1 --2- 172.21.15.53:0/3955178757 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f1efc0a3c30 0x7f1efc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:45.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.388+0000 7f1f12000640 1 -- 172.21.15.53:0/3955178757 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f1f000d0390 con 0x7f1f141095c0 2024-04-25T14:37:45.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.388+0000 7f1f19c00640 1 --2- 172.21.15.53:0/3955178757 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f1efc0a3c30 0x7f1efc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:45.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.389+0000 7f1f19c00640 1 --2- 172.21.15.53:0/3955178757 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f1efc0a3c30 0x7f1efc0a60f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1f080095f0 tx=0x7f1f08005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:37:45.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.389+0000 7f1f12000640 1 -- 172.21.15.53:0/3955178757 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1f0009d170 con 0x7f1f141095c0 2024-04-25T14:37:45.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:45 smithi053 ceph-mon[27773]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:45.636 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.637+0000 7f1f1b000640 1 -- 172.21.15.53:0/3955178757 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f1402d810 con 0x7f1f141095c0 2024-04-25T14:37:45.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.638+0000 7f1f12000640 1 -- 172.21.15.53:0/3955178757 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4693 (secure 0 0 0) 0x7f1f0009ca70 con 0x7f1f141095c0 2024-04-25T14:37:45.638 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:37:45.638 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2966992472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2966992472},{"type":"v1","addr":"172.21.15.150:6833","nonce":2966992472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/94143643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":94143643},{"type":"v1","addr":"172.21.15.183:6833","nonce":94143643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24470,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/506682943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":506682943},{"type":"v1","addr":"172.21.15.53:6835","nonce":506682943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1369488844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1369488844},{"type":"v1","addr":"172.21.15.150:6835","nonce":1369488844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3786232682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3786232682},{"type":"v1","addr":"172.21.15.53:6837","nonce":3786232682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[]} 2024-04-25T14:37:45.638 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 6 2024-04-25T14:37:45.640 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.640+0000 7f1f1b000640 1 -- 172.21.15.53:0/3955178757 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f1efc0a3c30 msgr2=0x7f1efc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:45.640 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.640+0000 7f1f1b000640 1 --2- 172.21.15.53:0/3955178757 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f1efc0a3c30 0x7f1efc0a60f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1f080095f0 tx=0x7f1f08005f90 comp rx=0 tx=0).stop 2024-04-25T14:37:45.640 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.640+0000 7f1f1b000640 1 -- 172.21.15.53:0/3955178757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f141095c0 msgr2=0x7f1f1419eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:45.640 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.640+0000 7f1f1b000640 1 --2- 172.21.15.53:0/3955178757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f141095c0 0x7f1f1419eaf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1f0000e9f0 tx=0x7f1f0000edf0 comp rx=0 tx=0).stop 2024-04-25T14:37:45.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.640+0000 7f1f1b000640 1 -- 172.21.15.53:0/3955178757 shutdown_connections 2024-04-25T14:37:45.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.640+0000 7f1f1b000640 1 --2- 172.21.15.53:0/3955178757 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f1efc0a3c30 0x7f1efc0a60f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:45.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.640+0000 7f1f1b000640 1 --2- 172.21.15.53:0/3955178757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f14109f80 0x7f1f141a5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:45.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.640+0000 7f1f1b000640 1 --2- 172.21.15.53:0/3955178757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f141095c0 0x7f1f1419eaf0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:45.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.640+0000 7f1f1b000640 1 --2- 172.21.15.53:0/3955178757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f141083d0 0x7f1f1419e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:45.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.640+0000 7f1f1b000640 1 -- 172.21.15.53:0/3955178757 >> 172.21.15.53:0/3955178757 conn(0x7f1f14077390 msgr2=0x7f1f1410c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:37:45.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.641+0000 7f1f1b000640 1 -- 172.21.15.53:0/3955178757 shutdown_connections 2024-04-25T14:37:45.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:45.641+0000 7f1f1b000640 1 -- 172.21.15.53:0/3955178757 wait complete. 2024-04-25T14:37:45.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:45 smithi183 ceph-mon[32584]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:45.704 INFO:journalctl@ceph.mds.f.smithi183.stdout:Apr 25 14:37:45 smithi183 systemd[1]: Starting Ceph mds.f for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:37:45.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:45 smithi150 ceph-mon[32504]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:45.992 INFO:journalctl@ceph.mds.f.smithi183.stdout:Apr 25 14:37:45 smithi183 podman[62268]: 2024-04-25 14:37:45.629122131 +0000 UTC m=+0.017983728 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T14:37:45.992 INFO:journalctl@ceph.mds.f.smithi183.stdout:Apr 25 14:37:45 smithi183 podman[62268]: 2024-04-25 14:37:45.76531824 +0000 UTC m=+0.154179828 container create 63624d8b533c07f75ba4a08835acc8c271e949ff7a0ecab5598466198f4089ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-25T14:37:45.992 INFO:journalctl@ceph.mds.f.smithi183.stdout:Apr 25 14:37:45 smithi183 podman[62268]: 2024-04-25 14:37:45.967668445 +0000 UTC m=+0.356530030 container init 63624d8b533c07f75ba4a08835acc8c271e949ff7a0ecab5598466198f4089ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS) 2024-04-25T14:37:45.993 INFO:journalctl@ceph.mds.f.smithi183.stdout:Apr 25 14:37:45 smithi183 podman[62268]: 2024-04-25 14:37:45.970146385 +0000 UTC m=+0.359007970 container start 63624d8b533c07f75ba4a08835acc8c271e949ff7a0ecab5598466198f4089ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-f, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:37:46.345 INFO:journalctl@ceph.mds.f.smithi183.stdout:Apr 25 14:37:45 smithi183 ceph-mds[62287]: -- 172.21.15.183:0/2338768839 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d90fddfa40 con 0x55d90fdb2c00 2024-04-25T14:37:46.346 INFO:journalctl@ceph.mds.f.smithi183.stdout:Apr 25 14:37:45 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-f[62283]: starting mds.f at 2024-04-25T14:37:46.346 INFO:journalctl@ceph.mds.f.smithi183.stdout:Apr 25 14:37:46 smithi183 bash[62268]: 63624d8b533c07f75ba4a08835acc8c271e949ff7a0ecab5598466198f4089ad 2024-04-25T14:37:46.346 INFO:journalctl@ceph.mds.f.smithi183.stdout:Apr 25 14:37:46 smithi183 systemd[1]: Started Ceph mds.f for fd30e6ce-030f-11ef-bc93-c7b262605968. 2024-04-25T14:37:46.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:46 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/3955178757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:37:46.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:46 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:46.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:46 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:46.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:46 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:46.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:46 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:37:46.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:46 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:37:46.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:46 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:37:46.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:46 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/3955178757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:37:46.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:46 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:46.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:46 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:46.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:46 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:46.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:46 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:37:46.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:46 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:37:46.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:46 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:37:46.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:46 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/3955178757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:37:46.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:46 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:46.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:46 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:46.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:46 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:46.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:46 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:37:46.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:46 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:37:46.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:46 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:37:47.660 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:37:47.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:47 smithi183 ceph-mon[32584]: Deploying daemon mds.c on smithi183 2024-04-25T14:37:47.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:47 smithi183 ceph-mon[32584]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:47.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:47 smithi183 ceph-mon[32584]: mds.? [v2:172.21.15.183:6834/1189328602,v1:172.21.15.183:6835/1189328602] up:boot 2024-04-25T14:37:47.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:47 smithi183 ceph-mon[32584]: fsmap 6 up:standby 2024-04-25T14:37:47.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:47 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T14:37:47.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:47 smithi150 ceph-mon[32504]: Deploying daemon mds.c on smithi183 2024-04-25T14:37:47.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:47 smithi150 ceph-mon[32504]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:47.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:47 smithi150 ceph-mon[32504]: mds.? [v2:172.21.15.183:6834/1189328602,v1:172.21.15.183:6835/1189328602] up:boot 2024-04-25T14:37:47.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:47 smithi150 ceph-mon[32504]: fsmap 6 up:standby 2024-04-25T14:37:47.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:47 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T14:37:47.739 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:47 smithi053 ceph-mon[27773]: Deploying daemon mds.c on smithi183 2024-04-25T14:37:47.740 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:47 smithi053 ceph-mon[27773]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:47.740 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:47 smithi053 ceph-mon[27773]: mds.? [v2:172.21.15.183:6834/1189328602,v1:172.21.15.183:6835/1189328602] up:boot 2024-04-25T14:37:47.740 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:47 smithi053 ceph-mon[27773]: fsmap 6 up:standby 2024-04-25T14:37:47.740 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:47 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T14:37:47.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:47.879+0000 7fde8d800640 1 -- 172.21.15.53:0/2871587709 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde88102850 msgr2=0x7fde88104c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:47.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:47.879+0000 7fde8d800640 1 --2- 172.21.15.53:0/2871587709 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde88102850 0x7fde88104c40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fde78009900 tx=0x7fde7802a3f0 comp rx=0 tx=0).stop 2024-04-25T14:37:47.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:47.879+0000 7fde8d800640 1 -- 172.21.15.53:0/2871587709 shutdown_connections 2024-04-25T14:37:47.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:47.879+0000 7fde8d800640 1 --2- 172.21.15.53:0/2871587709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde88107ab0 0x7fde88109f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:47.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:47.879+0000 7fde8d800640 1 --2- 172.21.15.53:0/2871587709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde88105180 0x7fde88107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:47.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:47.879+0000 7fde8d800640 1 --2- 172.21.15.53:0/2871587709 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde88102850 0x7fde88104c40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:47.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:47.879+0000 7fde8d800640 1 -- 172.21.15.53:0/2871587709 >> 172.21.15.53:0/2871587709 conn(0x7fde880fc260 msgr2=0x7fde880fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:37:47.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:47.879+0000 7fde8d800640 1 -- 172.21.15.53:0/2871587709 shutdown_connections 2024-04-25T14:37:47.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:47.879+0000 7fde8d800640 1 -- 172.21.15.53:0/2871587709 wait complete. 2024-04-25T14:37:47.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:47.881+0000 7fde8d800640 1 Processor -- start 2024-04-25T14:37:47.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:47.881+0000 7fde8d800640 1 -- start start 2024-04-25T14:37:47.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:47.881+0000 7fde8d800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde88102850 0x7fde881021c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:47.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:47.881+0000 7fde8d800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde88105180 0x7fde88100800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:47.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:47.881+0000 7fde8d800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde88107ab0 0x7fde88100d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:47.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:47.881+0000 7fde8d800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fde8810d950 con 0x7fde88102850 2024-04-25T14:37:47.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:47.881+0000 7fde8d800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fde8810d7d0 con 0x7fde88105180 2024-04-25T14:37:47.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:47.881+0000 7fde8d800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fde8810dad0 con 0x7fde88107ab0 2024-04-25T14:37:47.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:47.882+0000 7fde87e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde88102850 0x7fde881021c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:47.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:47.882+0000 7fde87400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde88105180 0x7fde88100800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:47.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:47.882+0000 7fde87e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde88102850 0x7fde881021c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38830/0 (socket says 172.21.15.53:38830) 2024-04-25T14:37:47.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:47.882+0000 7fde87e00640 1 -- 172.21.15.53:0/3691871304 learned_addr learned my addr 172.21.15.53:0/3691871304 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:37:47.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:47.882+0000 7fde87400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde88105180 0x7fde88100800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.53:32798/0 (socket says 172.21.15.53:32798) 2024-04-25T14:37:47.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:47.882+0000 7fde8ce00640 1 --2- 172.21.15.53:0/3691871304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde88107ab0 0x7fde88100d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:47.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:47.882+0000 7fde87400640 1 -- 172.21.15.53:0/3691871304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde88107ab0 msgr2=0x7fde88100d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:47.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:47.882+0000 7fde87400640 1 --2- 172.21.15.53:0/3691871304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde88107ab0 0x7fde88100d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:47.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:47.882+0000 7fde87400640 1 -- 172.21.15.53:0/3691871304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde88102850 msgr2=0x7fde881021c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:47.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:47.882+0000 7fde87400640 1 --2- 172.21.15.53:0/3691871304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde88102850 0x7fde881021c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:47.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:47.882+0000 7fde87400640 1 -- 172.21.15.53:0/3691871304 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde881015d0 con 0x7fde88105180 2024-04-25T14:37:47.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:47.882+0000 7fde87400640 1 --2- 172.21.15.53:0/3691871304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde88105180 0x7fde88100800 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fde7400d6a0 tx=0x7fde7400db60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:37:47.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:47.883+0000 7fde84c00640 1 -- 172.21.15.53:0/3691871304 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde74017070 con 0x7fde88105180 2024-04-25T14:37:47.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:47.883+0000 7fde84c00640 1 -- 172.21.15.53:0/3691871304 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde74009820 con 0x7fde88105180 2024-04-25T14:37:47.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:47.883+0000 7fde8d800640 1 -- 172.21.15.53:0/3691871304 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde881a2150 con 0x7fde88105180 2024-04-25T14:37:47.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:47.883+0000 7fde84c00640 1 -- 172.21.15.53:0/3691871304 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde74004c40 con 0x7fde88105180 2024-04-25T14:37:47.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:47.883+0000 7fde8d800640 1 -- 172.21.15.53:0/3691871304 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde881a26f0 con 0x7fde88105180 2024-04-25T14:37:47.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:47.884+0000 7fde8d800640 1 -- 172.21.15.53:0/3691871304 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde880fdd50 con 0x7fde88105180 2024-04-25T14:37:47.887 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:47.887+0000 7fde84c00640 1 -- 172.21.15.53:0/3691871304 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fde74005220 con 0x7fde88105180 2024-04-25T14:37:47.888 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:47.889+0000 7fde84c00640 1 --2- 172.21.15.53:0/3691871304 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fde6c0a3c30 0x7fde6c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:47.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:47.889+0000 7fde84c00640 1 -- 172.21.15.53:0/3691871304 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fde740cf7f0 con 0x7fde88105180 2024-04-25T14:37:47.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:47.889+0000 7fde87e00640 1 --2- 172.21.15.53:0/3691871304 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fde6c0a3c30 0x7fde6c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:47.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:47.890+0000 7fde87e00640 1 --2- 172.21.15.53:0/3691871304 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fde6c0a3c30 0x7fde6c0a60f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fde780096c0 tx=0x7fde78005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:37:47.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:47.893+0000 7fde84c00640 1 -- 172.21.15.53:0/3691871304 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fde7409bf80 con 0x7fde88105180 2024-04-25T14:37:48.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:48.144+0000 7fde8d800640 1 -- 172.21.15.53:0/3691871304 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde8802d5c0 con 0x7fde88105180 2024-04-25T14:37:48.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:48.145+0000 7fde84c00640 1 -- 172.21.15.53:0/3691871304 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5521 (secure 0 0 0) 0x7fde740a0280 con 0x7fde88105180 2024-04-25T14:37:48.145 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:37:48.145 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2966992472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2966992472},{"type":"v1","addr":"172.21.15.150:6833","nonce":2966992472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/94143643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":94143643},{"type":"v1","addr":"172.21.15.183:6833","nonce":94143643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24470,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/506682943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":506682943},{"type":"v1","addr":"172.21.15.53:6835","nonce":506682943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1369488844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1369488844},{"type":"v1","addr":"172.21.15.150:6835","nonce":1369488844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3786232682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3786232682},{"type":"v1","addr":"172.21.15.53:6837","nonce":3786232682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24532,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1189328602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1189328602},{"type":"v1","addr":"172.21.15.183:6835","nonce":1189328602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-25T14:37:48.145 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 7 2024-04-25T14:37:48.148 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:48.148+0000 7fde8d800640 1 -- 172.21.15.53:0/3691871304 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fde6c0a3c30 msgr2=0x7fde6c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:48.148 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:48.148+0000 7fde8d800640 1 --2- 172.21.15.53:0/3691871304 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fde6c0a3c30 0x7fde6c0a60f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fde780096c0 tx=0x7fde78005a20 comp rx=0 tx=0).stop 2024-04-25T14:37:48.148 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:48.148+0000 7fde8d800640 1 -- 172.21.15.53:0/3691871304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde88105180 msgr2=0x7fde88100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:48.148 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:48.148+0000 7fde8d800640 1 --2- 172.21.15.53:0/3691871304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde88105180 0x7fde88100800 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fde7400d6a0 tx=0x7fde7400db60 comp rx=0 tx=0).stop 2024-04-25T14:37:48.148 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:48.148+0000 7fde8d800640 1 -- 172.21.15.53:0/3691871304 shutdown_connections 2024-04-25T14:37:48.148 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:48.148+0000 7fde8d800640 1 --2- 172.21.15.53:0/3691871304 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fde6c0a3c30 0x7fde6c0a60f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:48.148 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:48.148+0000 7fde8d800640 1 --2- 172.21.15.53:0/3691871304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde88107ab0 0x7fde88100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:48.148 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:48.148+0000 7fde8d800640 1 --2- 172.21.15.53:0/3691871304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde88105180 0x7fde88100800 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:48.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:48.148+0000 7fde8d800640 1 --2- 172.21.15.53:0/3691871304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde88102850 0x7fde881021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:48.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:48.148+0000 7fde8d800640 1 -- 172.21.15.53:0/3691871304 >> 172.21.15.53:0/3691871304 conn(0x7fde880fc260 msgr2=0x7fde88103190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:37:48.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:48.148+0000 7fde8d800640 1 -- 172.21.15.53:0/3691871304 shutdown_connections 2024-04-25T14:37:48.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:48.148+0000 7fde8d800640 1 -- 172.21.15.53:0/3691871304 wait complete. 2024-04-25T14:37:48.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:48 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/3691871304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:37:48.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:48 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/3691871304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:37:48.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:48 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/3691871304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:37:48.778 INFO:journalctl@ceph.mds.c.smithi183.stdout:Apr 25 14:37:48 smithi183 podman[62575]: 2024-04-25 14:37:48.756662006 +0000 UTC m=+0.343817048 container start bd74bae21e32796343eea224b87bdff6caf6f3a7455e728ca7ce38eff177e013 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-c, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-25T14:37:49.204 INFO:journalctl@ceph.mds.c.smithi183.stdout:Apr 25 14:37:48 smithi183 ceph-mds[62594]: -- 172.21.15.183:0/1998987197 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5577d8db3a40 con 0x5577d8d86c00 2024-04-25T14:37:49.204 INFO:journalctl@ceph.mds.c.smithi183.stdout:Apr 25 14:37:48 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-c[62590]: starting mds.c at 2024-04-25T14:37:49.204 INFO:journalctl@ceph.mds.c.smithi183.stdout:Apr 25 14:37:48 smithi183 bash[62575]: bd74bae21e32796343eea224b87bdff6caf6f3a7455e728ca7ce38eff177e013 2024-04-25T14:37:49.204 INFO:journalctl@ceph.mds.c.smithi183.stdout:Apr 25 14:37:48 smithi183 systemd[1]: Started Ceph mds.c for fd30e6ce-030f-11ef-bc93-c7b262605968. 2024-04-25T14:37:50.168 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:37:50.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:49 smithi183 ceph-mon[32584]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:50.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:49 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:50.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:49 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:50.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:49 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:50.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:49 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:37:50.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:49 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:37:50.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:49 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:37:50.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:49 smithi183 ceph-mon[32584]: Deploying daemon mds.h on smithi150 2024-04-25T14:37:50.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:49 smithi183 ceph-mon[32584]: mds.? [v2:172.21.15.183:6836/1163336384,v1:172.21.15.183:6837/1163336384] up:boot 2024-04-25T14:37:50.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:49 smithi183 ceph-mon[32584]: fsmap 7 up:standby 2024-04-25T14:37:50.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:49 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T14:37:50.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:49 smithi150 ceph-mon[32504]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:50.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:49 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:50.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:49 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:50.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:49 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:50.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:49 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:37:50.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:49 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:37:50.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:49 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:37:50.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:49 smithi150 ceph-mon[32504]: Deploying daemon mds.h on smithi150 2024-04-25T14:37:50.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:49 smithi150 ceph-mon[32504]: mds.? [v2:172.21.15.183:6836/1163336384,v1:172.21.15.183:6837/1163336384] up:boot 2024-04-25T14:37:50.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:49 smithi150 ceph-mon[32504]: fsmap 7 up:standby 2024-04-25T14:37:50.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:49 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T14:37:50.250 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:49 smithi053 ceph-mon[27773]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:50.250 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:49 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:50.250 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:49 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:50.250 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:49 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:50.250 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:49 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:37:50.251 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:49 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:37:50.251 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:49 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:37:50.251 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:49 smithi053 ceph-mon[27773]: Deploying daemon mds.h on smithi150 2024-04-25T14:37:50.251 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:49 smithi053 ceph-mon[27773]: mds.? [v2:172.21.15.183:6836/1163336384,v1:172.21.15.183:6837/1163336384] up:boot 2024-04-25T14:37:50.251 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:49 smithi053 ceph-mon[27773]: fsmap 7 up:standby 2024-04-25T14:37:50.251 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:49 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T14:37:50.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.389+0000 7ff861c00640 1 -- 172.21.15.53:0/2056183491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff85c100800 msgr2=0x7ff85c100c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:50.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.389+0000 7ff861c00640 1 --2- 172.21.15.53:0/2056183491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff85c100800 0x7ff85c100c20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff84c009900 tx=0x7ff84c02a3f0 comp rx=0 tx=0).stop 2024-04-25T14:37:50.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.390+0000 7ff861c00640 1 -- 172.21.15.53:0/2056183491 shutdown_connections 2024-04-25T14:37:50.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.390+0000 7ff861c00640 1 --2- 172.21.15.53:0/2056183491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff85c101b00 0x7ff85c101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:50.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.390+0000 7ff861c00640 1 --2- 172.21.15.53:0/2056183491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff85c101160 0x7ff85c1015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:50.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.390+0000 7ff861c00640 1 --2- 172.21.15.53:0/2056183491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff85c100800 0x7ff85c100c20 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:50.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.390+0000 7ff861c00640 1 -- 172.21.15.53:0/2056183491 >> 172.21.15.53:0/2056183491 conn(0x7ff85c0fc400 msgr2=0x7ff85c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:37:50.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.390+0000 7ff861c00640 1 -- 172.21.15.53:0/2056183491 shutdown_connections 2024-04-25T14:37:50.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.390+0000 7ff861c00640 1 -- 172.21.15.53:0/2056183491 wait complete. 2024-04-25T14:37:50.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.391+0000 7ff861c00640 1 Processor -- start 2024-04-25T14:37:50.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.391+0000 7ff861c00640 1 -- start start 2024-04-25T14:37:50.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.392+0000 7ff861c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff85c100800 0x7ff85c19a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:50.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.392+0000 7ff861c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff85c101160 0x7ff85c19a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:50.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.392+0000 7ff861c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff85c101b00 0x7ff85c1a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:50.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.392+0000 7ff861c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c10fd30 con 0x7ff85c101160 2024-04-25T14:37:50.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.392+0000 7ff861c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c10fbb0 con 0x7ff85c101b00 2024-04-25T14:37:50.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.392+0000 7ff861c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c10feb0 con 0x7ff85c100800 2024-04-25T14:37:50.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.392+0000 7ff85aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff85c100800 0x7ff85c19a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:50.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.392+0000 7ff85aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff85c100800 0x7ff85c19a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:38792/0 (socket says 172.21.15.53:38792) 2024-04-25T14:37:50.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.392+0000 7ff85aa00640 1 -- 172.21.15.53:0/3646325365 learned_addr learned my addr 172.21.15.53:0/3646325365 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:37:50.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.392+0000 7ff85a000640 1 --2- 172.21.15.53:0/3646325365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff85c101160 0x7ff85c19a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:50.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.392+0000 7ff85b400640 1 --2- 172.21.15.53:0/3646325365 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff85c101b00 0x7ff85c1a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:50.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.392+0000 7ff85aa00640 1 -- 172.21.15.53:0/3646325365 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff85c101b00 msgr2=0x7ff85c1a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:50.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.392+0000 7ff85aa00640 1 --2- 172.21.15.53:0/3646325365 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff85c101b00 0x7ff85c1a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:50.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.392+0000 7ff85aa00640 1 -- 172.21.15.53:0/3646325365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff85c101160 msgr2=0x7ff85c19a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:50.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.392+0000 7ff85aa00640 1 --2- 172.21.15.53:0/3646325365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff85c101160 0x7ff85c19a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:50.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.392+0000 7ff85aa00640 1 -- 172.21.15.53:0/3646325365 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff85c1a1fd0 con 0x7ff85c100800 2024-04-25T14:37:50.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.393+0000 7ff85aa00640 1 --2- 172.21.15.53:0/3646325365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff85c100800 0x7ff85c19a3b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff84c005ab0 tx=0x7ff84c003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:37:50.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.393+0000 7ff853400640 1 -- 172.21.15.53:0/3646325365 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84c03f070 con 0x7ff85c100800 2024-04-25T14:37:50.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.393+0000 7ff853400640 1 -- 172.21.15.53:0/3646325365 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff84c031c50 con 0x7ff85c100800 2024-04-25T14:37:50.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.393+0000 7ff861c00640 1 -- 172.21.15.53:0/3646325365 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff85c1a2260 con 0x7ff85c100800 2024-04-25T14:37:50.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.393+0000 7ff853400640 1 -- 172.21.15.53:0/3646325365 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84c035030 con 0x7ff85c100800 2024-04-25T14:37:50.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.393+0000 7ff861c00640 1 -- 172.21.15.53:0/3646325365 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff85c1a2750 con 0x7ff85c100800 2024-04-25T14:37:50.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.394+0000 7ff861c00640 1 -- 172.21.15.53:0/3646325365 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff85c10b780 con 0x7ff85c100800 2024-04-25T14:37:50.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.396+0000 7ff853400640 1 -- 172.21.15.53:0/3646325365 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff84c042070 con 0x7ff85c100800 2024-04-25T14:37:50.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.398+0000 7ff853400640 1 --2- 172.21.15.53:0/3646325365 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff8400a3bb0 0x7ff8400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:50.398 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.398+0000 7ff853400640 1 -- 172.21.15.53:0/3646325365 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7ff84c0f0730 con 0x7ff85c100800 2024-04-25T14:37:50.398 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.398+0000 7ff85a000640 1 --2- 172.21.15.53:0/3646325365 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff8400a3bb0 0x7ff8400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:50.399 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.399+0000 7ff85a000640 1 --2- 172.21.15.53:0/3646325365 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff8400a3bb0 0x7ff8400a6070 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff85c19b8c0 tx=0x7ff848005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:37:50.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.402+0000 7ff853400640 1 -- 172.21.15.53:0/3646325365 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff84c0bcec0 con 0x7ff85c100800 2024-04-25T14:37:50.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.664+0000 7ff861c00640 1 -- 172.21.15.53:0/3646325365 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff85c02d5c0 con 0x7ff85c100800 2024-04-25T14:37:50.665 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:37:50.665 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2966992472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2966992472},{"type":"v1","addr":"172.21.15.150:6833","nonce":2966992472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/94143643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":94143643},{"type":"v1","addr":"172.21.15.183:6833","nonce":94143643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24470,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/506682943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":506682943},{"type":"v1","addr":"172.21.15.53:6835","nonce":506682943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1369488844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1369488844},{"type":"v1","addr":"172.21.15.150:6835","nonce":1369488844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3786232682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3786232682},{"type":"v1","addr":"172.21.15.53:6837","nonce":3786232682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24532,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1189328602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1189328602},{"type":"v1","addr":"172.21.15.183:6835","nonce":1189328602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24544,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/1163336384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1163336384},{"type":"v1","addr":"172.21.15.183:6837","nonce":1163336384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T14:37:50.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.665+0000 7ff853400640 1 -- 172.21.15.53:0/3646325365 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6349 (secure 0 0 0) 0x7ff84c0c11c0 con 0x7ff85c100800 2024-04-25T14:37:50.665 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 8 2024-04-25T14:37:50.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.667+0000 7ff861c00640 1 -- 172.21.15.53:0/3646325365 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff8400a3bb0 msgr2=0x7ff8400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:50.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.667+0000 7ff861c00640 1 --2- 172.21.15.53:0/3646325365 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff8400a3bb0 0x7ff8400a6070 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff85c19b8c0 tx=0x7ff848005f90 comp rx=0 tx=0).stop 2024-04-25T14:37:50.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.667+0000 7ff861c00640 1 -- 172.21.15.53:0/3646325365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff85c100800 msgr2=0x7ff85c19a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:50.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.667+0000 7ff861c00640 1 --2- 172.21.15.53:0/3646325365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff85c100800 0x7ff85c19a3b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff84c005ab0 tx=0x7ff84c003b90 comp rx=0 tx=0).stop 2024-04-25T14:37:50.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.667+0000 7ff861c00640 1 -- 172.21.15.53:0/3646325365 shutdown_connections 2024-04-25T14:37:50.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.667+0000 7ff861c00640 1 --2- 172.21.15.53:0/3646325365 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff8400a3bb0 0x7ff8400a6070 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:50.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.667+0000 7ff861c00640 1 --2- 172.21.15.53:0/3646325365 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff85c101b00 0x7ff85c1a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:50.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.667+0000 7ff861c00640 1 --2- 172.21.15.53:0/3646325365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff85c101160 0x7ff85c19a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:50.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.667+0000 7ff861c00640 1 --2- 172.21.15.53:0/3646325365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff85c100800 0x7ff85c19a3b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:50.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.667+0000 7ff861c00640 1 -- 172.21.15.53:0/3646325365 >> 172.21.15.53:0/3646325365 conn(0x7ff85c0fc400 msgr2=0x7ff85c0fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:37:50.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.667+0000 7ff861c00640 1 -- 172.21.15.53:0/3646325365 shutdown_connections 2024-04-25T14:37:50.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:50.667+0000 7ff861c00640 1 -- 172.21.15.53:0/3646325365 wait complete. 2024-04-25T14:37:51.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:50 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/3646325365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:37:51.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:50 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/3646325365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:37:51.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:50 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/3646325365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:37:51.211 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 25 14:37:51 smithi150 systemd[1]: Starting Ceph mds.h for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:37:51.494 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 25 14:37:51 smithi150 podman[62572]: 2024-04-25 14:37:51.131279559 +0000 UTC m=+0.018040067 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T14:37:51.494 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 25 14:37:51 smithi150 podman[62572]: 2024-04-25 14:37:51.266679048 +0000 UTC m=+0.153439543 container create 803c34c5a1c6314a346783f7e027fc67addeadfd9258b49078ddb2b62393b508 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-h, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-25T14:37:51.494 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 25 14:37:51 smithi150 podman[62572]: 2024-04-25 14:37:51.494088174 +0000 UTC m=+0.380848679 container init 803c34c5a1c6314a346783f7e027fc67addeadfd9258b49078ddb2b62393b508 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T14:37:51.826 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 25 14:37:51 smithi150 podman[62572]: 2024-04-25 14:37:51.496851599 +0000 UTC m=+0.383612095 container start 803c34c5a1c6314a346783f7e027fc67addeadfd9258b49078ddb2b62393b508 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-h, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T14:37:51.826 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 25 14:37:51 smithi150 ceph-mds[62592]: -- 172.21.15.150:0/2850958976 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55881b70ba40 con 0x55881b6dec00 2024-04-25T14:37:51.826 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 25 14:37:51 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-h[62587]: starting mds.h at 2024-04-25T14:37:51.826 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 25 14:37:51 smithi150 bash[62572]: 803c34c5a1c6314a346783f7e027fc67addeadfd9258b49078ddb2b62393b508 2024-04-25T14:37:51.826 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 25 14:37:51 smithi150 systemd[1]: Started Ceph mds.h for fd30e6ce-030f-11ef-bc93-c7b262605968. 2024-04-25T14:37:52.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:51 smithi183 ceph-mon[32584]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:52.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:51 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:52.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:51 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:52.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:51 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:52.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:51 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:37:52.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:51 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:37:52.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:51 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:37:52.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:51 smithi150 ceph-mon[32504]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:52.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:51 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:52.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:51 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:52.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:51 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:52.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:51 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:37:52.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:51 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:37:52.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:51 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:37:52.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:51 smithi053 ceph-mon[27773]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:52.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:51 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:52.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:51 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:52.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:51 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:52.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:51 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:37:52.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:51 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:37:52.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:51 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:37:52.685 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:37:52.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:52.906+0000 7f9919e00640 1 -- 172.21.15.53:0/331807020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9914105b80 msgr2=0x7f991410c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:52.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:52.906+0000 7f9919e00640 1 --2- 172.21.15.53:0/331807020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9914105b80 0x7f991410c410 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f990c009ca0 tx=0x7f990c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:37:52.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:52.906+0000 7f9919e00640 1 -- 172.21.15.53:0/331807020 shutdown_connections 2024-04-25T14:37:52.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:52.906+0000 7f9919e00640 1 --2- 172.21.15.53:0/331807020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9914105b80 0x7f991410c410 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:52.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:52.906+0000 7f9919e00640 1 --2- 172.21.15.53:0/331807020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99141051c0 0x7f9914105640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:52.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:52.906+0000 7f9919e00640 1 --2- 172.21.15.53:0/331807020 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9914103fd0 0x7f99141043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:52.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:52.906+0000 7f9919e00640 1 -- 172.21.15.53:0/331807020 >> 172.21.15.53:0/331807020 conn(0x7f99140ff560 msgr2=0x7f9914101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:37:52.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:52.906+0000 7f9919e00640 1 -- 172.21.15.53:0/331807020 shutdown_connections 2024-04-25T14:37:52.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:52.907+0000 7f9919e00640 1 -- 172.21.15.53:0/331807020 wait complete. 2024-04-25T14:37:52.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:52.908+0000 7f9919e00640 1 Processor -- start 2024-04-25T14:37:52.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:52.908+0000 7f9919e00640 1 -- start start 2024-04-25T14:37:52.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:52.908+0000 7f9919e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9914103fd0 0x7f991419a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:52.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:52.908+0000 7f9919e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99141051c0 0x7f991419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:52.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:52.908+0000 7f9919e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9914105b80 0x7f99141a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:52.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:52.908+0000 7f9919e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f991410fb60 con 0x7f9914105b80 2024-04-25T14:37:52.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:52.908+0000 7f9919e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f991410f9e0 con 0x7f99141051c0 2024-04-25T14:37:52.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:52.908+0000 7f9919e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f991410fce0 con 0x7f9914103fd0 2024-04-25T14:37:52.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:52.909+0000 7f9918a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9914103fd0 0x7f991419a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:52.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:52.909+0000 7f9918a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9914103fd0 0x7f991419a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:49906/0 (socket says 172.21.15.53:49906) 2024-04-25T14:37:52.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:52.909+0000 7f9918a00640 1 -- 172.21.15.53:0/2019972640 learned_addr learned my addr 172.21.15.53:0/2019972640 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:37:52.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:52.909+0000 7f9919400640 1 --2- 172.21.15.53:0/2019972640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9914105b80 0x7f99141a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:52.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:52.909+0000 7f9913e00640 1 --2- 172.21.15.53:0/2019972640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99141051c0 0x7f991419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:52.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:52.909+0000 7f9918a00640 1 -- 172.21.15.53:0/2019972640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99141051c0 msgr2=0x7f991419a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:52.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:52.909+0000 7f9918a00640 1 --2- 172.21.15.53:0/2019972640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99141051c0 0x7f991419a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:52.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:52.909+0000 7f9918a00640 1 -- 172.21.15.53:0/2019972640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9914105b80 msgr2=0x7f99141a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:52.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:52.909+0000 7f9918a00640 1 --2- 172.21.15.53:0/2019972640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9914105b80 0x7f99141a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:52.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:52.909+0000 7f9918a00640 1 -- 172.21.15.53:0/2019972640 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99141a1db0 con 0x7f9914103fd0 2024-04-25T14:37:52.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:52.910+0000 7f9918a00640 1 --2- 172.21.15.53:0/2019972640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9914103fd0 0x7f991419a190 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f990800e9c0 tx=0x7f990800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:37:52.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:52.910+0000 7f9911600640 1 -- 172.21.15.53:0/2019972640 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990800cde0 con 0x7f9914103fd0 2024-04-25T14:37:52.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:52.910+0000 7f9911600640 1 -- 172.21.15.53:0/2019972640 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99080108e0 con 0x7f9914103fd0 2024-04-25T14:37:52.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:52.910+0000 7f9911600640 1 -- 172.21.15.53:0/2019972640 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9908018960 con 0x7f9914103fd0 2024-04-25T14:37:52.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:52.910+0000 7f9919e00640 1 -- 172.21.15.53:0/2019972640 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99141a20a0 con 0x7f9914103fd0 2024-04-25T14:37:52.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:52.910+0000 7f9919e00640 1 -- 172.21.15.53:0/2019972640 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99141a2640 con 0x7f9914103fd0 2024-04-25T14:37:52.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:52.911+0000 7f9919e00640 1 -- 172.21.15.53:0/2019972640 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f991410b2f0 con 0x7f9914103fd0 2024-04-25T14:37:52.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:52.913+0000 7f9911600640 1 -- 172.21.15.53:0/2019972640 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f990801f020 con 0x7f9914103fd0 2024-04-25T14:37:52.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:52.915+0000 7f9911600640 1 --2- 172.21.15.53:0/2019972640 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f98fc0a3c30 0x7f98fc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:52.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:52.915+0000 7f9911600640 1 -- 172.21.15.53:0/2019972640 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f99080d0150 con 0x7f9914103fd0 2024-04-25T14:37:52.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:52.915+0000 7f9913e00640 1 --2- 172.21.15.53:0/2019972640 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f98fc0a3c30 0x7f98fc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:52.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:52.916+0000 7f9913e00640 1 --2- 172.21.15.53:0/2019972640 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f98fc0a3c30 0x7f98fc0a60f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f99000095f0 tx=0x7f9900005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:37:52.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:52.919+0000 7f9911600640 1 -- 172.21.15.53:0/2019972640 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9908053ff0 con 0x7f9914103fd0 2024-04-25T14:37:53.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:52 smithi053 ceph-mon[27773]: Deploying daemon mds.i on smithi183 2024-04-25T14:37:53.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:52 smithi053 ceph-mon[27773]: mds.? [v2:172.21.15.150:6836/2344343627,v1:172.21.15.150:6837/2344343627] up:boot 2024-04-25T14:37:53.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:52 smithi053 ceph-mon[27773]: fsmap 8 up:standby 2024-04-25T14:37:53.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:52 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T14:37:53.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:53.169+0000 7f9919e00640 1 -- 172.21.15.53:0/2019972640 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f991402d5c0 con 0x7f9914103fd0 2024-04-25T14:37:53.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:53.171+0000 7f9911600640 1 -- 172.21.15.53:0/2019972640 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7177 (secure 0 0 0) 0x7f990809cfb0 con 0x7f9914103fd0 2024-04-25T14:37:53.171 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:37:53.171 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2966992472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2966992472},{"type":"v1","addr":"172.21.15.150:6833","nonce":2966992472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/94143643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":94143643},{"type":"v1","addr":"172.21.15.183:6833","nonce":94143643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24470,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/506682943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":506682943},{"type":"v1","addr":"172.21.15.53:6835","nonce":506682943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1369488844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1369488844},{"type":"v1","addr":"172.21.15.150:6835","nonce":1369488844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3786232682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3786232682},{"type":"v1","addr":"172.21.15.53:6837","nonce":3786232682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24532,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1189328602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1189328602},{"type":"v1","addr":"172.21.15.183:6835","nonce":1189328602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2344343627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2344343627},{"type":"v1","addr":"172.21.15.150:6837","nonce":2344343627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24544,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/1163336384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1163336384},{"type":"v1","addr":"172.21.15.183:6837","nonce":1163336384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T14:37:53.171 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 9 2024-04-25T14:37:53.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:53.173+0000 7f9919e00640 1 -- 172.21.15.53:0/2019972640 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f98fc0a3c30 msgr2=0x7f98fc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:53.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:53.173+0000 7f9919e00640 1 --2- 172.21.15.53:0/2019972640 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f98fc0a3c30 0x7f98fc0a60f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f99000095f0 tx=0x7f9900005f90 comp rx=0 tx=0).stop 2024-04-25T14:37:53.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:53.173+0000 7f9919e00640 1 -- 172.21.15.53:0/2019972640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9914103fd0 msgr2=0x7f991419a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:53.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:53.173+0000 7f9919e00640 1 --2- 172.21.15.53:0/2019972640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9914103fd0 0x7f991419a190 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f990800e9c0 tx=0x7f990800ee80 comp rx=0 tx=0).stop 2024-04-25T14:37:53.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:53.174+0000 7f9919e00640 1 -- 172.21.15.53:0/2019972640 shutdown_connections 2024-04-25T14:37:53.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:53.174+0000 7f9919e00640 1 --2- 172.21.15.53:0/2019972640 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f98fc0a3c30 0x7f98fc0a60f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:53.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:53.174+0000 7f9919e00640 1 --2- 172.21.15.53:0/2019972640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9914105b80 0x7f99141a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:53.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:53.174+0000 7f9919e00640 1 --2- 172.21.15.53:0/2019972640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99141051c0 0x7f991419a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:53.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:53.174+0000 7f9919e00640 1 --2- 172.21.15.53:0/2019972640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9914103fd0 0x7f991419a190 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:53.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:53.174+0000 7f9919e00640 1 -- 172.21.15.53:0/2019972640 >> 172.21.15.53:0/2019972640 conn(0x7f99140ff560 msgr2=0x7f99141084c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:37:53.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:53.174+0000 7f9919e00640 1 -- 172.21.15.53:0/2019972640 shutdown_connections 2024-04-25T14:37:53.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:53.174+0000 7f9919e00640 1 -- 172.21.15.53:0/2019972640 wait complete. 2024-04-25T14:37:53.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:52 smithi183 ceph-mon[32584]: Deploying daemon mds.i on smithi183 2024-04-25T14:37:53.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:52 smithi183 ceph-mon[32584]: mds.? [v2:172.21.15.150:6836/2344343627,v1:172.21.15.150:6837/2344343627] up:boot 2024-04-25T14:37:53.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:52 smithi183 ceph-mon[32584]: fsmap 8 up:standby 2024-04-25T14:37:53.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:52 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T14:37:53.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:52 smithi150 ceph-mon[32504]: Deploying daemon mds.i on smithi183 2024-04-25T14:37:53.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:52 smithi150 ceph-mon[32504]: mds.? [v2:172.21.15.150:6836/2344343627,v1:172.21.15.150:6837/2344343627] up:boot 2024-04-25T14:37:53.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:52 smithi150 ceph-mon[32504]: fsmap 8 up:standby 2024-04-25T14:37:53.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:52 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T14:37:53.927 INFO:journalctl@ceph.mds.i.smithi183.stdout:Apr 25 14:37:53 smithi183 systemd[1]: Starting Ceph mds.i for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:37:54.198 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:53 smithi183 ceph-mon[32584]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:54.198 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:53 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/2019972640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:37:54.198 INFO:journalctl@ceph.mds.i.smithi183.stdout:Apr 25 14:37:54 smithi183 podman[62877]: 2024-04-25 14:37:53.973288847 +0000 UTC m=+0.017875561 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T14:37:54.198 INFO:journalctl@ceph.mds.i.smithi183.stdout:Apr 25 14:37:54 smithi183 podman[62877]: 2024-04-25 14:37:54.087910353 +0000 UTC m=+0.132497112 container create 90c0d1203a2f408baf231e8d0fc549627a5442b194bd80ecdf3c177681478b77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-i, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-25T14:37:54.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:53 smithi150 ceph-mon[32504]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:54.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:53 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/2019972640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:37:54.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:53 smithi053 ceph-mon[27773]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:54.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:53 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/2019972640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:37:54.704 INFO:journalctl@ceph.mds.i.smithi183.stdout:Apr 25 14:37:54 smithi183 podman[62877]: 2024-04-25 14:37:54.315183705 +0000 UTC m=+0.359770402 container init 90c0d1203a2f408baf231e8d0fc549627a5442b194bd80ecdf3c177681478b77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-i, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T14:37:54.704 INFO:journalctl@ceph.mds.i.smithi183.stdout:Apr 25 14:37:54 smithi183 podman[62877]: 2024-04-25 14:37:54.317919495 +0000 UTC m=+0.362506192 container start 90c0d1203a2f408baf231e8d0fc549627a5442b194bd80ecdf3c177681478b77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-i, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T14:37:54.704 INFO:journalctl@ceph.mds.i.smithi183.stdout:Apr 25 14:37:54 smithi183 ceph-mds[62896]: -- 172.21.15.183:0/2056810490 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55daad409a40 con 0x55daad3dcc00 2024-04-25T14:37:54.704 INFO:journalctl@ceph.mds.i.smithi183.stdout:Apr 25 14:37:54 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-i[62892]: starting mds.i at 2024-04-25T14:37:54.704 INFO:journalctl@ceph.mds.i.smithi183.stdout:Apr 25 14:37:54 smithi183 bash[62877]: 90c0d1203a2f408baf231e8d0fc549627a5442b194bd80ecdf3c177681478b77 2024-04-25T14:37:54.704 INFO:journalctl@ceph.mds.i.smithi183.stdout:Apr 25 14:37:54 smithi183 systemd[1]: Started Ceph mds.i for fd30e6ce-030f-11ef-bc93-c7b262605968. 2024-04-25T14:37:55.195 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:37:55.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.391+0000 7ffb75400640 1 -- 172.21.15.53:0/3380731583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb70102fe0 msgr2=0x7ffb70103460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:55.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.391+0000 7ffb75400640 1 --2- 172.21.15.53:0/3380731583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb70102fe0 0x7ffb70103460 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c009900 tx=0x7ffb5c02a410 comp rx=0 tx=0).stop 2024-04-25T14:37:55.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.391+0000 7ffb75400640 1 -- 172.21.15.53:0/3380731583 shutdown_connections 2024-04-25T14:37:55.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.391+0000 7ffb75400640 1 --2- 172.21.15.53:0/3380731583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb701039a0 0x7ffb7010a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:55.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.391+0000 7ffb75400640 1 --2- 172.21.15.53:0/3380731583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb70102fe0 0x7ffb70103460 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:55.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.391+0000 7ffb75400640 1 --2- 172.21.15.53:0/3380731583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb70101df0 0x7ffb70102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:55.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.391+0000 7ffb75400640 1 -- 172.21.15.53:0/3380731583 >> 172.21.15.53:0/3380731583 conn(0x7ffb700fd360 msgr2=0x7ffb700ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:37:55.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.391+0000 7ffb75400640 1 -- 172.21.15.53:0/3380731583 shutdown_connections 2024-04-25T14:37:55.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.391+0000 7ffb75400640 1 -- 172.21.15.53:0/3380731583 wait complete. 2024-04-25T14:37:55.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.392+0000 7ffb75400640 1 Processor -- start 2024-04-25T14:37:55.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.392+0000 7ffb75400640 1 -- start start 2024-04-25T14:37:55.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.393+0000 7ffb75400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb70101df0 0x7ffb7019a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:55.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.393+0000 7ffb75400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb70102fe0 0x7ffb7019a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:55.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.393+0000 7ffb75400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb701039a0 0x7ffb701a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:55.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.393+0000 7ffb75400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7010d970 con 0x7ffb701039a0 2024-04-25T14:37:55.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.393+0000 7ffb75400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7010d7f0 con 0x7ffb70101df0 2024-04-25T14:37:55.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.393+0000 7ffb75400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7010daf0 con 0x7ffb70102fe0 2024-04-25T14:37:55.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.393+0000 7ffb74a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb701039a0 0x7ffb701a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:55.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.393+0000 7ffb6fe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb70101df0 0x7ffb7019a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:55.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.393+0000 7ffb74a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb701039a0 0x7ffb701a1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45652/0 (socket says 172.21.15.53:45652) 2024-04-25T14:37:55.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.393+0000 7ffb74a00640 1 -- 172.21.15.53:0/3685041510 learned_addr learned my addr 172.21.15.53:0/3685041510 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:37:55.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.393+0000 7ffb6f400640 1 --2- 172.21.15.53:0/3685041510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb70102fe0 0x7ffb7019a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:55.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.394+0000 7ffb6fe00640 1 -- 172.21.15.53:0/3685041510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb70102fe0 msgr2=0x7ffb7019a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:55.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.394+0000 7ffb6fe00640 1 --2- 172.21.15.53:0/3685041510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb70102fe0 0x7ffb7019a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:55.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.394+0000 7ffb6fe00640 1 -- 172.21.15.53:0/3685041510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb701039a0 msgr2=0x7ffb701a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:55.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.394+0000 7ffb6fe00640 1 --2- 172.21.15.53:0/3685041510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb701039a0 0x7ffb701a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:55.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.394+0000 7ffb6fe00640 1 -- 172.21.15.53:0/3685041510 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb701a1ea0 con 0x7ffb70101df0 2024-04-25T14:37:55.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.394+0000 7ffb74a00640 1 --2- 172.21.15.53:0/3685041510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb701039a0 0x7ffb701a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:37:55.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.394+0000 7ffb6fe00640 1 --2- 172.21.15.53:0/3685041510 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb70101df0 0x7ffb7019a2d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ffb6000e9c0 tx=0x7ffb6000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:37:55.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.395+0000 7ffb6cc00640 1 -- 172.21.15.53:0/3685041510 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6000ccc0 con 0x7ffb70101df0 2024-04-25T14:37:55.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.395+0000 7ffb6cc00640 1 -- 172.21.15.53:0/3685041510 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb600035f0 con 0x7ffb70101df0 2024-04-25T14:37:55.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.395+0000 7ffb75400640 1 -- 172.21.15.53:0/3685041510 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb701a2190 con 0x7ffb70101df0 2024-04-25T14:37:55.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.395+0000 7ffb6cc00640 1 -- 172.21.15.53:0/3685041510 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6000ce60 con 0x7ffb70101df0 2024-04-25T14:37:55.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.395+0000 7ffb75400640 1 -- 172.21.15.53:0/3685041510 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb701a26b0 con 0x7ffb70101df0 2024-04-25T14:37:55.398 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.396+0000 7ffb75400640 1 -- 172.21.15.53:0/3685041510 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb701a2970 con 0x7ffb70101df0 2024-04-25T14:37:55.398 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.398+0000 7ffb6cc00640 1 -- 172.21.15.53:0/3685041510 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ffb60004000 con 0x7ffb70101df0 2024-04-25T14:37:55.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.400+0000 7ffb6cc00640 1 --2- 172.21.15.53:0/3685041510 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ffb540a3bb0 0x7ffb540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:55.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.401+0000 7ffb6cc00640 1 -- 172.21.15.53:0/3685041510 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7ffb600d0420 con 0x7ffb70101df0 2024-04-25T14:37:55.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.401+0000 7ffb6f400640 1 --2- 172.21.15.53:0/3685041510 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ffb540a3bb0 0x7ffb540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:55.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.402+0000 7ffb6f400640 1 --2- 172.21.15.53:0/3685041510 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ffb540a3bb0 0x7ffb540a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c004770 tx=0x7ffb5c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:37:55.404 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.404+0000 7ffb6cc00640 1 -- 172.21.15.53:0/3685041510 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffb6009d200 con 0x7ffb70101df0 2024-04-25T14:37:55.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:55 smithi053 ceph-mon[27773]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:55.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:55 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:55.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:55 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:55.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:55 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:55.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:55 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:37:55.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:55 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:37:55.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:55 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:37:55.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:55 smithi053 ceph-mon[27773]: Deploying daemon mds.g on smithi053 2024-04-25T14:37:55.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:55 smithi053 ceph-mon[27773]: mds.? [v2:172.21.15.183:6838/1554258915,v1:172.21.15.183:6839/1554258915] up:boot 2024-04-25T14:37:55.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:55 smithi053 ceph-mon[27773]: fsmap 9 up:standby 2024-04-25T14:37:55.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:55 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T14:37:55.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.671+0000 7ffb75400640 1 -- 172.21.15.53:0/3685041510 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb701a2b40 con 0x7ffb70101df0 2024-04-25T14:37:55.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.673+0000 7ffb6cc00640 1 -- 172.21.15.53:0/3685041510 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8007 (secure 0 0 0) 0x7ffb6009cb00 con 0x7ffb70101df0 2024-04-25T14:37:55.673 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:37:55.673 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2966992472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2966992472},{"type":"v1","addr":"172.21.15.150:6833","nonce":2966992472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/94143643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":94143643},{"type":"v1","addr":"172.21.15.183:6833","nonce":94143643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24470,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/506682943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":506682943},{"type":"v1","addr":"172.21.15.53:6835","nonce":506682943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1369488844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1369488844},{"type":"v1","addr":"172.21.15.150:6835","nonce":1369488844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3786232682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3786232682},{"type":"v1","addr":"172.21.15.53:6837","nonce":3786232682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24532,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1189328602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1189328602},{"type":"v1","addr":"172.21.15.183:6835","nonce":1189328602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2344343627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2344343627},{"type":"v1","addr":"172.21.15.150:6837","nonce":2344343627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24544,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/1163336384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1163336384},{"type":"v1","addr":"172.21.15.183:6837","nonce":1163336384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24565,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/1554258915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1554258915},{"type":"v1","addr":"172.21.15.183:6839","nonce":1554258915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T14:37:55.673 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 10 2024-04-25T14:37:55.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.675+0000 7ffb75400640 1 -- 172.21.15.53:0/3685041510 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ffb540a3bb0 msgr2=0x7ffb540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:55.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.675+0000 7ffb75400640 1 --2- 172.21.15.53:0/3685041510 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ffb540a3bb0 0x7ffb540a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c004770 tx=0x7ffb5c0314a0 comp rx=0 tx=0).stop 2024-04-25T14:37:55.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.675+0000 7ffb75400640 1 -- 172.21.15.53:0/3685041510 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb70101df0 msgr2=0x7ffb7019a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:55.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.675+0000 7ffb75400640 1 --2- 172.21.15.53:0/3685041510 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb70101df0 0x7ffb7019a2d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ffb6000e9c0 tx=0x7ffb6000ee80 comp rx=0 tx=0).stop 2024-04-25T14:37:55.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.675+0000 7ffb75400640 1 -- 172.21.15.53:0/3685041510 shutdown_connections 2024-04-25T14:37:55.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.675+0000 7ffb75400640 1 --2- 172.21.15.53:0/3685041510 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ffb540a3bb0 0x7ffb540a6070 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:55.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.675+0000 7ffb75400640 1 --2- 172.21.15.53:0/3685041510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb701039a0 0x7ffb701a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:55.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.675+0000 7ffb75400640 1 --2- 172.21.15.53:0/3685041510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb70102fe0 0x7ffb7019a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:55.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.675+0000 7ffb75400640 1 --2- 172.21.15.53:0/3685041510 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb70101df0 0x7ffb7019a2d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:55.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.675+0000 7ffb75400640 1 -- 172.21.15.53:0/3685041510 >> 172.21.15.53:0/3685041510 conn(0x7ffb700fd360 msgr2=0x7ffb70074180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:37:55.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.675+0000 7ffb75400640 1 -- 172.21.15.53:0/3685041510 shutdown_connections 2024-04-25T14:37:55.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:55.676+0000 7ffb75400640 1 -- 172.21.15.53:0/3685041510 wait complete. 2024-04-25T14:37:55.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:55 smithi150 ceph-mon[32504]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:55.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:55 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:55.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:55 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:55.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:55 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:55.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:55 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:37:55.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:55 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:37:55.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:55 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:37:55.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:55 smithi150 ceph-mon[32504]: Deploying daemon mds.g on smithi053 2024-04-25T14:37:55.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:55 smithi150 ceph-mon[32504]: mds.? [v2:172.21.15.183:6838/1554258915,v1:172.21.15.183:6839/1554258915] up:boot 2024-04-25T14:37:55.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:55 smithi150 ceph-mon[32504]: fsmap 9 up:standby 2024-04-25T14:37:55.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:55 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T14:37:55.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:55 smithi183 ceph-mon[32584]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:55.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:55 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:55.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:55 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:55.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:55 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:55.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:55 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:37:55.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:55 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:37:55.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:55 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:37:55.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:55 smithi183 ceph-mon[32584]: Deploying daemon mds.g on smithi053 2024-04-25T14:37:55.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:55 smithi183 ceph-mon[32584]: mds.? [v2:172.21.15.183:6838/1554258915,v1:172.21.15.183:6839/1554258915] up:boot 2024-04-25T14:37:55.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:55 smithi183 ceph-mon[32584]: fsmap 9 up:standby 2024-04-25T14:37:55.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:55 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T14:37:56.621 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:56 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/3685041510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:37:56.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:56 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/3685041510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:37:56.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:56 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/3685041510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:37:57.694 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:37:57.755 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:57 smithi053 ceph-mon[27773]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:57.882 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:57.881+0000 7f98e7c00640 1 -- 172.21.15.53:0/1068196813 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98e0105d60 msgr2=0x7f98e010c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:57.882 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:57.881+0000 7f98e7c00640 1 --2- 172.21.15.53:0/1068196813 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98e0105d60 0x7f98e010c5f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f98d8009900 tx=0x7f98d802a410 comp rx=0 tx=0).stop 2024-04-25T14:37:57.882 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:57.882+0000 7f98e7c00640 1 -- 172.21.15.53:0/1068196813 shutdown_connections 2024-04-25T14:37:57.882 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:57.882+0000 7f98e7c00640 1 --2- 172.21.15.53:0/1068196813 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98e0105d60 0x7f98e010c5f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:57.882 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:57.882+0000 7f98e7c00640 1 --2- 172.21.15.53:0/1068196813 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98e01053a0 0x7f98e0105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:57.882 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:57.882+0000 7f98e7c00640 1 --2- 172.21.15.53:0/1068196813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98e01041b0 0x7f98e01045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:57.882 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:57.882+0000 7f98e7c00640 1 -- 172.21.15.53:0/1068196813 >> 172.21.15.53:0/1068196813 conn(0x7f98e00ff740 msgr2=0x7f98e0101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:37:57.882 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:57.882+0000 7f98e7c00640 1 -- 172.21.15.53:0/1068196813 shutdown_connections 2024-04-25T14:37:57.882 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:57.882+0000 7f98e7c00640 1 -- 172.21.15.53:0/1068196813 wait complete. 2024-04-25T14:37:57.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:57.883+0000 7f98e7c00640 1 Processor -- start 2024-04-25T14:37:57.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:57.883+0000 7f98e7c00640 1 -- start start 2024-04-25T14:37:57.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:57.883+0000 7f98e7c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98e01041b0 0x7f98e019a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:57.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:57.883+0000 7f98e7c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98e01053a0 0x7f98e019a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:57.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:57.883+0000 7f98e7c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98e0105d60 0x7f98e01a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:57.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:57.883+0000 7f98e7c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f98e010fd10 con 0x7f98e01053a0 2024-04-25T14:37:57.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:57.883+0000 7f98e7c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f98e010fb90 con 0x7f98e0105d60 2024-04-25T14:37:57.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:57.883+0000 7f98e7c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f98e010fe90 con 0x7f98e01041b0 2024-04-25T14:37:57.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:57.883+0000 7f98dfe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98e01053a0 0x7f98e019a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:57.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:57.883+0000 7f98dfe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98e01053a0 0x7f98e019a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45672/0 (socket says 172.21.15.53:45672) 2024-04-25T14:37:57.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:57.883+0000 7f98dfe00640 1 -- 172.21.15.53:0/3266195345 learned_addr learned my addr 172.21.15.53:0/3266195345 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:37:57.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:57.884+0000 7f98e5c00640 1 --2- 172.21.15.53:0/3266195345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98e0105d60 0x7f98e01a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:57.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:57.884+0000 7f98e5200640 1 --2- 172.21.15.53:0/3266195345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98e01041b0 0x7f98e019a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:57.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:57.884+0000 7f98dfe00640 1 -- 172.21.15.53:0/3266195345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98e01041b0 msgr2=0x7f98e019a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:57.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:57.884+0000 7f98dfe00640 1 --2- 172.21.15.53:0/3266195345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98e01041b0 0x7f98e019a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:57.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:57.884+0000 7f98dfe00640 1 -- 172.21.15.53:0/3266195345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98e0105d60 msgr2=0x7f98e01a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:57.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:57.884+0000 7f98dfe00640 1 --2- 172.21.15.53:0/3266195345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98e0105d60 0x7f98e01a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:57.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:57.884+0000 7f98dfe00640 1 -- 172.21.15.53:0/3266195345 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e01a1fd0 con 0x7f98e01053a0 2024-04-25T14:37:57.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:57.884+0000 7f98dfe00640 1 --2- 172.21.15.53:0/3266195345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98e01053a0 0x7f98e019a8f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f98c800d950 tx=0x7f98c800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:37:57.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:57.884+0000 7f98dd600640 1 -- 172.21.15.53:0/3266195345 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c8017030 con 0x7f98e01053a0 2024-04-25T14:37:57.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:57.884+0000 7f98dd600640 1 -- 172.21.15.53:0/3266195345 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98c8004020 con 0x7f98e01053a0 2024-04-25T14:37:57.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:57.885+0000 7f98dd600640 1 -- 172.21.15.53:0/3266195345 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c80047a0 con 0x7f98e01053a0 2024-04-25T14:37:57.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:57.885+0000 7f98e7c00640 1 -- 172.21.15.53:0/3266195345 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98e01a22c0 con 0x7f98e01053a0 2024-04-25T14:37:57.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:57.885+0000 7f98e7c00640 1 -- 172.21.15.53:0/3266195345 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98e01a27c0 con 0x7f98e01053a0 2024-04-25T14:37:57.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:57.886+0000 7f98e7c00640 1 -- 172.21.15.53:0/3266195345 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98e010b4d0 con 0x7f98e01053a0 2024-04-25T14:37:57.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:57.889+0000 7f98dd600640 1 -- 172.21.15.53:0/3266195345 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f98c8009a30 con 0x7f98e01053a0 2024-04-25T14:37:57.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:57.891+0000 7f98dd600640 1 --2- 172.21.15.53:0/3266195345 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f98c40a3bb0 0x7f98c40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:37:57.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:57.891+0000 7f98dd600640 1 -- 172.21.15.53:0/3266195345 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f98c80cf9c0 con 0x7f98e01053a0 2024-04-25T14:37:57.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:57.891+0000 7f98e5200640 1 --2- 172.21.15.53:0/3266195345 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f98c40a3bb0 0x7f98c40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:37:57.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:57.892+0000 7f98e5200640 1 --2- 172.21.15.53:0/3266195345 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f98c40a3bb0 0x7f98c40a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f98e0105200 tx=0x7f98d0009670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:37:57.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:57.900+0000 7f98dd600640 1 -- 172.21.15.53:0/3266195345 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f98c809c7a0 con 0x7f98e01053a0 2024-04-25T14:37:57.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:57 smithi183 ceph-mon[32584]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:57.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:57 smithi150 ceph-mon[32504]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:58.014 INFO:journalctl@ceph.mds.g.smithi053.stdout:Apr 25 14:37:57 smithi053 podman[68928]: 2024-04-25 14:37:57.745151257 +0000 UTC m=+0.036339117 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T14:37:58.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:58.153+0000 7f98e7c00640 1 -- 172.21.15.53:0/3266195345 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98e002d5c0 con 0x7f98e01053a0 2024-04-25T14:37:58.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:58.155+0000 7f98dd600640 1 -- 172.21.15.53:0/3266195345 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8007 (secure 0 0 0) 0x7f98c809c0a0 con 0x7f98e01053a0 2024-04-25T14:37:58.155 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:37:58.155 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2966992472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2966992472},{"type":"v1","addr":"172.21.15.150:6833","nonce":2966992472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/94143643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":94143643},{"type":"v1","addr":"172.21.15.183:6833","nonce":94143643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24470,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/506682943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":506682943},{"type":"v1","addr":"172.21.15.53:6835","nonce":506682943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1369488844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1369488844},{"type":"v1","addr":"172.21.15.150:6835","nonce":1369488844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3786232682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3786232682},{"type":"v1","addr":"172.21.15.53:6837","nonce":3786232682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24532,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1189328602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1189328602},{"type":"v1","addr":"172.21.15.183:6835","nonce":1189328602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2344343627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2344343627},{"type":"v1","addr":"172.21.15.150:6837","nonce":2344343627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24544,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/1163336384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1163336384},{"type":"v1","addr":"172.21.15.183:6837","nonce":1163336384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24565,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/1554258915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1554258915},{"type":"v1","addr":"172.21.15.183:6839","nonce":1554258915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T14:37:58.155 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 10 2024-04-25T14:37:58.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:58.157+0000 7f98e7c00640 1 -- 172.21.15.53:0/3266195345 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f98c40a3bb0 msgr2=0x7f98c40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:58.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:58.157+0000 7f98e7c00640 1 --2- 172.21.15.53:0/3266195345 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f98c40a3bb0 0x7f98c40a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f98e0105200 tx=0x7f98d0009670 comp rx=0 tx=0).stop 2024-04-25T14:37:58.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:58.157+0000 7f98e7c00640 1 -- 172.21.15.53:0/3266195345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98e01053a0 msgr2=0x7f98e019a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:37:58.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:58.157+0000 7f98e7c00640 1 --2- 172.21.15.53:0/3266195345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98e01053a0 0x7f98e019a8f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f98c800d950 tx=0x7f98c800de10 comp rx=0 tx=0).stop 2024-04-25T14:37:58.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:58.157+0000 7f98e7c00640 1 -- 172.21.15.53:0/3266195345 shutdown_connections 2024-04-25T14:37:58.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:58.157+0000 7f98e7c00640 1 --2- 172.21.15.53:0/3266195345 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f98c40a3bb0 0x7f98c40a6070 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:58.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:58.157+0000 7f98e7c00640 1 --2- 172.21.15.53:0/3266195345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98e0105d60 0x7f98e01a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:58.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:58.157+0000 7f98e7c00640 1 --2- 172.21.15.53:0/3266195345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98e01053a0 0x7f98e019a8f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:58.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:58.157+0000 7f98e7c00640 1 --2- 172.21.15.53:0/3266195345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98e01041b0 0x7f98e019a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:37:58.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:58.157+0000 7f98e7c00640 1 -- 172.21.15.53:0/3266195345 >> 172.21.15.53:0/3266195345 conn(0x7f98e00ff740 msgr2=0x7f98e01086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:37:58.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:58.158+0000 7f98e7c00640 1 -- 172.21.15.53:0/3266195345 shutdown_connections 2024-04-25T14:37:58.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:37:58.158+0000 7f98e7c00640 1 -- 172.21.15.53:0/3266195345 wait complete. 2024-04-25T14:37:58.473 INFO:journalctl@ceph.mds.g.smithi053.stdout:Apr 25 14:37:58 smithi053 podman[68928]: 2024-04-25 14:37:58.107839454 +0000 UTC m=+0.399027343 container init 67aef1549e186d1f5ceee6e83d3e6a40111ed214391c2bc4eec3ac75699e56e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T14:37:58.474 INFO:journalctl@ceph.mds.g.smithi053.stdout:Apr 25 14:37:58 smithi053 podman[68928]: 2024-04-25 14:37:58.119851305 +0000 UTC m=+0.411039171 container start 67aef1549e186d1f5ceee6e83d3e6a40111ed214391c2bc4eec3ac75699e56e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-g, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-25T14:37:58.474 INFO:journalctl@ceph.mds.g.smithi053.stdout:Apr 25 14:37:58 smithi053 ceph-mds[68989]: -- 172.21.15.53:0/451135807 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a18daf1a40 con 0x55a18dac5000 2024-04-25T14:37:58.474 INFO:journalctl@ceph.mds.g.smithi053.stdout:Apr 25 14:37:58 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-g[68985]: starting mds.g at 2024-04-25T14:37:58.474 INFO:journalctl@ceph.mds.g.smithi053.stdout:Apr 25 14:37:58 smithi053 bash[68928]: 67aef1549e186d1f5ceee6e83d3e6a40111ed214391c2bc4eec3ac75699e56e0 2024-04-25T14:37:58.474 INFO:journalctl@ceph.mds.g.smithi053.stdout:Apr 25 14:37:58 smithi053 systemd[1]: Started Ceph mds.g for fd30e6ce-030f-11ef-bc93-c7b262605968. 2024-04-25T14:37:58.474 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:58 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/3266195345' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:37:58.474 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:58 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:58.474 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:58 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:58.475 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:58 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:58.475 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:58 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:37:58.475 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:58 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:37:58.740 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:58 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:37:58.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:58 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/3266195345' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:37:58.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:58 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:58.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:58 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:58.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:58 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:58.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:58 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:37:58.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:58 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:37:58.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:58 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:37:58.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:58 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/3266195345' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:37:58.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:58 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:58.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:58 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:58.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:58 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:37:58.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:58 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:37:58.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:58 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:37:58.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:58 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:37:59.624 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:59 smithi053 ceph-mon[27773]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:59.624 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:59 smithi053 ceph-mon[27773]: Deploying daemon mds.a on smithi053 2024-04-25T14:37:59.624 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:59 smithi053 ceph-mon[27773]: mds.? [v2:172.21.15.53:6838/570000540,v1:172.21.15.53:6839/570000540] up:boot 2024-04-25T14:37:59.624 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:59 smithi053 ceph-mon[27773]: fsmap 10 up:standby 2024-04-25T14:37:59.624 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:37:59 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T14:37:59.855 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:59 smithi183 ceph-mon[32584]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:59.855 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:59 smithi183 ceph-mon[32584]: Deploying daemon mds.a on smithi053 2024-04-25T14:37:59.855 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:59 smithi183 ceph-mon[32584]: mds.? [v2:172.21.15.53:6838/570000540,v1:172.21.15.53:6839/570000540] up:boot 2024-04-25T14:37:59.855 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:59 smithi183 ceph-mon[32584]: fsmap 10 up:standby 2024-04-25T14:37:59.855 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:37:59 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T14:37:59.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:59 smithi150 ceph-mon[32504]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:37:59.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:59 smithi150 ceph-mon[32504]: Deploying daemon mds.a on smithi053 2024-04-25T14:37:59.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:59 smithi150 ceph-mon[32504]: mds.? [v2:172.21.15.53:6838/570000540,v1:172.21.15.53:6839/570000540] up:boot 2024-04-25T14:37:59.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:59 smithi150 ceph-mon[32504]: fsmap 10 up:standby 2024-04-25T14:37:59.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:37:59 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T14:38:00.177 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:38:00.405 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.404+0000 7f5ff3200640 1 -- 172.21.15.53:0/299937762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ff40749b0 msgr2=0x7f5ff4074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:00.405 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.404+0000 7f5ff3200640 1 --2- 172.21.15.53:0/299937762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ff40749b0 0x7f5ff4074e30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4009be0 tx=0x7f5fe402a6c0 comp rx=0 tx=0).stop 2024-04-25T14:38:00.405 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.405+0000 7f5ff3200640 1 -- 172.21.15.53:0/299937762 shutdown_connections 2024-04-25T14:38:00.405 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.405+0000 7f5ff3200640 1 --2- 172.21.15.53:0/299937762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ff4109cb0 0x7f5ff410c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:00.405 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.405+0000 7f5ff3200640 1 --2- 172.21.15.53:0/299937762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ff40749b0 0x7f5ff4074e30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:00.405 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.405+0000 7f5ff3200640 1 --2- 172.21.15.53:0/299937762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ff4075f90 0x7f5ff40743e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:00.405 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.405+0000 7f5ff3200640 1 -- 172.21.15.53:0/299937762 >> 172.21.15.53:0/299937762 conn(0x7f5ff40fd360 msgr2=0x7f5ff40ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:00.405 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.405+0000 7f5ff3200640 1 -- 172.21.15.53:0/299937762 shutdown_connections 2024-04-25T14:38:00.405 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.405+0000 7f5ff3200640 1 -- 172.21.15.53:0/299937762 wait complete. 2024-04-25T14:38:00.406 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.406+0000 7f5ff3200640 1 Processor -- start 2024-04-25T14:38:00.406 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.406+0000 7f5ff3200640 1 -- start start 2024-04-25T14:38:00.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.407+0000 7f5ff3200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ff40749b0 0x7f5ff419a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:00.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.407+0000 7f5ff3200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ff4075f90 0x7f5ff419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:00.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.407+0000 7f5ff3200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ff4109cb0 0x7f5ff41a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:00.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.407+0000 7f5ff3200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff410fae0 con 0x7f5ff4109cb0 2024-04-25T14:38:00.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.407+0000 7f5ff3200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff410f960 con 0x7f5ff4075f90 2024-04-25T14:38:00.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.407+0000 7f5ff3200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff410fc60 con 0x7f5ff40749b0 2024-04-25T14:38:00.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.407+0000 7f5ff1e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ff40749b0 0x7f5ff419a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:00.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.407+0000 7f5ff1e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ff40749b0 0x7f5ff419a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:49960/0 (socket says 172.21.15.53:49960) 2024-04-25T14:38:00.409 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.407+0000 7f5ff1e00640 1 -- 172.21.15.53:0/3685066240 learned_addr learned my addr 172.21.15.53:0/3685066240 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:38:00.409 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.407+0000 7f5ff1400640 1 --2- 172.21.15.53:0/3685066240 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ff4075f90 0x7f5ff419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:00.409 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.407+0000 7f5ff2800640 1 --2- 172.21.15.53:0/3685066240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ff4109cb0 0x7f5ff41a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:00.409 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.407+0000 7f5ff1e00640 1 -- 172.21.15.53:0/3685066240 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ff4075f90 msgr2=0x7f5ff419a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:00.409 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.408+0000 7f5ff1e00640 1 --2- 172.21.15.53:0/3685066240 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ff4075f90 0x7f5ff419a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:00.409 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.408+0000 7f5ff1e00640 1 -- 172.21.15.53:0/3685066240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ff4109cb0 msgr2=0x7f5ff41a17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:00.409 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.408+0000 7f5ff1e00640 1 --2- 172.21.15.53:0/3685066240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ff4109cb0 0x7f5ff41a17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:00.409 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.408+0000 7f5ff1e00640 1 -- 172.21.15.53:0/3685066240 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff41a1e30 con 0x7f5ff40749b0 2024-04-25T14:38:00.409 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.408+0000 7f5ff1400640 1 --2- 172.21.15.53:0/3685066240 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ff4075f90 0x7f5ff419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:38:00.409 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.408+0000 7f5ff1e00640 1 --2- 172.21.15.53:0/3685066240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ff40749b0 0x7f5ff419a210 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00e9c0 tx=0x7f5fdc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:00.409 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.408+0000 7f5feaa00640 1 -- 172.21.15.53:0/3685066240 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc00cde0 con 0x7f5ff40749b0 2024-04-25T14:38:00.409 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.408+0000 7f5feaa00640 1 -- 172.21.15.53:0/3685066240 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fdc0035f0 con 0x7f5ff40749b0 2024-04-25T14:38:00.409 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.408+0000 7f5ff3200640 1 -- 172.21.15.53:0/3685066240 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ff41a2120 con 0x7f5ff40749b0 2024-04-25T14:38:00.409 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.408+0000 7f5feaa00640 1 -- 172.21.15.53:0/3685066240 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc010690 con 0x7f5ff40749b0 2024-04-25T14:38:00.409 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.409+0000 7f5ff3200640 1 -- 172.21.15.53:0/3685066240 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ff41a2640 con 0x7f5ff40749b0 2024-04-25T14:38:00.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.410+0000 7f5ff3200640 1 -- 172.21.15.53:0/3685066240 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ff41089e0 con 0x7f5ff40749b0 2024-04-25T14:38:00.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.412+0000 7f5feaa00640 1 -- 172.21.15.53:0/3685066240 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5fdc004000 con 0x7f5ff40749b0 2024-04-25T14:38:00.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.414+0000 7f5feaa00640 1 --2- 172.21.15.53:0/3685066240 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5fd80a3bb0 0x7f5fd80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:00.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.414+0000 7f5feaa00640 1 -- 172.21.15.53:0/3685066240 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f5fdc0d02f0 con 0x7f5ff40749b0 2024-04-25T14:38:00.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.414+0000 7f5ff1400640 1 --2- 172.21.15.53:0/3685066240 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5fd80a3bb0 0x7f5fd80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:00.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.415+0000 7f5ff1400640 1 --2- 172.21.15.53:0/3685066240 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5fd80a3bb0 0x7f5fd80a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5fe402acb0 tx=0x7f5fe4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:00.417 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.418+0000 7f5feaa00640 1 -- 172.21.15.53:0/3685066240 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5fdc009510 con 0x7f5ff40749b0 2024-04-25T14:38:00.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.671+0000 7f5ff3200640 1 -- 172.21.15.53:0/3685066240 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ff4062b50 con 0x7f5ff40749b0 2024-04-25T14:38:00.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.672+0000 7f5feaa00640 1 -- 172.21.15.53:0/3685066240 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8830 (secure 0 0 0) 0x7f5fdc09d0d0 con 0x7f5ff40749b0 2024-04-25T14:38:00.672 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:38:00.672 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2966992472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2966992472},{"type":"v1","addr":"172.21.15.150:6833","nonce":2966992472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/94143643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":94143643},{"type":"v1","addr":"172.21.15.183:6833","nonce":94143643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24470,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/506682943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":506682943},{"type":"v1","addr":"172.21.15.53:6835","nonce":506682943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1369488844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1369488844},{"type":"v1","addr":"172.21.15.150:6835","nonce":1369488844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3786232682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3786232682},{"type":"v1","addr":"172.21.15.53:6837","nonce":3786232682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24532,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1189328602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1189328602},{"type":"v1","addr":"172.21.15.183:6835","nonce":1189328602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2344343627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2344343627},{"type":"v1","addr":"172.21.15.150:6837","nonce":2344343627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24544,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/1163336384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1163336384},{"type":"v1","addr":"172.21.15.183:6837","nonce":1163336384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24560,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6839/570000540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":570000540},{"type":"v1","addr":"172.21.15.53:6839","nonce":570000540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24565,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/1554258915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1554258915},{"type":"v1","addr":"172.21.15.183:6839","nonce":1554258915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T14:38:00.672 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 11 2024-04-25T14:38:00.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.674+0000 7f5ff3200640 1 -- 172.21.15.53:0/3685066240 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5fd80a3bb0 msgr2=0x7f5fd80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:00.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.674+0000 7f5ff3200640 1 --2- 172.21.15.53:0/3685066240 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5fd80a3bb0 0x7f5fd80a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5fe402acb0 tx=0x7f5fe4005fd0 comp rx=0 tx=0).stop 2024-04-25T14:38:00.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.674+0000 7f5ff3200640 1 -- 172.21.15.53:0/3685066240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ff40749b0 msgr2=0x7f5ff419a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:00.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.674+0000 7f5ff3200640 1 --2- 172.21.15.53:0/3685066240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ff40749b0 0x7f5ff419a210 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00e9c0 tx=0x7f5fdc00ee80 comp rx=0 tx=0).stop 2024-04-25T14:38:00.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.674+0000 7f5ff3200640 1 -- 172.21.15.53:0/3685066240 shutdown_connections 2024-04-25T14:38:00.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.675+0000 7f5ff3200640 1 --2- 172.21.15.53:0/3685066240 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5fd80a3bb0 0x7f5fd80a6070 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:00.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.675+0000 7f5ff3200640 1 --2- 172.21.15.53:0/3685066240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ff4109cb0 0x7f5ff41a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:00.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.675+0000 7f5ff3200640 1 --2- 172.21.15.53:0/3685066240 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ff4075f90 0x7f5ff419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:00.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.675+0000 7f5ff3200640 1 --2- 172.21.15.53:0/3685066240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ff40749b0 0x7f5ff419a210 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:00.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.675+0000 7f5ff3200640 1 -- 172.21.15.53:0/3685066240 >> 172.21.15.53:0/3685066240 conn(0x7f5ff40fd360 msgr2=0x7f5ff4105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:00.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.675+0000 7f5ff3200640 1 -- 172.21.15.53:0/3685066240 shutdown_connections 2024-04-25T14:38:00.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:00.675+0000 7f5ff3200640 1 -- 172.21.15.53:0/3685066240 wait complete. 2024-04-25T14:38:01.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:01 smithi053 ceph-mon[27773]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:38:01.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:01 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/3685066240' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:38:01.783 INFO:journalctl@ceph.mds.a.smithi053.stdout:Apr 25 14:38:01 smithi053 systemd[1]: Starting Ceph mds.a for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:38:01.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:01 smithi183 ceph-mon[32584]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:38:01.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:01 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/3685066240' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:38:01.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:01 smithi150 ceph-mon[32504]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:38:01.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:01 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/3685066240' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:38:02.205 INFO:journalctl@ceph.mds.a.smithi053.stdout:Apr 25 14:38:01 smithi053 podman[69318]: 2024-04-25 14:38:01.804537372 +0000 UTC m=+0.039999832 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T14:38:02.205 INFO:journalctl@ceph.mds.a.smithi053.stdout:Apr 25 14:38:01 smithi053 podman[69318]: 2024-04-25 14:38:01.958284959 +0000 UTC m=+0.193747456 container create 39903d2c0c72d2e30a91ee177c1e850fe7d5415ddee8343ac05e0e5abd7ebe0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-25T14:38:02.205 INFO:journalctl@ceph.mds.a.smithi053.stdout:Apr 25 14:38:02 smithi053 podman[69318]: 2024-04-25 14:38:02.203763836 +0000 UTC m=+0.439226357 container init 39903d2c0c72d2e30a91ee177c1e850fe7d5415ddee8343ac05e0e5abd7ebe0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-a, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-25T14:38:02.533 INFO:journalctl@ceph.mds.a.smithi053.stdout:Apr 25 14:38:02 smithi053 podman[69318]: 2024-04-25 14:38:02.215350005 +0000 UTC m=+0.450812478 container start 39903d2c0c72d2e30a91ee177c1e850fe7d5415ddee8343ac05e0e5abd7ebe0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-25T14:38:02.533 INFO:journalctl@ceph.mds.a.smithi053.stdout:Apr 25 14:38:02 smithi053 ceph-mds[69336]: -- 172.21.15.53:0/3482848913 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b7f8bbfa40 con 0x55b7f8b92c00 2024-04-25T14:38:02.533 INFO:journalctl@ceph.mds.a.smithi053.stdout:Apr 25 14:38:02 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-a[69332]: starting mds.a at 2024-04-25T14:38:02.533 INFO:journalctl@ceph.mds.a.smithi053.stdout:Apr 25 14:38:02 smithi053 bash[69318]: 39903d2c0c72d2e30a91ee177c1e850fe7d5415ddee8343ac05e0e5abd7ebe0e 2024-04-25T14:38:02.533 INFO:journalctl@ceph.mds.a.smithi053.stdout:Apr 25 14:38:02 smithi053 systemd[1]: Started Ceph mds.a for fd30e6ce-030f-11ef-bc93-c7b262605968. 2024-04-25T14:38:02.696 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:38:02.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:02.925+0000 7fde51c00640 1 -- 172.21.15.53:0/1708020261 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde4c1053a0 msgr2=0x7fde4c105820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:02.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:02.925+0000 7fde51c00640 1 --2- 172.21.15.53:0/1708020261 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde4c1053a0 0x7fde4c105820 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fde38009900 tx=0x7fde3802a3f0 comp rx=0 tx=0).stop 2024-04-25T14:38:02.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:02.925+0000 7fde51c00640 1 -- 172.21.15.53:0/1708020261 shutdown_connections 2024-04-25T14:38:02.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:02.925+0000 7fde51c00640 1 --2- 172.21.15.53:0/1708020261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde4c105d60 0x7fde4c10c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:02.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:02.925+0000 7fde51c00640 1 --2- 172.21.15.53:0/1708020261 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde4c1053a0 0x7fde4c105820 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:02.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:02.925+0000 7fde51c00640 1 --2- 172.21.15.53:0/1708020261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde4c1041b0 0x7fde4c1045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:02.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:02.925+0000 7fde51c00640 1 -- 172.21.15.53:0/1708020261 >> 172.21.15.53:0/1708020261 conn(0x7fde4c0ff740 msgr2=0x7fde4c101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:02.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:02.925+0000 7fde51c00640 1 -- 172.21.15.53:0/1708020261 shutdown_connections 2024-04-25T14:38:02.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:02.925+0000 7fde51c00640 1 -- 172.21.15.53:0/1708020261 wait complete. 2024-04-25T14:38:02.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:02.926+0000 7fde51c00640 1 Processor -- start 2024-04-25T14:38:02.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:02.927+0000 7fde51c00640 1 -- start start 2024-04-25T14:38:02.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:02.927+0000 7fde51c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde4c1041b0 0x7fde4c19a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:02.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:02.927+0000 7fde51c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde4c1053a0 0x7fde4c19a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:02.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:02.927+0000 7fde51c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde4c105d60 0x7fde4c1a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:02.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:02.927+0000 7fde51c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fde4c10fd40 con 0x7fde4c105d60 2024-04-25T14:38:02.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:02.927+0000 7fde51c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fde4c10fbc0 con 0x7fde4c1053a0 2024-04-25T14:38:02.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:02.927+0000 7fde51c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fde4c10fec0 con 0x7fde4c1041b0 2024-04-25T14:38:02.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:02.927+0000 7fde4a000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde4c1053a0 0x7fde4c19a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:02.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:02.927+0000 7fde4a000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde4c1053a0 0x7fde4c19a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.53:52674/0 (socket says 172.21.15.53:52674) 2024-04-25T14:38:02.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:02.927+0000 7fde4a000640 1 -- 172.21.15.53:0/728668026 learned_addr learned my addr 172.21.15.53:0/728668026 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:38:02.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:02.927+0000 7fde4b400640 1 --2- 172.21.15.53:0/728668026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde4c105d60 0x7fde4c1a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:02.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:02.927+0000 7fde4aa00640 1 --2- 172.21.15.53:0/728668026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde4c1041b0 0x7fde4c19a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:02.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:02.928+0000 7fde4a000640 1 -- 172.21.15.53:0/728668026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde4c1041b0 msgr2=0x7fde4c19a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:02.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:02.928+0000 7fde4a000640 1 --2- 172.21.15.53:0/728668026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde4c1041b0 0x7fde4c19a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:02.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:02.928+0000 7fde4a000640 1 -- 172.21.15.53:0/728668026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde4c105d60 msgr2=0x7fde4c1a1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:02.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:02.928+0000 7fde4a000640 1 --2- 172.21.15.53:0/728668026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde4c105d60 0x7fde4c1a1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:02.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:02.928+0000 7fde4a000640 1 -- 172.21.15.53:0/728668026 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde4c1a1fc0 con 0x7fde4c1053a0 2024-04-25T14:38:02.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:02.928+0000 7fde4a000640 1 --2- 172.21.15.53:0/728668026 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde4c1053a0 0x7fde4c19a8e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fde38005ab0 tx=0x7fde38003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:02.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:02.928+0000 7fde43400640 1 -- 172.21.15.53:0/728668026 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3803f070 con 0x7fde4c1053a0 2024-04-25T14:38:02.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:02.928+0000 7fde43400640 1 -- 172.21.15.53:0/728668026 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde3803b460 con 0x7fde4c1053a0 2024-04-25T14:38:02.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:02.928+0000 7fde43400640 1 -- 172.21.15.53:0/728668026 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde38035030 con 0x7fde4c1053a0 2024-04-25T14:38:02.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:02.928+0000 7fde51c00640 1 -- 172.21.15.53:0/728668026 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde4c1a2250 con 0x7fde4c1053a0 2024-04-25T14:38:02.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:02.928+0000 7fde51c00640 1 -- 172.21.15.53:0/728668026 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde4c1a2740 con 0x7fde4c1053a0 2024-04-25T14:38:02.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:02.930+0000 7fde51c00640 1 -- 172.21.15.53:0/728668026 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde4c10b4d0 con 0x7fde4c1053a0 2024-04-25T14:38:02.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:02.931+0000 7fde43400640 1 -- 172.21.15.53:0/728668026 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fde38042070 con 0x7fde4c1053a0 2024-04-25T14:38:02.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:02.933+0000 7fde43400640 1 --2- 172.21.15.53:0/728668026 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fde300a3bb0 0x7fde300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:02.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:02.933+0000 7fde43400640 1 -- 172.21.15.53:0/728668026 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fde380f0830 con 0x7fde4c1053a0 2024-04-25T14:38:02.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:02.933+0000 7fde4aa00640 1 --2- 172.21.15.53:0/728668026 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fde300a3bb0 0x7fde300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:02.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:02.934+0000 7fde4aa00640 1 --2- 172.21.15.53:0/728668026 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fde300a3bb0 0x7fde300a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fde3c005fd0 tx=0x7fde3c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:02.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:02.937+0000 7fde43400640 1 -- 172.21.15.53:0/728668026 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fde380bcfc0 con 0x7fde4c1053a0 2024-04-25T14:38:03.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:03.185+0000 7fde51c00640 1 -- 172.21.15.53:0/728668026 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde4c062b50 con 0x7fde4c1053a0 2024-04-25T14:38:03.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:03.187+0000 7fde43400640 1 -- 172.21.15.53:0/728668026 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9650 (secure 0 0 0) 0x7fde380c12c0 con 0x7fde4c1053a0 2024-04-25T14:38:03.187 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:38:03.188 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14460,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6841/41110727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6840","nonce":41110727},{"type":"v1","addr":"172.21.15.53:6841","nonce":41110727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2966992472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2966992472},{"type":"v1","addr":"172.21.15.150:6833","nonce":2966992472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/94143643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":94143643},{"type":"v1","addr":"172.21.15.183:6833","nonce":94143643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24470,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/506682943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":506682943},{"type":"v1","addr":"172.21.15.53:6835","nonce":506682943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1369488844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1369488844},{"type":"v1","addr":"172.21.15.150:6835","nonce":1369488844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3786232682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3786232682},{"type":"v1","addr":"172.21.15.53:6837","nonce":3786232682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24532,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1189328602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1189328602},{"type":"v1","addr":"172.21.15.183:6835","nonce":1189328602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2344343627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2344343627},{"type":"v1","addr":"172.21.15.150:6837","nonce":2344343627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24544,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/1163336384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1163336384},{"type":"v1","addr":"172.21.15.183:6837","nonce":1163336384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24560,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6839/570000540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":570000540},{"type":"v1","addr":"172.21.15.53:6839","nonce":570000540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24565,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/1554258915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1554258915},{"type":"v1","addr":"172.21.15.183:6839","nonce":1554258915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T14:38:03.188 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 12 2024-04-25T14:38:03.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:03.189+0000 7fde51c00640 1 -- 172.21.15.53:0/728668026 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fde300a3bb0 msgr2=0x7fde300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:03.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:03.189+0000 7fde51c00640 1 --2- 172.21.15.53:0/728668026 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fde300a3bb0 0x7fde300a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fde3c005fd0 tx=0x7fde3c00b410 comp rx=0 tx=0).stop 2024-04-25T14:38:03.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:03.189+0000 7fde51c00640 1 -- 172.21.15.53:0/728668026 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde4c1053a0 msgr2=0x7fde4c19a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:03.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:03.189+0000 7fde51c00640 1 --2- 172.21.15.53:0/728668026 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde4c1053a0 0x7fde4c19a8e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fde38005ab0 tx=0x7fde38003cd0 comp rx=0 tx=0).stop 2024-04-25T14:38:03.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:03.189+0000 7fde51c00640 1 -- 172.21.15.53:0/728668026 shutdown_connections 2024-04-25T14:38:03.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:03.189+0000 7fde51c00640 1 --2- 172.21.15.53:0/728668026 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fde300a3bb0 0x7fde300a6070 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:03.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:03.189+0000 7fde51c00640 1 --2- 172.21.15.53:0/728668026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde4c105d60 0x7fde4c1a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:03.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:03.189+0000 7fde51c00640 1 --2- 172.21.15.53:0/728668026 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde4c1053a0 0x7fde4c19a8e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:03.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:03.189+0000 7fde51c00640 1 --2- 172.21.15.53:0/728668026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde4c1041b0 0x7fde4c19a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:03.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:03.189+0000 7fde51c00640 1 -- 172.21.15.53:0/728668026 >> 172.21.15.53:0/728668026 conn(0x7fde4c0ff740 msgr2=0x7fde4c1086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:03.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:03.190+0000 7fde51c00640 1 -- 172.21.15.53:0/728668026 shutdown_connections 2024-04-25T14:38:03.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:03.190+0000 7fde51c00640 1 -- 172.21.15.53:0/728668026 wait complete. 2024-04-25T14:38:03.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:03 smithi053 ceph-mon[27773]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:38:03.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:03 smithi053 ceph-mon[27773]: mds.? [v2:172.21.15.53:6840/41110727,v1:172.21.15.53:6841/41110727] up:boot 2024-04-25T14:38:03.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:03 smithi053 ceph-mon[27773]: fsmap 11 up:standby 2024-04-25T14:38:03.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:03 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T14:38:03.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:03 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:03.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:03 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:03.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:03 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:03.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:03 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:38:03.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:03 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:38:03.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:03 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:38:03.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:03 smithi053 ceph-mon[27773]: Deploying daemon mds.b on smithi150 2024-04-25T14:38:03.784 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:03 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/728668026' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:38:03.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:03 smithi150 ceph-mon[32504]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:38:03.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:03 smithi150 ceph-mon[32504]: mds.? [v2:172.21.15.53:6840/41110727,v1:172.21.15.53:6841/41110727] up:boot 2024-04-25T14:38:03.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:03 smithi150 ceph-mon[32504]: fsmap 11 up:standby 2024-04-25T14:38:03.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:03 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T14:38:03.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:03 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:03.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:03 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:03.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:03 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:03.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:03 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:38:03.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:03 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:38:03.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:03 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:38:03.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:03 smithi150 ceph-mon[32504]: Deploying daemon mds.b on smithi150 2024-04-25T14:38:03.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:03 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/728668026' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:38:03.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:03 smithi183 ceph-mon[32584]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:38:03.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:03 smithi183 ceph-mon[32584]: mds.? [v2:172.21.15.53:6840/41110727,v1:172.21.15.53:6841/41110727] up:boot 2024-04-25T14:38:03.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:03 smithi183 ceph-mon[32584]: fsmap 11 up:standby 2024-04-25T14:38:03.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:03 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T14:38:03.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:03 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:03.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:03 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:03.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:03 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:03.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:03 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:38:03.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:03 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:38:03.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:03 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:38:03.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:03 smithi183 ceph-mon[32584]: Deploying daemon mds.b on smithi150 2024-04-25T14:38:03.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:03 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/728668026' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:38:04.897 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 25 14:38:04 smithi150 systemd[1]: Starting Ceph mds.b for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:38:05.188 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 25 14:38:04 smithi150 podman[62896]: 2024-04-25 14:38:04.800947409 +0000 UTC m=+0.017828822 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T14:38:05.188 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 25 14:38:04 smithi150 podman[62896]: 2024-04-25 14:38:04.902706961 +0000 UTC m=+0.119588349 container create 835db8e020fa3e17336f5ae2c39735bae50f67950623744fd55403d26ad17869 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T14:38:05.188 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 25 14:38:05 smithi150 podman[62896]: 2024-04-25 14:38:05.188507174 +0000 UTC m=+0.405388563 container init 835db8e020fa3e17336f5ae2c39735bae50f67950623744fd55403d26ad17869 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=) 2024-04-25T14:38:05.211 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:38:05.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.435+0000 7eff18a00640 1 -- 172.21.15.53:0/3296695975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff1410e3c0 msgr2=0x7eff141107b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:05.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.435+0000 7eff18a00640 1 --2- 172.21.15.53:0/3296695975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff1410e3c0 0x7eff141107b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7eff0c009c10 tx=0x7eff0c02a6e0 comp rx=0 tx=0).stop 2024-04-25T14:38:05.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.435+0000 7eff18a00640 1 -- 172.21.15.53:0/3296695975 shutdown_connections 2024-04-25T14:38:05.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.435+0000 7eff18a00640 1 --2- 172.21.15.53:0/3296695975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff1410e3c0 0x7eff141107b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:05.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.435+0000 7eff18a00640 1 --2- 172.21.15.53:0/3296695975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff140749b0 0x7eff14074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:05.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.435+0000 7eff18a00640 1 --2- 172.21.15.53:0/3296695975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff14075f90 0x7eff140743e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:05.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.435+0000 7eff18a00640 1 -- 172.21.15.53:0/3296695975 >> 172.21.15.53:0/3296695975 conn(0x7eff140fd360 msgr2=0x7eff140ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:05.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.435+0000 7eff18a00640 1 -- 172.21.15.53:0/3296695975 shutdown_connections 2024-04-25T14:38:05.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.435+0000 7eff18a00640 1 -- 172.21.15.53:0/3296695975 wait complete. 2024-04-25T14:38:05.438 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.436+0000 7eff18a00640 1 Processor -- start 2024-04-25T14:38:05.438 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.437+0000 7eff18a00640 1 -- start start 2024-04-25T14:38:05.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.437+0000 7eff18a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff140749b0 0x7eff1419e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:05.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.437+0000 7eff18a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff14075f90 0x7eff1419eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:05.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.437+0000 7eff18a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff1410e3c0 0x7eff141a5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:05.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.437+0000 7eff18a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7eff14113f30 con 0x7eff140749b0 2024-04-25T14:38:05.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.437+0000 7eff18a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7eff14113db0 con 0x7eff14075f90 2024-04-25T14:38:05.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.437+0000 7eff18a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7eff141140b0 con 0x7eff1410e3c0 2024-04-25T14:38:05.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.437+0000 7eff12800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff140749b0 0x7eff1419e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:05.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.437+0000 7eff12800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff140749b0 0x7eff1419e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37586/0 (socket says 172.21.15.53:37586) 2024-04-25T14:38:05.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.437+0000 7eff12800640 1 -- 172.21.15.53:0/2891635950 learned_addr learned my addr 172.21.15.53:0/2891635950 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:38:05.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.437+0000 7eff09e00640 1 --2- 172.21.15.53:0/2891635950 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff14075f90 0x7eff1419eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:05.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.437+0000 7eff13200640 1 --2- 172.21.15.53:0/2891635950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff1410e3c0 0x7eff141a5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:05.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.438+0000 7eff12800640 1 -- 172.21.15.53:0/2891635950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff1410e3c0 msgr2=0x7eff141a5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:05.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.438+0000 7eff12800640 1 --2- 172.21.15.53:0/2891635950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff1410e3c0 0x7eff141a5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:05.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.438+0000 7eff12800640 1 -- 172.21.15.53:0/2891635950 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff14075f90 msgr2=0x7eff1419eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:05.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.438+0000 7eff12800640 1 --2- 172.21.15.53:0/2891635950 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff14075f90 0x7eff1419eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:05.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.438+0000 7eff12800640 1 -- 172.21.15.53:0/2891635950 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff141a6210 con 0x7eff140749b0 2024-04-25T14:38:05.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.438+0000 7eff12800640 1 --2- 172.21.15.53:0/2891635950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff140749b0 0x7eff1419e5f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7eff0400e9c0 tx=0x7eff0400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:05.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.438+0000 7eff0be00640 1 -- 172.21.15.53:0/2891635950 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0400ccc0 con 0x7eff140749b0 2024-04-25T14:38:05.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.439+0000 7eff18a00640 1 -- 172.21.15.53:0/2891635950 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff141a6500 con 0x7eff140749b0 2024-04-25T14:38:05.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.439+0000 7eff0be00640 1 -- 172.21.15.53:0/2891635950 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff040054d0 con 0x7eff140749b0 2024-04-25T14:38:05.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.439+0000 7eff0be00640 1 -- 172.21.15.53:0/2891635950 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff04010430 con 0x7eff140749b0 2024-04-25T14:38:05.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.439+0000 7eff18a00640 1 -- 172.21.15.53:0/2891635950 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff141a6aa0 con 0x7eff140749b0 2024-04-25T14:38:05.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.440+0000 7eff18a00640 1 -- 172.21.15.53:0/2891635950 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff14104480 con 0x7eff140749b0 2024-04-25T14:38:05.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.442+0000 7eff0be00640 1 -- 172.21.15.53:0/2891635950 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7eff04004f80 con 0x7eff140749b0 2024-04-25T14:38:05.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.444+0000 7eff0be00640 1 --2- 172.21.15.53:0/2891635950 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7efef80a3bb0 0x7efef80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:05.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.444+0000 7eff0be00640 1 -- 172.21.15.53:0/2891635950 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7eff040d0410 con 0x7eff140749b0 2024-04-25T14:38:05.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.444+0000 7eff09e00640 1 --2- 172.21.15.53:0/2891635950 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7efef80a3bb0 0x7efef80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:05.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.445+0000 7eff09e00640 1 --2- 172.21.15.53:0/2891635950 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7efef80a3bb0 0x7efef80a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7eff1419fb00 tx=0x7efefc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:05.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.448+0000 7eff0be00640 1 -- 172.21.15.53:0/2891635950 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7eff04005270 con 0x7eff140749b0 2024-04-25T14:38:05.460 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 25 14:38:05 smithi150 podman[62896]: 2024-04-25 14:38:05.191554203 +0000 UTC m=+0.408435592 container start 835db8e020fa3e17336f5ae2c39735bae50f67950623744fd55403d26ad17869 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-b, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-25T14:38:05.460 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 25 14:38:05 smithi150 ceph-mds[62915]: -- 172.21.15.150:0/2877605121 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564e2db11a40 con 0x564e2dae5000 2024-04-25T14:38:05.460 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 25 14:38:05 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-b[62911]: starting mds.b at 2024-04-25T14:38:05.461 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 25 14:38:05 smithi150 bash[62896]: 835db8e020fa3e17336f5ae2c39735bae50f67950623744fd55403d26ad17869 2024-04-25T14:38:05.461 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 25 14:38:05 smithi150 systemd[1]: Started Ceph mds.b for fd30e6ce-030f-11ef-bc93-c7b262605968. 2024-04-25T14:38:05.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.703+0000 7eff18a00640 1 -- 172.21.15.53:0/2891635950 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7eff1402d5c0 con 0x7eff140749b0 2024-04-25T14:38:05.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.704+0000 7eff0be00640 1 -- 172.21.15.53:0/2891635950 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10476 (secure 0 0 0) 0x7eff0409d1f0 con 0x7eff140749b0 2024-04-25T14:38:05.707 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:38:05.707 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14460,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6841/41110727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6840","nonce":41110727},{"type":"v1","addr":"172.21.15.53:6841","nonce":41110727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2966992472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2966992472},{"type":"v1","addr":"172.21.15.150:6833","nonce":2966992472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/94143643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":94143643},{"type":"v1","addr":"172.21.15.183:6833","nonce":94143643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24470,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/506682943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":506682943},{"type":"v1","addr":"172.21.15.53:6835","nonce":506682943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1369488844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1369488844},{"type":"v1","addr":"172.21.15.150:6835","nonce":1369488844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3786232682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3786232682},{"type":"v1","addr":"172.21.15.53:6837","nonce":3786232682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24532,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1189328602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1189328602},{"type":"v1","addr":"172.21.15.183:6835","nonce":1189328602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2344343627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2344343627},{"type":"v1","addr":"172.21.15.150:6837","nonce":2344343627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24544,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/1163336384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1163336384},{"type":"v1","addr":"172.21.15.183:6837","nonce":1163336384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24560,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6839/570000540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":570000540},{"type":"v1","addr":"172.21.15.53:6839","nonce":570000540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24565,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/1554258915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1554258915},{"type":"v1","addr":"172.21.15.183:6839","nonce":1554258915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/149573616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":149573616},{"type":"v1","addr":"172.21.15.150:6839","nonce":149573616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-25T14:38:05.708 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 13 2024-04-25T14:38:05.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.707+0000 7eff18a00640 1 -- 172.21.15.53:0/2891635950 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7efef80a3bb0 msgr2=0x7efef80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:05.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.707+0000 7eff18a00640 1 --2- 172.21.15.53:0/2891635950 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7efef80a3bb0 0x7efef80a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7eff1419fb00 tx=0x7efefc005f90 comp rx=0 tx=0).stop 2024-04-25T14:38:05.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.707+0000 7eff18a00640 1 -- 172.21.15.53:0/2891635950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff140749b0 msgr2=0x7eff1419e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:05.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.707+0000 7eff18a00640 1 --2- 172.21.15.53:0/2891635950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff140749b0 0x7eff1419e5f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7eff0400e9c0 tx=0x7eff0400ee80 comp rx=0 tx=0).stop 2024-04-25T14:38:05.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.707+0000 7eff18a00640 1 -- 172.21.15.53:0/2891635950 shutdown_connections 2024-04-25T14:38:05.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.707+0000 7eff18a00640 1 --2- 172.21.15.53:0/2891635950 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7efef80a3bb0 0x7efef80a6070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:05.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.707+0000 7eff18a00640 1 --2- 172.21.15.53:0/2891635950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff1410e3c0 0x7eff141a5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:05.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.707+0000 7eff18a00640 1 --2- 172.21.15.53:0/2891635950 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff14075f90 0x7eff1419eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:05.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.707+0000 7eff18a00640 1 --2- 172.21.15.53:0/2891635950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff140749b0 0x7eff1419e5f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:05.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.707+0000 7eff18a00640 1 -- 172.21.15.53:0/2891635950 >> 172.21.15.53:0/2891635950 conn(0x7eff140fd360 msgr2=0x7eff14101650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:05.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.707+0000 7eff18a00640 1 -- 172.21.15.53:0/2891635950 shutdown_connections 2024-04-25T14:38:05.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.707+0000 7eff18a00640 1 -- 172.21.15.53:0/2891635950 wait complete. 2024-04-25T14:38:05.724 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:05 smithi053 ceph-mon[27773]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:38:05.724 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:05 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:05.725 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:05 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:05.725 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:05 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:05.725 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:05 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:05.725 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:05 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:05.728 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:38:05.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.907+0000 7fa24d400640 1 -- 172.21.15.53:0/60848482 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2481041d0 msgr2=0x7fa2481045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:05.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.907+0000 7fa24d400640 1 --2- 172.21.15.53:0/60848482 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2481041d0 0x7fa2481045f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa238009900 tx=0x7fa23802a410 comp rx=0 tx=0).stop 2024-04-25T14:38:05.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.907+0000 7fa24d400640 1 -- 172.21.15.53:0/60848482 shutdown_connections 2024-04-25T14:38:05.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.907+0000 7fa24d400640 1 --2- 172.21.15.53:0/60848482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa248105d80 0x7fa24810c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:05.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.907+0000 7fa24d400640 1 --2- 172.21.15.53:0/60848482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2481053c0 0x7fa248105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:05.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.907+0000 7fa24d400640 1 --2- 172.21.15.53:0/60848482 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2481041d0 0x7fa2481045f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:05.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.907+0000 7fa24d400640 1 -- 172.21.15.53:0/60848482 >> 172.21.15.53:0/60848482 conn(0x7fa2480ff740 msgr2=0x7fa248101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:05.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.907+0000 7fa24d400640 1 -- 172.21.15.53:0/60848482 shutdown_connections 2024-04-25T14:38:05.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.907+0000 7fa24d400640 1 -- 172.21.15.53:0/60848482 wait complete. 2024-04-25T14:38:05.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.908+0000 7fa24d400640 1 Processor -- start 2024-04-25T14:38:05.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.908+0000 7fa24d400640 1 -- start start 2024-04-25T14:38:05.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.908+0000 7fa24d400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2481041d0 0x7fa24819a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:05.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.909+0000 7fa24d400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2481053c0 0x7fa24819a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:05.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.909+0000 7fa24d400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa248105d80 0x7fa2481a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:05.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.909+0000 7fa24d400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa24810fd60 con 0x7fa2481041d0 2024-04-25T14:38:05.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.909+0000 7fa24d400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa24810fbe0 con 0x7fa2481053c0 2024-04-25T14:38:05.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.909+0000 7fa24d400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa24810fee0 con 0x7fa248105d80 2024-04-25T14:38:05.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.909+0000 7fa23de00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2481053c0 0x7fa24819a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:05.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.909+0000 7fa23de00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2481053c0 0x7fa24819a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.53:52710/0 (socket says 172.21.15.53:52710) 2024-04-25T14:38:05.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.909+0000 7fa23de00640 1 -- 172.21.15.53:0/2982500102 learned_addr learned my addr 172.21.15.53:0/2982500102 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:38:05.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.909+0000 7fa247200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa248105d80 0x7fa2481a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:05.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.909+0000 7fa246800640 1 --2- 172.21.15.53:0/2982500102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2481041d0 0x7fa24819a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:05.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.909+0000 7fa23de00640 1 -- 172.21.15.53:0/2982500102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa248105d80 msgr2=0x7fa2481a19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:05.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.909+0000 7fa23de00640 1 --2- 172.21.15.53:0/2982500102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa248105d80 0x7fa2481a19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:05.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.909+0000 7fa23de00640 1 -- 172.21.15.53:0/2982500102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2481041d0 msgr2=0x7fa24819a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:05.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.909+0000 7fa23de00640 1 --2- 172.21.15.53:0/2982500102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2481041d0 0x7fa24819a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:05.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.909+0000 7fa23de00640 1 -- 172.21.15.53:0/2982500102 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2481a2030 con 0x7fa2481053c0 2024-04-25T14:38:05.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.909+0000 7fa23de00640 1 --2- 172.21.15.53:0/2982500102 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2481053c0 0x7fa24819a950 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa23000e9f0 tx=0x7fa23000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:05.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.910+0000 7fa23fe00640 1 -- 172.21.15.53:0/2982500102 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23000cd80 con 0x7fa2481053c0 2024-04-25T14:38:05.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.910+0000 7fa23fe00640 1 -- 172.21.15.53:0/2982500102 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2300108e0 con 0x7fa2481053c0 2024-04-25T14:38:05.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.910+0000 7fa24d400640 1 -- 172.21.15.53:0/2982500102 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2481a2320 con 0x7fa2481053c0 2024-04-25T14:38:05.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.910+0000 7fa23fe00640 1 -- 172.21.15.53:0/2982500102 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa230018980 con 0x7fa2481053c0 2024-04-25T14:38:05.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.910+0000 7fa24d400640 1 -- 172.21.15.53:0/2982500102 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2481a2870 con 0x7fa2481053c0 2024-04-25T14:38:05.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.911+0000 7fa24d400640 1 -- 172.21.15.53:0/2982500102 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa24810b570 con 0x7fa2481053c0 2024-04-25T14:38:05.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.913+0000 7fa23fe00640 1 -- 172.21.15.53:0/2982500102 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa230010430 con 0x7fa2481053c0 2024-04-25T14:38:05.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.914+0000 7fa23fe00640 1 --2- 172.21.15.53:0/2982500102 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fa22c0a3bb0 0x7fa22c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:05.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.914+0000 7fa23fe00640 1 -- 172.21.15.53:0/2982500102 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fa2300d02d0 con 0x7fa2481053c0 2024-04-25T14:38:05.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.915+0000 7fa246800640 1 --2- 172.21.15.53:0/2982500102 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fa22c0a3bb0 0x7fa22c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:05.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.915+0000 7fa246800640 1 --2- 172.21.15.53:0/2982500102 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fa22c0a3bb0 0x7fa22c0a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa238005ab0 tx=0x7fa238005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:05.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:05.918+0000 7fa23fe00640 1 -- 172.21.15.53:0/2982500102 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa23009ca60 con 0x7fa2481053c0 2024-04-25T14:38:05.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:05 smithi183 ceph-mon[32584]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:38:05.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:05 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:05.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:05 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:05.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:05 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:05.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:05 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:05.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:05 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:05.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:05 smithi150 ceph-mon[32504]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:38:05.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:05 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:05.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:05 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:05.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:05 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:05.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:05 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:05.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:05 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:06.126 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.126+0000 7fa24d400640 1 -- 172.21.15.53:0/2982500102 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa248062b50 con 0x7fa2481053c0 2024-04-25T14:38:06.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.127+0000 7fa23fe00640 1 -- 172.21.15.53:0/2982500102 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+18408 (secure 0 0 0) 0x7fa2300a0d60 con 0x7fa2481053c0 2024-04-25T14:38:06.128 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:38:06.129 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":84,"fsid":"fd30e6ce-030f-11ef-bc93-c7b262605968","created":"2024-04-25T14:28:54.330326+0000","modified":"2024-04-25T14:37:11.928831+0000","last_up_change":"2024-04-25T14:37:10.921148+0000","last_in_change":"2024-04-25T14:36:49.566459+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:33:02.338115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aecfbaf0-39da-4749-a49b-c7205dabd4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6803","nonce":464176015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6805","nonce":464176015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6809","nonce":464176015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6807","nonce":464176015}]},"public_addr":"172.21.15.53:6803/464176015","cluster_addr":"172.21.15.53:6805/464176015","heartbeat_back_addr":"172.21.15.53:6809/464176015","heartbeat_front_addr":"172.21.15.53:6807/464176015","state":["exists","up"]},{"osd":1,"uuid":"2c4a21ae-c43e-4839-8a56-c3fc63aeb7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6801","nonce":1304621982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6803","nonce":1304621982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6807","nonce":1304621982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6805","nonce":1304621982}]},"public_addr":"172.21.15.150:6801/1304621982","cluster_addr":"172.21.15.150:6803/1304621982","heartbeat_back_addr":"172.21.15.150:6807/1304621982","heartbeat_front_addr":"172.21.15.150:6805/1304621982","state":["exists","up"]},{"osd":2,"uuid":"e8effd43-adc0-472a-9eed-a9273b06dc0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6801","nonce":574509311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6803","nonce":574509311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6807","nonce":574509311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6805","nonce":574509311}]},"public_addr":"172.21.15.183:6801/574509311","cluster_addr":"172.21.15.183:6803/574509311","heartbeat_back_addr":"172.21.15.183:6807/574509311","heartbeat_front_addr":"172.21.15.183:6805/574509311","state":["exists","up"]},{"osd":3,"uuid":"e7cfced4-9b40-40de-a303-75ea8d5df4a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6811","nonce":570672219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6813","nonce":570672219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6817","nonce":570672219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6815","nonce":570672219}]},"public_addr":"172.21.15.53:6811/570672219","cluster_addr":"172.21.15.53:6813/570672219","heartbeat_back_addr":"172.21.15.53:6817/570672219","heartbeat_front_addr":"172.21.15.53:6815/570672219","state":["exists","up"]},{"osd":4,"uuid":"44a9e070-2282-43f1-9f05-b1f8ff81e540","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6809","nonce":1526850103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6811","nonce":1526850103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6815","nonce":1526850103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6813","nonce":1526850103}]},"public_addr":"172.21.15.150:6809/1526850103","cluster_addr":"172.21.15.150:6811/1526850103","heartbeat_back_addr":"172.21.15.150:6815/1526850103","heartbeat_front_addr":"172.21.15.150:6813/1526850103","state":["exists","up"]},{"osd":5,"uuid":"2c5619d2-c0f9-47d6-bf7e-2f4afd1b09f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6809","nonce":773726576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6811","nonce":773726576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6815","nonce":773726576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6813","nonce":773726576}]},"public_addr":"172.21.15.183:6809/773726576","cluster_addr":"172.21.15.183:6811/773726576","heartbeat_back_addr":"172.21.15.183:6815/773726576","heartbeat_front_addr":"172.21.15.183:6813/773726576","state":["exists","up"]},{"osd":6,"uuid":"1cf74d94-c118-4738-b6db-040af458a8eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6819","nonce":355884319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6821","nonce":355884319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6825","nonce":355884319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6823","nonce":355884319}]},"public_addr":"172.21.15.53:6819/355884319","cluster_addr":"172.21.15.53:6821/355884319","heartbeat_back_addr":"172.21.15.53:6825/355884319","heartbeat_front_addr":"172.21.15.53:6823/355884319","state":["exists","up"]},{"osd":7,"uuid":"0ce97c04-cd41-4080-bbbc-be9cf5fadc02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6817","nonce":229022562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6819","nonce":229022562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6823","nonce":229022562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6821","nonce":229022562}]},"public_addr":"172.21.15.150:6817/229022562","cluster_addr":"172.21.15.150:6819/229022562","heartbeat_back_addr":"172.21.15.150:6823/229022562","heartbeat_front_addr":"172.21.15.150:6821/229022562","state":["exists","up"]},{"osd":8,"uuid":"5d57787d-2a7c-4514-bd01-357ced413f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6817","nonce":301003844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6819","nonce":301003844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6823","nonce":301003844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6821","nonce":301003844}]},"public_addr":"172.21.15.183:6817/301003844","cluster_addr":"172.21.15.183:6819/301003844","heartbeat_back_addr":"172.21.15.183:6823/301003844","heartbeat_front_addr":"172.21.15.183:6821/301003844","state":["exists","up"]},{"osd":9,"uuid":"e11dd8eb-c55a-4d70-8c58-457540b7a50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6827","nonce":3449260978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6829","nonce":3449260978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6833","nonce":3449260978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6831","nonce":3449260978}]},"public_addr":"172.21.15.53:6827/3449260978","cluster_addr":"172.21.15.53:6829/3449260978","heartbeat_back_addr":"172.21.15.53:6833/3449260978","heartbeat_front_addr":"172.21.15.53:6831/3449260978","state":["exists","up"]},{"osd":10,"uuid":"5de4803d-ad37-4a90-add5-0048f44fc4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6825","nonce":3435992308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6827","nonce":3435992308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6831","nonce":3435992308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6829","nonce":3435992308}]},"public_addr":"172.21.15.150:6825/3435992308","cluster_addr":"172.21.15.150:6827/3435992308","heartbeat_back_addr":"172.21.15.150:6831/3435992308","heartbeat_front_addr":"172.21.15.150:6829/3435992308","state":["exists","up"]},{"osd":11,"uuid":"0e819a68-ed56-477e-b9c0-81e8cc0bce11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6825","nonce":1957842686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6827","nonce":1957842686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6831","nonce":1957842686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6829","nonce":1957842686}]},"public_addr":"172.21.15.183:6825/1957842686","cluster_addr":"172.21.15.183:6827/1957842686","heartbeat_back_addr":"172.21.15.183:6831/1957842686","heartbeat_front_addr":"172.21.15.183:6829/1957842686","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:09.840629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:34.769537+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:58.112969+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:33:29.656790+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:33:55.181287+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:34:19.772645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:34:53.833921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:35:19.076733+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:35:43.163390+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:36:16.568340+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:36:42.492768+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:37:07.944639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6800/3887954889":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/1014501022":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/2576091441":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/1233527569":"2024-04-26T14:29:38.106242+0000","172.21.15.53:6801/3887954889":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/931889810":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/177196141":"2024-04-26T14:30:14.162383+0000","172.21.15.53:6800/1722497390":"2024-04-26T14:29:38.106242+0000","172.21.15.53:6801/1722497390":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/1927549198":"2024-04-26T14:30:14.162383+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:38:06.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.131+0000 7fa24d400640 1 -- 172.21.15.53:0/2982500102 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fa22c0a3bb0 msgr2=0x7fa22c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:06.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.131+0000 7fa24d400640 1 --2- 172.21.15.53:0/2982500102 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fa22c0a3bb0 0x7fa22c0a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa238005ab0 tx=0x7fa238005a20 comp rx=0 tx=0).stop 2024-04-25T14:38:06.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.131+0000 7fa24d400640 1 -- 172.21.15.53:0/2982500102 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2481053c0 msgr2=0x7fa24819a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:06.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.131+0000 7fa24d400640 1 --2- 172.21.15.53:0/2982500102 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2481053c0 0x7fa24819a950 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa23000e9f0 tx=0x7fa23000eeb0 comp rx=0 tx=0).stop 2024-04-25T14:38:06.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.131+0000 7fa24d400640 1 -- 172.21.15.53:0/2982500102 shutdown_connections 2024-04-25T14:38:06.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.131+0000 7fa24d400640 1 --2- 172.21.15.53:0/2982500102 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fa22c0a3bb0 0x7fa22c0a6070 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:06.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.131+0000 7fa24d400640 1 --2- 172.21.15.53:0/2982500102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa248105d80 0x7fa2481a19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:06.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.131+0000 7fa24d400640 1 --2- 172.21.15.53:0/2982500102 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2481053c0 0x7fa24819a950 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:06.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.131+0000 7fa24d400640 1 --2- 172.21.15.53:0/2982500102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2481041d0 0x7fa24819a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:06.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.131+0000 7fa24d400640 1 -- 172.21.15.53:0/2982500102 >> 172.21.15.53:0/2982500102 conn(0x7fa2480ff740 msgr2=0x7fa248108740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:06.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.131+0000 7fa24d400640 1 -- 172.21.15.53:0/2982500102 shutdown_connections 2024-04-25T14:38:06.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.131+0000 7fa24d400640 1 -- 172.21.15.53:0/2982500102 wait complete. 2024-04-25T14:38:06.159 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:33:02.338115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:38:06.159 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:38:06.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.364+0000 7f1cdee00640 1 -- 172.21.15.53:0/3331445761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cd8101e80 msgr2=0x7f1cd81022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:06.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.364+0000 7f1cdee00640 1 --2- 172.21.15.53:0/3331445761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cd8101e80 0x7f1cd81022a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc009900 tx=0x7f1ccc02a3f0 comp rx=0 tx=0).stop 2024-04-25T14:38:06.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.364+0000 7f1cdee00640 1 -- 172.21.15.53:0/3331445761 shutdown_connections 2024-04-25T14:38:06.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.364+0000 7f1cdee00640 1 --2- 172.21.15.53:0/3331445761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cd8103990 0x7f1cd810a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:06.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.364+0000 7f1cdee00640 1 --2- 172.21.15.53:0/3331445761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cd8102fd0 0x7f1cd8103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:06.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.364+0000 7f1cdee00640 1 --2- 172.21.15.53:0/3331445761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cd8101e80 0x7f1cd81022a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:06.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.365+0000 7f1cdee00640 1 -- 172.21.15.53:0/3331445761 >> 172.21.15.53:0/3331445761 conn(0x7f1cd80fd3f0 msgr2=0x7f1cd80ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:06.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.365+0000 7f1cdee00640 1 -- 172.21.15.53:0/3331445761 shutdown_connections 2024-04-25T14:38:06.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.365+0000 7f1cdee00640 1 -- 172.21.15.53:0/3331445761 wait complete. 2024-04-25T14:38:06.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.365+0000 7f1cdee00640 1 Processor -- start 2024-04-25T14:38:06.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.366+0000 7f1cdee00640 1 -- start start 2024-04-25T14:38:06.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.366+0000 7f1cdee00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cd8101e80 0x7f1cd81980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:06.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.366+0000 7f1cdee00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cd8102fd0 0x7f1cd81985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:06.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.366+0000 7f1cdee00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cd8103990 0x7f1cd819f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:06.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.366+0000 7f1cdee00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd810da00 con 0x7f1cd8103990 2024-04-25T14:38:06.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.366+0000 7f1cdee00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd810d880 con 0x7f1cd8101e80 2024-04-25T14:38:06.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.366+0000 7f1cdee00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd810db80 con 0x7f1cd8102fd0 2024-04-25T14:38:06.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.366+0000 7f1cde400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cd8103990 0x7f1cd819f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:06.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.366+0000 7f1cdda00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cd8101e80 0x7f1cd81980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:06.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.366+0000 7f1cdd000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cd8102fd0 0x7f1cd81985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:06.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.366+0000 7f1cde400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cd8103990 0x7f1cd819f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37620/0 (socket says 172.21.15.53:37620) 2024-04-25T14:38:06.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.366+0000 7f1cde400640 1 -- 172.21.15.53:0/604737471 learned_addr learned my addr 172.21.15.53:0/604737471 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:38:06.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.366+0000 7f1cdda00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cd8101e80 0x7f1cd81980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.53:52732/0 (socket says 172.21.15.53:52732) 2024-04-25T14:38:06.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.367+0000 7f1cdd000640 1 -- 172.21.15.53:0/604737471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cd8101e80 msgr2=0x7f1cd81980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:06.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.367+0000 7f1cdd000640 1 --2- 172.21.15.53:0/604737471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cd8101e80 0x7f1cd81980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:06.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.367+0000 7f1cdd000640 1 -- 172.21.15.53:0/604737471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cd8103990 msgr2=0x7f1cd819f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:06.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.367+0000 7f1cdd000640 1 --2- 172.21.15.53:0/604737471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cd8103990 0x7f1cd819f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:06.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.367+0000 7f1cdd000640 1 -- 172.21.15.53:0/604737471 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd819fcd0 con 0x7f1cd8102fd0 2024-04-25T14:38:06.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.367+0000 7f1cdd000640 1 --2- 172.21.15.53:0/604737471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cd8102fd0 0x7f1cd81985f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1cc400e9c0 tx=0x7f1cc400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:06.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.367+0000 7f1cd6000640 1 -- 172.21.15.53:0/604737471 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc400ccc0 con 0x7f1cd8102fd0 2024-04-25T14:38:06.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.367+0000 7f1cd6000640 1 -- 172.21.15.53:0/604737471 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cc40035f0 con 0x7f1cd8102fd0 2024-04-25T14:38:06.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.367+0000 7f1cd6000640 1 -- 172.21.15.53:0/604737471 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc400ce60 con 0x7f1cd8102fd0 2024-04-25T14:38:06.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.367+0000 7f1cdee00640 1 -- 172.21.15.53:0/604737471 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cd819ffc0 con 0x7f1cd8102fd0 2024-04-25T14:38:06.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.367+0000 7f1cdee00640 1 -- 172.21.15.53:0/604737471 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cd81a0430 con 0x7f1cd8102fd0 2024-04-25T14:38:06.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.368+0000 7f1cdee00640 1 -- 172.21.15.53:0/604737471 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cd8109200 con 0x7f1cd8102fd0 2024-04-25T14:38:06.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.371+0000 7f1cd6000640 1 -- 172.21.15.53:0/604737471 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1cc4004000 con 0x7f1cd8102fd0 2024-04-25T14:38:06.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.372+0000 7f1cd6000640 1 --2- 172.21.15.53:0/604737471 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f1cc00a3bb0 0x7f1cc00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:06.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.373+0000 7f1cd6000640 1 -- 172.21.15.53:0/604737471 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f1cc40d0400 con 0x7f1cd8102fd0 2024-04-25T14:38:06.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.373+0000 7f1cdda00640 1 --2- 172.21.15.53:0/604737471 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f1cc00a3bb0 0x7f1cc00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:06.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.373+0000 7f1cdda00640 1 --2- 172.21.15.53:0/604737471 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f1cc00a3bb0 0x7f1cc00a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc031500 tx=0x7f1ccc005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:06.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.377+0000 7f1cd6000640 1 -- 172.21.15.53:0/604737471 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1cc409d1e0 con 0x7f1cd8102fd0 2024-04-25T14:38:06.597 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.596+0000 7f1cdee00640 1 -- 172.21.15.53:0/604737471 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1cd8062b50 con 0x7f1cd8102fd0 2024-04-25T14:38:06.597 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.597+0000 7f1cd6000640 1 -- 172.21.15.53:0/604737471 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7f1cc409cae0 con 0x7f1cd8102fd0 2024-04-25T14:38:06.597 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2024-04-25T14:38:06.600 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.599+0000 7f1cdee00640 1 -- 172.21.15.53:0/604737471 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f1cc00a3bb0 msgr2=0x7f1cc00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:06.600 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.599+0000 7f1cdee00640 1 --2- 172.21.15.53:0/604737471 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f1cc00a3bb0 0x7f1cc00a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc031500 tx=0x7f1ccc005a20 comp rx=0 tx=0).stop 2024-04-25T14:38:06.600 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.599+0000 7f1cdee00640 1 -- 172.21.15.53:0/604737471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cd8102fd0 msgr2=0x7f1cd81985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:06.600 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.599+0000 7f1cdee00640 1 --2- 172.21.15.53:0/604737471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cd8102fd0 0x7f1cd81985f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1cc400e9c0 tx=0x7f1cc400ee80 comp rx=0 tx=0).stop 2024-04-25T14:38:06.600 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.599+0000 7f1cdee00640 1 -- 172.21.15.53:0/604737471 shutdown_connections 2024-04-25T14:38:06.600 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.599+0000 7f1cdee00640 1 --2- 172.21.15.53:0/604737471 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f1cc00a3bb0 0x7f1cc00a6070 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:06.600 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.599+0000 7f1cdee00640 1 --2- 172.21.15.53:0/604737471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cd8103990 0x7f1cd819f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:06.600 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.599+0000 7f1cdee00640 1 --2- 172.21.15.53:0/604737471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cd8102fd0 0x7f1cd81985f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:06.600 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.599+0000 7f1cdee00640 1 --2- 172.21.15.53:0/604737471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cd8101e80 0x7f1cd81980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:06.600 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.599+0000 7f1cdee00640 1 -- 172.21.15.53:0/604737471 >> 172.21.15.53:0/604737471 conn(0x7f1cd80fd3f0 msgr2=0x7f1cd81063d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:06.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.599+0000 7f1cdee00640 1 -- 172.21.15.53:0/604737471 shutdown_connections 2024-04-25T14:38:06.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.599+0000 7f1cdee00640 1 -- 172.21.15.53:0/604737471 wait complete. 2024-04-25T14:38:06.619 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T14:38:06.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:06 smithi150 ceph-mon[32504]: mds.? [v2:172.21.15.150:6838/149573616,v1:172.21.15.150:6839/149573616] up:boot 2024-04-25T14:38:06.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:06 smithi150 ceph-mon[32504]: fsmap 12 up:standby 2024-04-25T14:38:06.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:06 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T14:38:06.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:06 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:38:06.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:06 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/2891635950' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:38:06.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:06 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/2982500102' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:38:06.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:06 smithi053 ceph-mon[27773]: mds.? [v2:172.21.15.150:6838/149573616,v1:172.21.15.150:6839/149573616] up:boot 2024-04-25T14:38:06.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:06 smithi053 ceph-mon[27773]: fsmap 12 up:standby 2024-04-25T14:38:06.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:06 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T14:38:06.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:06 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:38:06.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:06 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/2891635950' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:38:06.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:06 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/2982500102' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:38:06.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.811+0000 7f76a5800640 1 -- 172.21.15.53:0/3544824641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76a0075f90 msgr2=0x7f76a00743e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:06.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.811+0000 7f76a5800640 1 --2- 172.21.15.53:0/3544824641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76a0075f90 0x7f76a00743e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7688009900 tx=0x7f768802a3f0 comp rx=0 tx=0).stop 2024-04-25T14:38:06.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.811+0000 7f76a5800640 1 -- 172.21.15.53:0/3544824641 shutdown_connections 2024-04-25T14:38:06.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.811+0000 7f76a5800640 1 --2- 172.21.15.53:0/3544824641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76a0109c80 0x7f76a010c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:06.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.811+0000 7f76a5800640 1 --2- 172.21.15.53:0/3544824641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76a00749b0 0x7f76a0074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:06.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.811+0000 7f76a5800640 1 --2- 172.21.15.53:0/3544824641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76a0075f90 0x7f76a00743e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:06.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.811+0000 7f76a5800640 1 -- 172.21.15.53:0/3544824641 >> 172.21.15.53:0/3544824641 conn(0x7f76a00fd330 msgr2=0x7f76a00ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:06.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.811+0000 7f76a5800640 1 -- 172.21.15.53:0/3544824641 shutdown_connections 2024-04-25T14:38:06.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.811+0000 7f76a5800640 1 -- 172.21.15.53:0/3544824641 wait complete. 2024-04-25T14:38:06.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.812+0000 7f76a5800640 1 Processor -- start 2024-04-25T14:38:06.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.812+0000 7f76a5800640 1 -- start start 2024-04-25T14:38:06.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.813+0000 7f76a5800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76a00749b0 0x7f76a019a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:06.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.813+0000 7f76a5800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76a0075f90 0x7f76a019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:06.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.813+0000 7f76a5800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76a0109c80 0x7f76a01a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:06.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.813+0000 7f76a5800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f76a010fb40 con 0x7f76a00749b0 2024-04-25T14:38:06.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.813+0000 7f76a5800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f76a010f9c0 con 0x7f76a0075f90 2024-04-25T14:38:06.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.813+0000 7f76a5800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f76a010fcc0 con 0x7f76a0109c80 2024-04-25T14:38:06.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.813+0000 7f769fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76a00749b0 0x7f76a019a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:06.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.813+0000 7f769f400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76a0075f90 0x7f76a019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:06.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.813+0000 7f769fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76a00749b0 0x7f76a019a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37630/0 (socket says 172.21.15.53:37630) 2024-04-25T14:38:06.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.813+0000 7f769fe00640 1 -- 172.21.15.53:0/1928467887 learned_addr learned my addr 172.21.15.53:0/1928467887 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:38:06.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.813+0000 7f76a4e00640 1 --2- 172.21.15.53:0/1928467887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76a0109c80 0x7f76a01a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:06.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.813+0000 7f769f400640 1 -- 172.21.15.53:0/1928467887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76a0109c80 msgr2=0x7f76a01a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:06.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.813+0000 7f769f400640 1 --2- 172.21.15.53:0/1928467887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76a0109c80 0x7f76a01a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:06.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.813+0000 7f769f400640 1 -- 172.21.15.53:0/1928467887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76a00749b0 msgr2=0x7f76a019a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:06.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.813+0000 7f769f400640 1 --2- 172.21.15.53:0/1928467887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76a00749b0 0x7f76a019a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:06.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.813+0000 7f769f400640 1 -- 172.21.15.53:0/1928467887 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a01a1e50 con 0x7f76a0075f90 2024-04-25T14:38:06.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.814+0000 7f76a4e00640 1 --2- 172.21.15.53:0/1928467887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76a0109c80 0x7f76a01a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:38:06.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.814+0000 7f769f400640 1 --2- 172.21.15.53:0/1928467887 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76a0075f90 0x7f76a019a770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f769000dab0 tx=0x7f769000df70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:06.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.814+0000 7f769cc00640 1 -- 172.21.15.53:0/1928467887 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769000be30 con 0x7f76a0075f90 2024-04-25T14:38:06.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.814+0000 7f76a5800640 1 -- 172.21.15.53:0/1928467887 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76a01a2140 con 0x7f76a0075f90 2024-04-25T14:38:06.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.814+0000 7f769cc00640 1 -- 172.21.15.53:0/1928467887 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7690003e20 con 0x7f76a0075f90 2024-04-25T14:38:06.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.814+0000 7f769cc00640 1 -- 172.21.15.53:0/1928467887 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76900106c0 con 0x7f76a0075f90 2024-04-25T14:38:06.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.814+0000 7f76a5800640 1 -- 172.21.15.53:0/1928467887 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76a01a26e0 con 0x7f76a0075f90 2024-04-25T14:38:06.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.816+0000 7f76a5800640 1 -- 172.21.15.53:0/1928467887 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76a01089b0 con 0x7f76a0075f90 2024-04-25T14:38:06.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.818+0000 7f769cc00640 1 -- 172.21.15.53:0/1928467887 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f76900042e0 con 0x7f76a0075f90 2024-04-25T14:38:06.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.819+0000 7f769cc00640 1 --2- 172.21.15.53:0/1928467887 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f76840a3bb0 0x7f76840a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:06.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.819+0000 7f769cc00640 1 -- 172.21.15.53:0/1928467887 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f76900d05f0 con 0x7f76a0075f90 2024-04-25T14:38:06.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.819+0000 7f769fe00640 1 --2- 172.21.15.53:0/1928467887 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f76840a3bb0 0x7f76840a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:06.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.820+0000 7f769fe00640 1 --2- 172.21.15.53:0/1928467887 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f76840a3bb0 0x7f76840a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f76880096c0 tx=0x7f7688005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:06.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:06.826+0000 7f769cc00640 1 -- 172.21.15.53:0/1928467887 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f769009d3d0 con 0x7f76a0075f90 2024-04-25T14:38:06.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:06 smithi183 ceph-mon[32584]: mds.? [v2:172.21.15.150:6838/149573616,v1:172.21.15.150:6839/149573616] up:boot 2024-04-25T14:38:06.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:06 smithi183 ceph-mon[32584]: fsmap 12 up:standby 2024-04-25T14:38:06.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:06 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T14:38:06.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:06 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:38:06.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:06 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/2891635950' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:38:06.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:06 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/2982500102' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:38:07.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.038+0000 7f76a5800640 1 -- 172.21.15.53:0/1928467887 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f76a002d7c0 con 0x7f76a0075f90 2024-04-25T14:38:07.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.039+0000 7f769cc00640 1 -- 172.21.15.53:0/1928467887 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v84) ==== 84+0+65 (secure 0 0 0) 0x7f769009ccd0 con 0x7f76a0075f90 2024-04-25T14:38:07.040 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:38:07.040 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2024-04-25T14:38:07.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:38:07.041 INFO:teuthology.orchestra.run.smithi053.stdout: "poolnum": 1, 2024-04-25T14:38:07.041 INFO:teuthology.orchestra.run.smithi053.stdout: "poolname": ".mgr" 2024-04-25T14:38:07.041 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:38:07.041 INFO:teuthology.orchestra.run.smithi053.stdout:] 2024-04-25T14:38:07.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.042+0000 7f76a5800640 1 -- 172.21.15.53:0/1928467887 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f76840a3bb0 msgr2=0x7f76840a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:07.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.042+0000 7f76a5800640 1 --2- 172.21.15.53:0/1928467887 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f76840a3bb0 0x7f76840a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f76880096c0 tx=0x7f7688005a20 comp rx=0 tx=0).stop 2024-04-25T14:38:07.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.042+0000 7f76a5800640 1 -- 172.21.15.53:0/1928467887 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76a0075f90 msgr2=0x7f76a019a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:07.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.042+0000 7f76a5800640 1 --2- 172.21.15.53:0/1928467887 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76a0075f90 0x7f76a019a770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f769000dab0 tx=0x7f769000df70 comp rx=0 tx=0).stop 2024-04-25T14:38:07.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.042+0000 7f76a5800640 1 -- 172.21.15.53:0/1928467887 shutdown_connections 2024-04-25T14:38:07.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.042+0000 7f76a5800640 1 --2- 172.21.15.53:0/1928467887 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f76840a3bb0 0x7f76840a6070 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:07.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.042+0000 7f76a5800640 1 --2- 172.21.15.53:0/1928467887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76a0109c80 0x7f76a01a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:07.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.042+0000 7f76a5800640 1 --2- 172.21.15.53:0/1928467887 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76a0075f90 0x7f76a019a770 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:07.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.042+0000 7f76a5800640 1 --2- 172.21.15.53:0/1928467887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76a00749b0 0x7f76a019a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:07.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.042+0000 7f76a5800640 1 -- 172.21.15.53:0/1928467887 >> 172.21.15.53:0/1928467887 conn(0x7f76a00fd330 msgr2=0x7f76a0105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:07.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.042+0000 7f76a5800640 1 -- 172.21.15.53:0/1928467887 shutdown_connections 2024-04-25T14:38:07.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.043+0000 7f76a5800640 1 -- 172.21.15.53:0/1928467887 wait complete. 2024-04-25T14:38:07.061 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T14:38:07.062 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T14:38:07.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.273+0000 7f9067e00640 1 -- 172.21.15.53:0/2707762250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90680749b0 msgr2=0x7f9068074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:07.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.273+0000 7f9067e00640 1 --2- 172.21.15.53:0/2707762250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90680749b0 0x7f9068074e30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9054009ca0 tx=0x7f905402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:38:07.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.273+0000 7f9067e00640 1 -- 172.21.15.53:0/2707762250 shutdown_connections 2024-04-25T14:38:07.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.273+0000 7f9067e00640 1 --2- 172.21.15.53:0/2707762250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9068109c20 0x7f906810c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:07.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.273+0000 7f9067e00640 1 --2- 172.21.15.53:0/2707762250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90680749b0 0x7f9068074e30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:07.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.273+0000 7f9067e00640 1 --2- 172.21.15.53:0/2707762250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9068075f90 0x7f90680743e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:07.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.273+0000 7f9067e00640 1 -- 172.21.15.53:0/2707762250 >> 172.21.15.53:0/2707762250 conn(0x7f90680fd2d0 msgr2=0x7f90680ff710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:07.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.273+0000 7f9067e00640 1 -- 172.21.15.53:0/2707762250 shutdown_connections 2024-04-25T14:38:07.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.273+0000 7f9067e00640 1 -- 172.21.15.53:0/2707762250 wait complete. 2024-04-25T14:38:07.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.274+0000 7f9067e00640 1 Processor -- start 2024-04-25T14:38:07.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.275+0000 7f9067e00640 1 -- start start 2024-04-25T14:38:07.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.275+0000 7f9067e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90680749b0 0x7f906819a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:07.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.275+0000 7f9067e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9068075f90 0x7f906819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:07.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.275+0000 7f9067e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9068109c20 0x7f90681a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:07.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.275+0000 7f9067e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f906810fa70 con 0x7f90680749b0 2024-04-25T14:38:07.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.275+0000 7f9067e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f906810f8f0 con 0x7f9068075f90 2024-04-25T14:38:07.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.275+0000 7f9067e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f906810fbf0 con 0x7f9068109c20 2024-04-25T14:38:07.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.275+0000 7f9066a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90680749b0 0x7f906819a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:07.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.275+0000 7f9066a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90680749b0 0x7f906819a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37646/0 (socket says 172.21.15.53:37646) 2024-04-25T14:38:07.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.275+0000 7f9066a00640 1 -- 172.21.15.53:0/1729081203 learned_addr learned my addr 172.21.15.53:0/1729081203 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:38:07.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.275+0000 7f9066000640 1 --2- 172.21.15.53:0/1729081203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9068075f90 0x7f906819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:07.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.275+0000 7f9067400640 1 --2- 172.21.15.53:0/1729081203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9068109c20 0x7f90681a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:07.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.276+0000 7f9066a00640 1 -- 172.21.15.53:0/1729081203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9068109c20 msgr2=0x7f90681a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:07.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.276+0000 7f9066a00640 1 --2- 172.21.15.53:0/1729081203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9068109c20 0x7f90681a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:07.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.276+0000 7f9066a00640 1 -- 172.21.15.53:0/1729081203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9068075f90 msgr2=0x7f906819a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:07.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.276+0000 7f9066a00640 1 --2- 172.21.15.53:0/1729081203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9068075f90 0x7f906819a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:07.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.276+0000 7f9066a00640 1 -- 172.21.15.53:0/1729081203 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90681a1db0 con 0x7f90680749b0 2024-04-25T14:38:07.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.276+0000 7f9067400640 1 --2- 172.21.15.53:0/1729081203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9068109c20 0x7f90681a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:38:07.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.276+0000 7f9066a00640 1 --2- 172.21.15.53:0/1729081203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90680749b0 0x7f906819a190 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f905800e9f0 tx=0x7f905800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:07.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.276+0000 7f905f400640 1 -- 172.21.15.53:0/1729081203 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905800cd80 con 0x7f90680749b0 2024-04-25T14:38:07.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.276+0000 7f905f400640 1 -- 172.21.15.53:0/1729081203 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90580054d0 con 0x7f90680749b0 2024-04-25T14:38:07.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.277+0000 7f9067e00640 1 -- 172.21.15.53:0/1729081203 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90681a20a0 con 0x7f90680749b0 2024-04-25T14:38:07.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.277+0000 7f905f400640 1 -- 172.21.15.53:0/1729081203 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9058010690 con 0x7f90680749b0 2024-04-25T14:38:07.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.277+0000 7f9067e00640 1 -- 172.21.15.53:0/1729081203 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90681a25a0 con 0x7f90680749b0 2024-04-25T14:38:07.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.278+0000 7f9067e00640 1 -- 172.21.15.53:0/1729081203 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9068108950 con 0x7f90680749b0 2024-04-25T14:38:07.282 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.281+0000 7f905f400640 1 -- 172.21.15.53:0/1729081203 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9058004e30 con 0x7f90680749b0 2024-04-25T14:38:07.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.283+0000 7f905f400640 1 --2- 172.21.15.53:0/1729081203 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f904c0a3bb0 0x7f904c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:07.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.283+0000 7f905f400640 1 -- 172.21.15.53:0/1729081203 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f90580d0570 con 0x7f90680749b0 2024-04-25T14:38:07.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.283+0000 7f9066000640 1 --2- 172.21.15.53:0/1729081203 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f904c0a3bb0 0x7f904c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:07.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.284+0000 7f9066000640 1 --2- 172.21.15.53:0/1729081203 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f904c0a3bb0 0x7f904c0a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f906819b6a0 tx=0x7f9054005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:07.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.286+0000 7f905f400640 1 -- 172.21.15.53:0/1729081203 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9058053ff0 con 0x7f90680749b0 2024-04-25T14:38:07.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:07.506+0000 7f9067e00640 1 -- 172.21.15.53:0/1729081203 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f906802d5c0 con 0x7f90680749b0 2024-04-25T14:38:07.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:07 smithi053 ceph-mon[27773]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:38:07.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:07 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/604737471' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:38:07.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:07 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1928467887' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T14:38:07.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:07 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:07.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:07 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:07.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:07 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:07.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:07 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:07.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:07 smithi183 ceph-mon[32584]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:38:07.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:07 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/604737471' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:38:07.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:07 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/1928467887' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T14:38:07.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:07 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:07.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:07 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:07.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:07 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:07.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:07 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:07.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:07 smithi150 ceph-mon[32504]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:38:07.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:07 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/604737471' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:38:07.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:07 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/1928467887' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T14:38:07.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:07 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:07.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:07 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:07.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:07 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:07.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:07 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:08.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.519+0000 7f905f400640 1 -- 172.21.15.53:0/1729081203 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v85) ==== 138+0+0 (secure 0 0 0) 0x7f905809d3f0 con 0x7f90680749b0 2024-04-25T14:38:08.521 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' created 2024-04-25T14:38:08.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.525+0000 7f9067e00640 1 -- 172.21.15.53:0/1729081203 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f904c0a3bb0 msgr2=0x7f904c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:08.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.525+0000 7f9067e00640 1 --2- 172.21.15.53:0/1729081203 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f904c0a3bb0 0x7f904c0a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f906819b6a0 tx=0x7f9054005dc0 comp rx=0 tx=0).stop 2024-04-25T14:38:08.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.525+0000 7f9067e00640 1 -- 172.21.15.53:0/1729081203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90680749b0 msgr2=0x7f906819a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:08.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.525+0000 7f9067e00640 1 --2- 172.21.15.53:0/1729081203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90680749b0 0x7f906819a190 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f905800e9f0 tx=0x7f905800edf0 comp rx=0 tx=0).stop 2024-04-25T14:38:08.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.525+0000 7f9067e00640 1 -- 172.21.15.53:0/1729081203 shutdown_connections 2024-04-25T14:38:08.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.525+0000 7f9067e00640 1 --2- 172.21.15.53:0/1729081203 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f904c0a3bb0 0x7f904c0a6070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:08.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.526+0000 7f9067e00640 1 --2- 172.21.15.53:0/1729081203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9068109c20 0x7f90681a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:08.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.526+0000 7f9067e00640 1 --2- 172.21.15.53:0/1729081203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9068075f90 0x7f906819a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:08.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.526+0000 7f9067e00640 1 --2- 172.21.15.53:0/1729081203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90680749b0 0x7f906819a190 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:08.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.526+0000 7f9067e00640 1 -- 172.21.15.53:0/1729081203 >> 172.21.15.53:0/1729081203 conn(0x7f90680fd2d0 msgr2=0x7f9068105b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:08.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.526+0000 7f9067e00640 1 -- 172.21.15.53:0/1729081203 shutdown_connections 2024-04-25T14:38:08.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.527+0000 7f9067e00640 1 -- 172.21.15.53:0/1729081203 wait complete. 2024-04-25T14:38:08.556 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T14:38:08.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.754+0000 7fb89b200640 1 -- 172.21.15.53:0/1764376794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb89c101e90 msgr2=0x7fb89c1022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:08.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.754+0000 7fb89b200640 1 --2- 172.21.15.53:0/1764376794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb89c101e90 0x7fb89c1022b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb88c009900 tx=0x7fb88c02a400 comp rx=0 tx=0).stop 2024-04-25T14:38:08.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.755+0000 7fb89b200640 1 -- 172.21.15.53:0/1764376794 shutdown_connections 2024-04-25T14:38:08.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.755+0000 7fb89b200640 1 --2- 172.21.15.53:0/1764376794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb89c103a30 0x7fb89c10a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:08.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.755+0000 7fb89b200640 1 --2- 172.21.15.53:0/1764376794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb89c103070 0x7fb89c1034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:08.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.755+0000 7fb89b200640 1 --2- 172.21.15.53:0/1764376794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb89c101e90 0x7fb89c1022b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:08.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.755+0000 7fb89b200640 1 -- 172.21.15.53:0/1764376794 >> 172.21.15.53:0/1764376794 conn(0x7fb89c0fd450 msgr2=0x7fb89c0ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:08.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.755+0000 7fb89b200640 1 -- 172.21.15.53:0/1764376794 shutdown_connections 2024-04-25T14:38:08.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.755+0000 7fb89b200640 1 -- 172.21.15.53:0/1764376794 wait complete. 2024-04-25T14:38:08.757 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.756+0000 7fb89b200640 1 Processor -- start 2024-04-25T14:38:08.757 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.756+0000 7fb89b200640 1 -- start start 2024-04-25T14:38:08.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.757+0000 7fb89b200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb89c101e90 0x7fb89c19a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:08.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.757+0000 7fb89b200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb89c103070 0x7fb89c19a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:08.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.757+0000 7fb89b200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb89c103a30 0x7fb89c1a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:08.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.757+0000 7fb89b200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb89c10d9b0 con 0x7fb89c103070 2024-04-25T14:38:08.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.757+0000 7fb89b200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb89c10d830 con 0x7fb89c101e90 2024-04-25T14:38:08.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.757+0000 7fb89b200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb89c10db30 con 0x7fb89c103a30 2024-04-25T14:38:08.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.757+0000 7fb89a800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb89c103a30 0x7fb89c1a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:08.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.757+0000 7fb899e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb89c101e90 0x7fb89c19a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:08.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.757+0000 7fb89a800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb89c103a30 0x7fb89c1a1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:41906/0 (socket says 172.21.15.53:41906) 2024-04-25T14:38:08.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.757+0000 7fb899400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb89c103070 0x7fb89c19a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:08.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.757+0000 7fb89a800640 1 -- 172.21.15.53:0/1319784435 learned_addr learned my addr 172.21.15.53:0/1319784435 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:38:08.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.758+0000 7fb89a800640 1 -- 172.21.15.53:0/1319784435 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb89c101e90 msgr2=0x7fb89c19a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:08.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.758+0000 7fb89a800640 1 --2- 172.21.15.53:0/1319784435 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb89c101e90 0x7fb89c19a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:08.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.758+0000 7fb89a800640 1 -- 172.21.15.53:0/1319784435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb89c103070 msgr2=0x7fb89c19a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:08.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.758+0000 7fb89a800640 1 --2- 172.21.15.53:0/1319784435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb89c103070 0x7fb89c19a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:08.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.758+0000 7fb89a800640 1 -- 172.21.15.53:0/1319784435 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb89c1a1ee0 con 0x7fb89c103a30 2024-04-25T14:38:08.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.758+0000 7fb899e00640 1 --2- 172.21.15.53:0/1319784435 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb89c101e90 0x7fb89c19a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:38:08.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.758+0000 7fb89a800640 1 --2- 172.21.15.53:0/1319784435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb89c103a30 0x7fb89c1a1870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb89400eb10 tx=0x7fb89400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:08.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.758+0000 7fb892a00640 1 -- 172.21.15.53:0/1319784435 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb894004b20 con 0x7fb89c103a30 2024-04-25T14:38:08.761 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.758+0000 7fb89b200640 1 -- 172.21.15.53:0/1319784435 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb89c1a21d0 con 0x7fb89c103a30 2024-04-25T14:38:08.761 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.758+0000 7fb892a00640 1 -- 172.21.15.53:0/1319784435 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb894004cc0 con 0x7fb89c103a30 2024-04-25T14:38:08.761 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.758+0000 7fb892a00640 1 -- 172.21.15.53:0/1319784435 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb894010670 con 0x7fb89c103a30 2024-04-25T14:38:08.761 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.758+0000 7fb89b200640 1 -- 172.21.15.53:0/1319784435 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb89c1a2720 con 0x7fb89c103a30 2024-04-25T14:38:08.761 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.760+0000 7fb89b200640 1 -- 172.21.15.53:0/1319784435 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb89c1a29e0 con 0x7fb89c103a30 2024-04-25T14:38:08.762 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.762+0000 7fb892a00640 1 -- 172.21.15.53:0/1319784435 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb894005020 con 0x7fb89c103a30 2024-04-25T14:38:08.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.763+0000 7fb892a00640 1 --2- 172.21.15.53:0/1319784435 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fb8800a3bb0 0x7fb8800a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:08.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.763+0000 7fb892a00640 1 -- 172.21.15.53:0/1319784435 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7497+0+0 (secure 0 0 0) 0x7fb8940d0690 con 0x7fb89c103a30 2024-04-25T14:38:08.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.764+0000 7fb899e00640 1 --2- 172.21.15.53:0/1319784435 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fb8800a3bb0 0x7fb8800a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:08.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.765+0000 7fb899e00640 1 --2- 172.21.15.53:0/1319784435 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fb8800a3bb0 0x7fb8800a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb88c009230 tx=0x7fb88c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:08.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.768+0000 7fb892a00640 1 -- 172.21.15.53:0/1319784435 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb89409d300 con 0x7fb89c103a30 2024-04-25T14:38:08.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:08 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1729081203' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T14:38:08.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:08 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/1729081203' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T14:38:08.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:08 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/1729081203' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T14:38:08.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:08.982+0000 7fb89b200640 1 -- 172.21.15.53:0/1319784435 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb89c1a2bb0 con 0x7fb89c103a30 2024-04-25T14:38:09.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.534+0000 7fb892a00640 1 -- 172.21.15.53:0/1319784435 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v86) ==== 170+0+0 (secure 0 0 0) 0x7fb89409cc00 con 0x7fb89c103a30 2024-04-25T14:38:09.536 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' created 2024-04-25T14:38:09.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.540+0000 7fb89b200640 1 -- 172.21.15.53:0/1319784435 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fb8800a3bb0 msgr2=0x7fb8800a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:09.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.540+0000 7fb89b200640 1 --2- 172.21.15.53:0/1319784435 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fb8800a3bb0 0x7fb8800a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb88c009230 tx=0x7fb88c0314a0 comp rx=0 tx=0).stop 2024-04-25T14:38:09.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.540+0000 7fb89b200640 1 -- 172.21.15.53:0/1319784435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb89c103a30 msgr2=0x7fb89c1a1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:09.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.540+0000 7fb89b200640 1 --2- 172.21.15.53:0/1319784435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb89c103a30 0x7fb89c1a1870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb89400eb10 tx=0x7fb89400c2d0 comp rx=0 tx=0).stop 2024-04-25T14:38:09.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.543+0000 7fb89b200640 1 -- 172.21.15.53:0/1319784435 shutdown_connections 2024-04-25T14:38:09.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.543+0000 7fb89b200640 1 --2- 172.21.15.53:0/1319784435 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fb8800a3bb0 0x7fb8800a6070 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:09.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.543+0000 7fb89b200640 1 --2- 172.21.15.53:0/1319784435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb89c103a30 0x7fb89c1a1870 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:09.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.543+0000 7fb89b200640 1 --2- 172.21.15.53:0/1319784435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb89c103070 0x7fb89c19a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:09.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.543+0000 7fb89b200640 1 --2- 172.21.15.53:0/1319784435 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb89c101e90 0x7fb89c19a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:09.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.543+0000 7fb89b200640 1 -- 172.21.15.53:0/1319784435 >> 172.21.15.53:0/1319784435 conn(0x7fb89c0fd450 msgr2=0x7fb89c074180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:09.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.546+0000 7fb89b200640 1 -- 172.21.15.53:0/1319784435 shutdown_connections 2024-04-25T14:38:09.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.546+0000 7fb89b200640 1 -- 172.21.15.53:0/1319784435 wait complete. 2024-04-25T14:38:09.572 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:09 smithi053 ceph-mon[27773]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:38:09.572 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:09 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1729081203' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T14:38:09.572 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:09 smithi053 ceph-mon[27773]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T14:38:09.572 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:09 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:09.572 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:09 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:09.572 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:09 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:38:09.572 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:09 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:38:09.572 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:09 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:09.573 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:09 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:38:09.573 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:09 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1319784435' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T14:38:09.573 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:09 smithi053 ceph-mon[27773]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T14:38:09.573 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:09 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:09.574 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T14:38:09.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.767+0000 7f4c76e00640 1 -- 172.21.15.53:0/2891929737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c701053c0 msgr2=0x7f4c70105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:09.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.767+0000 7f4c76e00640 1 --2- 172.21.15.53:0/2891929737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c701053c0 0x7f4c70105840 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c009860 tx=0x7f4c5c02a340 comp rx=0 tx=0).stop 2024-04-25T14:38:09.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.768+0000 7f4c76e00640 1 -- 172.21.15.53:0/2891929737 shutdown_connections 2024-04-25T14:38:09.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.768+0000 7f4c76e00640 1 --2- 172.21.15.53:0/2891929737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c70105d80 0x7f4c7010c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:09.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.768+0000 7f4c76e00640 1 --2- 172.21.15.53:0/2891929737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c701053c0 0x7f4c70105840 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:09.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.768+0000 7f4c76e00640 1 --2- 172.21.15.53:0/2891929737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c701041d0 0x7f4c701045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:09.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.768+0000 7f4c76e00640 1 -- 172.21.15.53:0/2891929737 >> 172.21.15.53:0/2891929737 conn(0x7f4c700ff740 msgr2=0x7f4c70101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:09.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.768+0000 7f4c76e00640 1 -- 172.21.15.53:0/2891929737 shutdown_connections 2024-04-25T14:38:09.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.768+0000 7f4c76e00640 1 -- 172.21.15.53:0/2891929737 wait complete. 2024-04-25T14:38:09.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.769+0000 7f4c76e00640 1 Processor -- start 2024-04-25T14:38:09.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.769+0000 7f4c76e00640 1 -- start start 2024-04-25T14:38:09.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.769+0000 7f4c76e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c701041d0 0x7f4c7019a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:09.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.769+0000 7f4c76e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c701053c0 0x7f4c7019a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:09.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.769+0000 7f4c76e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c70105d80 0x7f4c701a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:09.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.769+0000 7f4c76e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7010fd10 con 0x7f4c701053c0 2024-04-25T14:38:09.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.769+0000 7f4c76e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7010fb90 con 0x7f4c70105d80 2024-04-25T14:38:09.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.769+0000 7f4c76e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7010fe90 con 0x7f4c701041d0 2024-04-25T14:38:09.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.769+0000 7f4c6fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c701041d0 0x7f4c7019a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:09.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.770+0000 7f4c6fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c701041d0 0x7f4c7019a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:41918/0 (socket says 172.21.15.53:41918) 2024-04-25T14:38:09.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.770+0000 7f4c6fe00640 1 -- 172.21.15.53:0/338552465 learned_addr learned my addr 172.21.15.53:0/338552465 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:38:09.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.770+0000 7f4c6f400640 1 --2- 172.21.15.53:0/338552465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c701053c0 0x7f4c7019a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:09.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.770+0000 7f4c74e00640 1 --2- 172.21.15.53:0/338552465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c70105d80 0x7f4c701a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:09.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.770+0000 7f4c6fe00640 1 -- 172.21.15.53:0/338552465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c70105d80 msgr2=0x7f4c701a1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:09.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.770+0000 7f4c6fe00640 1 --2- 172.21.15.53:0/338552465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c70105d80 0x7f4c701a1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:09.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.770+0000 7f4c6fe00640 1 -- 172.21.15.53:0/338552465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c701053c0 msgr2=0x7f4c7019a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:09.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.770+0000 7f4c6fe00640 1 --2- 172.21.15.53:0/338552465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c701053c0 0x7f4c7019a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:09.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.770+0000 7f4c6fe00640 1 -- 172.21.15.53:0/338552465 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c701a1ff0 con 0x7f4c701041d0 2024-04-25T14:38:09.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.770+0000 7f4c6fe00640 1 --2- 172.21.15.53:0/338552465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c701041d0 0x7f4c7019a3d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4c6000e9c0 tx=0x7f4c6000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:09.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.771+0000 7f4c6cc00640 1 -- 172.21.15.53:0/338552465 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6000cd80 con 0x7f4c701041d0 2024-04-25T14:38:09.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.771+0000 7f4c6cc00640 1 -- 172.21.15.53:0/338552465 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c600035f0 con 0x7f4c701041d0 2024-04-25T14:38:09.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.771+0000 7f4c76e00640 1 -- 172.21.15.53:0/338552465 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c701a22e0 con 0x7f4c701041d0 2024-04-25T14:38:09.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.771+0000 7f4c6cc00640 1 -- 172.21.15.53:0/338552465 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c60010690 con 0x7f4c701041d0 2024-04-25T14:38:09.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.771+0000 7f4c76e00640 1 -- 172.21.15.53:0/338552465 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c701a2800 con 0x7f4c701041d0 2024-04-25T14:38:09.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.772+0000 7f4c76e00640 1 -- 172.21.15.53:0/338552465 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c7010b590 con 0x7f4c701041d0 2024-04-25T14:38:09.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.774+0000 7f4c6cc00640 1 -- 172.21.15.53:0/338552465 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4c600037a0 con 0x7f4c701041d0 2024-04-25T14:38:09.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.775+0000 7f4c6cc00640 1 --2- 172.21.15.53:0/338552465 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f4c540a3bb0 0x7f4c540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:09.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.776+0000 7f4c6cc00640 1 -- 172.21.15.53:0/338552465 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7879+0+0 (secure 0 0 0) 0x7f4c600d0650 con 0x7f4c701041d0 2024-04-25T14:38:09.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.776+0000 7f4c6f400640 1 --2- 172.21.15.53:0/338552465 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f4c540a3bb0 0x7f4c540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:09.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.777+0000 7f4c6f400640 1 --2- 172.21.15.53:0/338552465 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f4c540a3bb0 0x7f4c540a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c02a930 tx=0x7f4c5c030450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:09.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:09.779+0000 7f4c6cc00640 1 -- 172.21.15.53:0/338552465 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4c6009d140 con 0x7f4c701041d0 2024-04-25T14:38:09.795 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:09 smithi150 ceph-mon[32504]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:38:09.795 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:09 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/1729081203' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T14:38:09.795 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:09 smithi150 ceph-mon[32504]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T14:38:09.795 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:09 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:09.795 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:09 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:09.795 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:09 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:38:09.795 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:09 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:38:09.795 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:09 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:09.796 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:09 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:38:09.796 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:09 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/1319784435' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T14:38:09.796 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:09 smithi150 ceph-mon[32504]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T14:38:09.796 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:09 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:09.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:09 smithi183 ceph-mon[32584]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:38:09.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:09 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/1729081203' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T14:38:09.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:09 smithi183 ceph-mon[32584]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T14:38:09.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:09 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:09.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:09 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:09.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:09 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:38:09.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:09 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:38:09.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:09 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:09.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:09 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:38:09.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:09 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/1319784435' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T14:38:09.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:09 smithi183 ceph-mon[32584]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T14:38:09.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:09 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:10.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.034+0000 7f4c76e00640 1 -- 172.21.15.53:0/338552465 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f4c70062b50 con 0x7f4c701041d0 2024-04-25T14:38:10.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.537+0000 7f4c6cc00640 1 -- 172.21.15.53:0/338552465 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f4c6009ca40 con 0x7f4c701041d0 2024-04-25T14:38:10.540 INFO:teuthology.orchestra.run.smithi053.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T14:38:10.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.545+0000 7f4c76e00640 1 -- 172.21.15.53:0/338552465 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f4c540a3bb0 msgr2=0x7f4c540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:10.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.545+0000 7f4c76e00640 1 --2- 172.21.15.53:0/338552465 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f4c540a3bb0 0x7f4c540a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c02a930 tx=0x7f4c5c030450 comp rx=0 tx=0).stop 2024-04-25T14:38:10.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.545+0000 7f4c76e00640 1 -- 172.21.15.53:0/338552465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c701041d0 msgr2=0x7f4c7019a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:10.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.545+0000 7f4c76e00640 1 --2- 172.21.15.53:0/338552465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c701041d0 0x7f4c7019a3d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4c6000e9c0 tx=0x7f4c6000ee80 comp rx=0 tx=0).stop 2024-04-25T14:38:10.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.545+0000 7f4c76e00640 1 -- 172.21.15.53:0/338552465 shutdown_connections 2024-04-25T14:38:10.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.545+0000 7f4c76e00640 1 --2- 172.21.15.53:0/338552465 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f4c540a3bb0 0x7f4c540a6070 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:10.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.545+0000 7f4c76e00640 1 --2- 172.21.15.53:0/338552465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c70105d80 0x7f4c701a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:10.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.545+0000 7f4c76e00640 1 --2- 172.21.15.53:0/338552465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c701053c0 0x7f4c7019a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:10.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.545+0000 7f4c76e00640 1 --2- 172.21.15.53:0/338552465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c701041d0 0x7f4c7019a3d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:10.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.545+0000 7f4c76e00640 1 -- 172.21.15.53:0/338552465 >> 172.21.15.53:0/338552465 conn(0x7f4c700ff740 msgr2=0x7f4c70108760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:10.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.545+0000 7f4c76e00640 1 -- 172.21.15.53:0/338552465 shutdown_connections 2024-04-25T14:38:10.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.545+0000 7f4c76e00640 1 -- 172.21.15.53:0/338552465 wait complete. 2024-04-25T14:38:10.566 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:38:10.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.739+0000 7f207d200640 1 -- 172.21.15.53:0/4020531756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2078102880 msgr2=0x7f2078109110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:10.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.739+0000 7f207d200640 1 --2- 172.21.15.53:0/4020531756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2078102880 0x7f2078109110 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2070009ca0 tx=0x7f207002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:38:10.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.740+0000 7f207d200640 1 -- 172.21.15.53:0/4020531756 shutdown_connections 2024-04-25T14:38:10.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.740+0000 7f207d200640 1 --2- 172.21.15.53:0/4020531756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2078102880 0x7f2078109110 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:10.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.740+0000 7f207d200640 1 --2- 172.21.15.53:0/4020531756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2078101ec0 0x7f2078102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:10.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.740+0000 7f207d200640 1 --2- 172.21.15.53:0/4020531756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2078100cd0 0x7f20781010f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:10.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.740+0000 7f207d200640 1 -- 172.21.15.53:0/4020531756 >> 172.21.15.53:0/4020531756 conn(0x7f20780fc220 msgr2=0x7f20780fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:10.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.740+0000 7f207d200640 1 -- 172.21.15.53:0/4020531756 shutdown_connections 2024-04-25T14:38:10.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.740+0000 7f207d200640 1 -- 172.21.15.53:0/4020531756 wait complete. 2024-04-25T14:38:10.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.741+0000 7f207d200640 1 Processor -- start 2024-04-25T14:38:10.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.741+0000 7f207d200640 1 -- start start 2024-04-25T14:38:10.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.741+0000 7f207d200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2078100cd0 0x7f2078104bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:10.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.741+0000 7f207d200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2078101ec0 0x7f20781032a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:10.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.741+0000 7f207d200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2078102880 0x7f20781037e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:10.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.741+0000 7f207d200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f207810c7e0 con 0x7f2078100cd0 2024-04-25T14:38:10.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.741+0000 7f207d200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f207810c660 con 0x7f2078101ec0 2024-04-25T14:38:10.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.741+0000 7f207d200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f207810c960 con 0x7f2078102880 2024-04-25T14:38:10.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.741+0000 7f2076a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2078101ec0 0x7f20781032a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:10.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.741+0000 7f2076a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2078101ec0 0x7f20781032a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.53:52806/0 (socket says 172.21.15.53:52806) 2024-04-25T14:38:10.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.741+0000 7f2077400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2078100cd0 0x7f2078104bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:10.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.741+0000 7f2076a00640 1 -- 172.21.15.53:0/1809570553 learned_addr learned my addr 172.21.15.53:0/1809570553 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:38:10.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.741+0000 7f2077e00640 1 --2- 172.21.15.53:0/1809570553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2078102880 0x7f20781037e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:10.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.742+0000 7f2076a00640 1 -- 172.21.15.53:0/1809570553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2078102880 msgr2=0x7f20781037e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:10.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.742+0000 7f2076a00640 1 --2- 172.21.15.53:0/1809570553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2078102880 0x7f20781037e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:10.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.742+0000 7f2076a00640 1 -- 172.21.15.53:0/1809570553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2078100cd0 msgr2=0x7f2078104bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:10.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.742+0000 7f2076a00640 1 --2- 172.21.15.53:0/1809570553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2078100cd0 0x7f2078104bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:10.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.742+0000 7f2076a00640 1 -- 172.21.15.53:0/1809570553 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2078104000 con 0x7f2078101ec0 2024-04-25T14:38:10.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.742+0000 7f2077e00640 1 --2- 172.21.15.53:0/1809570553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2078102880 0x7f20781037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:38:10.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.742+0000 7f2076a00640 1 --2- 172.21.15.53:0/1809570553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2078101ec0 0x7f20781032a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f206400e9f0 tx=0x7f206400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:10.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.742+0000 7f206fe00640 1 -- 172.21.15.53:0/1809570553 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206400cd80 con 0x7f2078101ec0 2024-04-25T14:38:10.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.742+0000 7f206fe00640 1 -- 172.21.15.53:0/1809570553 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20640108e0 con 0x7f2078101ec0 2024-04-25T14:38:10.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.742+0000 7f207d200640 1 -- 172.21.15.53:0/1809570553 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20781a20d0 con 0x7f2078101ec0 2024-04-25T14:38:10.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.742+0000 7f206fe00640 1 -- 172.21.15.53:0/1809570553 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2064018980 con 0x7f2078101ec0 2024-04-25T14:38:10.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.743+0000 7f207d200640 1 -- 172.21.15.53:0/1809570553 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20781a2620 con 0x7f2078101ec0 2024-04-25T14:38:10.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.744+0000 7f207d200640 1 -- 172.21.15.53:0/1809570553 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20781a2880 con 0x7f2078101ec0 2024-04-25T14:38:10.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.746+0000 7f206fe00640 1 -- 172.21.15.53:0/1809570553 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2064010430 con 0x7f2078101ec0 2024-04-25T14:38:10.748 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.747+0000 7f206fe00640 1 --2- 172.21.15.53:0/1809570553 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f205c0a3bb0 0x7f205c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:10.748 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.748+0000 7f206fe00640 1 -- 172.21.15.53:0/1809570553 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f20640d06d0 con 0x7f2078101ec0 2024-04-25T14:38:10.748 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.748+0000 7f2077400640 1 --2- 172.21.15.53:0/1809570553 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f205c0a3bb0 0x7f205c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:10.749 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.748+0000 7f2077400640 1 --2- 172.21.15.53:0/1809570553 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f205c0a3bb0 0x7f205c0a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2078101d20 tx=0x7f2068005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:10.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.752+0000 7f206fe00640 1 -- 172.21.15.53:0/1809570553 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f206409d160 con 0x7f2078101ec0 2024-04-25T14:38:10.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:10 smithi183 ceph-mon[32584]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T14:38:10.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:10 smithi183 ceph-mon[32584]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T14:38:10.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:10 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/338552465' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T14:38:10.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:10 smithi183 ceph-mon[32584]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T14:38:10.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:10 smithi183 ceph-mon[32584]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T14:38:10.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:10 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:10.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:10 smithi183 ceph-mon[32584]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T14:38:10.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:10 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:10.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:10 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:10.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:10 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:10.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:10 smithi150 ceph-mon[32504]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T14:38:10.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:10 smithi150 ceph-mon[32504]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T14:38:10.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:10 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/338552465' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T14:38:10.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:10 smithi150 ceph-mon[32504]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T14:38:10.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:10 smithi150 ceph-mon[32504]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T14:38:10.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:10 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:10.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:10 smithi150 ceph-mon[32504]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T14:38:10.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:10 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:10.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:10 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:10.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:10 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:10.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.986+0000 7f207d200640 1 -- 172.21.15.53:0/1809570553 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f20781a2f50 con 0x7f2078101ec0 2024-04-25T14:38:10.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.988+0000 7f206fe00640 1 -- 172.21.15.53:0/1809570553 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22184 (secure 0 0 0) 0x7f206409ca60 con 0x7f2078101ec0 2024-04-25T14:38:10.990 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:38:10.990 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":87,"fsid":"fd30e6ce-030f-11ef-bc93-c7b262605968","created":"2024-04-25T14:28:54.330326+0000","modified":"2024-04-25T14:38:10.037503+0000","last_up_change":"2024-04-25T14:37:10.921148+0000","last_in_change":"2024-04-25T14:36:49.566459+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:33:02.338115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:38:07.517974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:38:08.994122+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aecfbaf0-39da-4749-a49b-c7205dabd4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6803","nonce":464176015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6805","nonce":464176015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6809","nonce":464176015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6807","nonce":464176015}]},"public_addr":"172.21.15.53:6803/464176015","cluster_addr":"172.21.15.53:6805/464176015","heartbeat_back_addr":"172.21.15.53:6809/464176015","heartbeat_front_addr":"172.21.15.53:6807/464176015","state":["exists","up"]},{"osd":1,"uuid":"2c4a21ae-c43e-4839-8a56-c3fc63aeb7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6801","nonce":1304621982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6803","nonce":1304621982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6807","nonce":1304621982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6805","nonce":1304621982}]},"public_addr":"172.21.15.150:6801/1304621982","cluster_addr":"172.21.15.150:6803/1304621982","heartbeat_back_addr":"172.21.15.150:6807/1304621982","heartbeat_front_addr":"172.21.15.150:6805/1304621982","state":["exists","up"]},{"osd":2,"uuid":"e8effd43-adc0-472a-9eed-a9273b06dc0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6801","nonce":574509311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6803","nonce":574509311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6807","nonce":574509311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6805","nonce":574509311}]},"public_addr":"172.21.15.183:6801/574509311","cluster_addr":"172.21.15.183:6803/574509311","heartbeat_back_addr":"172.21.15.183:6807/574509311","heartbeat_front_addr":"172.21.15.183:6805/574509311","state":["exists","up"]},{"osd":3,"uuid":"e7cfced4-9b40-40de-a303-75ea8d5df4a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6811","nonce":570672219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6813","nonce":570672219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6817","nonce":570672219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6815","nonce":570672219}]},"public_addr":"172.21.15.53:6811/570672219","cluster_addr":"172.21.15.53:6813/570672219","heartbeat_back_addr":"172.21.15.53:6817/570672219","heartbeat_front_addr":"172.21.15.53:6815/570672219","state":["exists","up"]},{"osd":4,"uuid":"44a9e070-2282-43f1-9f05-b1f8ff81e540","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6809","nonce":1526850103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6811","nonce":1526850103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6815","nonce":1526850103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6813","nonce":1526850103}]},"public_addr":"172.21.15.150:6809/1526850103","cluster_addr":"172.21.15.150:6811/1526850103","heartbeat_back_addr":"172.21.15.150:6815/1526850103","heartbeat_front_addr":"172.21.15.150:6813/1526850103","state":["exists","up"]},{"osd":5,"uuid":"2c5619d2-c0f9-47d6-bf7e-2f4afd1b09f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6809","nonce":773726576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6811","nonce":773726576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6815","nonce":773726576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6813","nonce":773726576}]},"public_addr":"172.21.15.183:6809/773726576","cluster_addr":"172.21.15.183:6811/773726576","heartbeat_back_addr":"172.21.15.183:6815/773726576","heartbeat_front_addr":"172.21.15.183:6813/773726576","state":["exists","up"]},{"osd":6,"uuid":"1cf74d94-c118-4738-b6db-040af458a8eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6819","nonce":355884319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6821","nonce":355884319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6825","nonce":355884319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6823","nonce":355884319}]},"public_addr":"172.21.15.53:6819/355884319","cluster_addr":"172.21.15.53:6821/355884319","heartbeat_back_addr":"172.21.15.53:6825/355884319","heartbeat_front_addr":"172.21.15.53:6823/355884319","state":["exists","up"]},{"osd":7,"uuid":"0ce97c04-cd41-4080-bbbc-be9cf5fadc02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6817","nonce":229022562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6819","nonce":229022562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6823","nonce":229022562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6821","nonce":229022562}]},"public_addr":"172.21.15.150:6817/229022562","cluster_addr":"172.21.15.150:6819/229022562","heartbeat_back_addr":"172.21.15.150:6823/229022562","heartbeat_front_addr":"172.21.15.150:6821/229022562","state":["exists","up"]},{"osd":8,"uuid":"5d57787d-2a7c-4514-bd01-357ced413f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6817","nonce":301003844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6819","nonce":301003844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6823","nonce":301003844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6821","nonce":301003844}]},"public_addr":"172.21.15.183:6817/301003844","cluster_addr":"172.21.15.183:6819/301003844","heartbeat_back_addr":"172.21.15.183:6823/301003844","heartbeat_front_addr":"172.21.15.183:6821/301003844","state":["exists","up"]},{"osd":9,"uuid":"e11dd8eb-c55a-4d70-8c58-457540b7a50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6827","nonce":3449260978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6829","nonce":3449260978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6833","nonce":3449260978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6831","nonce":3449260978}]},"public_addr":"172.21.15.53:6827/3449260978","cluster_addr":"172.21.15.53:6829/3449260978","heartbeat_back_addr":"172.21.15.53:6833/3449260978","heartbeat_front_addr":"172.21.15.53:6831/3449260978","state":["exists","up"]},{"osd":10,"uuid":"5de4803d-ad37-4a90-add5-0048f44fc4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6825","nonce":3435992308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6827","nonce":3435992308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6831","nonce":3435992308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6829","nonce":3435992308}]},"public_addr":"172.21.15.150:6825/3435992308","cluster_addr":"172.21.15.150:6827/3435992308","heartbeat_back_addr":"172.21.15.150:6831/3435992308","heartbeat_front_addr":"172.21.15.150:6829/3435992308","state":["exists","up"]},{"osd":11,"uuid":"0e819a68-ed56-477e-b9c0-81e8cc0bce11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6825","nonce":1957842686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6827","nonce":1957842686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6831","nonce":1957842686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6829","nonce":1957842686}]},"public_addr":"172.21.15.183:6825/1957842686","cluster_addr":"172.21.15.183:6827/1957842686","heartbeat_back_addr":"172.21.15.183:6831/1957842686","heartbeat_front_addr":"172.21.15.183:6829/1957842686","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:09.840629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:34.769537+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:58.112969+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:33:29.656790+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:33:55.181287+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:34:19.772645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:34:53.833921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:35:19.076733+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:35:43.163390+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:36:16.568340+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:36:42.492768+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:37:07.944639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6800/3887954889":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/1014501022":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/2576091441":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/1233527569":"2024-04-26T14:29:38.106242+0000","172.21.15.53:6801/3887954889":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/931889810":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/177196141":"2024-04-26T14:30:14.162383+0000","172.21.15.53:6800/1722497390":"2024-04-26T14:29:38.106242+0000","172.21.15.53:6801/1722497390":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/1927549198":"2024-04-26T14:30:14.162383+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:38:10.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.991+0000 7f207d200640 1 -- 172.21.15.53:0/1809570553 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f205c0a3bb0 msgr2=0x7f205c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:10.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.991+0000 7f207d200640 1 --2- 172.21.15.53:0/1809570553 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f205c0a3bb0 0x7f205c0a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2078101d20 tx=0x7f2068005f90 comp rx=0 tx=0).stop 2024-04-25T14:38:10.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.991+0000 7f207d200640 1 -- 172.21.15.53:0/1809570553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2078101ec0 msgr2=0x7f20781032a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:10.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.991+0000 7f207d200640 1 --2- 172.21.15.53:0/1809570553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2078101ec0 0x7f20781032a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f206400e9f0 tx=0x7f206400eeb0 comp rx=0 tx=0).stop 2024-04-25T14:38:10.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.991+0000 7f207d200640 1 -- 172.21.15.53:0/1809570553 shutdown_connections 2024-04-25T14:38:10.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.991+0000 7f207d200640 1 --2- 172.21.15.53:0/1809570553 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f205c0a3bb0 0x7f205c0a6070 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:10.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.991+0000 7f207d200640 1 --2- 172.21.15.53:0/1809570553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2078102880 0x7f20781037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:10.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.991+0000 7f207d200640 1 --2- 172.21.15.53:0/1809570553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2078101ec0 0x7f20781032a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:10.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.991+0000 7f207d200640 1 --2- 172.21.15.53:0/1809570553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2078100cd0 0x7f2078104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:10.993 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.991+0000 7f207d200640 1 -- 172.21.15.53:0/1809570553 >> 172.21.15.53:0/1809570553 conn(0x7f20780fc220 msgr2=0x7f20780fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:10.993 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.991+0000 7f207d200640 1 -- 172.21.15.53:0/1809570553 shutdown_connections 2024-04-25T14:38:10.993 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:10.991+0000 7f207d200640 1 -- 172.21.15.53:0/1809570553 wait complete. 2024-04-25T14:38:11.007 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:10 smithi053 ceph-mon[27773]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T14:38:11.008 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:10 smithi053 ceph-mon[27773]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T14:38:11.008 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:10 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/338552465' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T14:38:11.008 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:10 smithi053 ceph-mon[27773]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T14:38:11.008 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:10 smithi053 ceph-mon[27773]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T14:38:11.008 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:10 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:11.008 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:10 smithi053 ceph-mon[27773]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T14:38:11.008 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:10 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:11.008 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:10 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:11.008 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:10 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:11.010 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:38:11.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.202+0000 7f1b48a00640 1 -- 172.21.15.53:0/4215997294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b44077b50 msgr2=0x7f1b44077fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:11.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.202+0000 7f1b48a00640 1 --2- 172.21.15.53:0/4215997294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b44077b50 0x7f1b44077fb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1b30009900 tx=0x7f1b3002a410 comp rx=0 tx=0).stop 2024-04-25T14:38:11.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.202+0000 7f1b48a00640 1 -- 172.21.15.53:0/4215997294 shutdown_connections 2024-04-25T14:38:11.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.202+0000 7f1b48a00640 1 --2- 172.21.15.53:0/4215997294 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b440784f0 0x7f1b44078d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:11.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.202+0000 7f1b48a00640 1 --2- 172.21.15.53:0/4215997294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b44077b50 0x7f1b44077fb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:11.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.202+0000 7f1b48a00640 1 --2- 172.21.15.53:0/4215997294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b44076910 0x7f1b44076d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:11.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.202+0000 7f1b48a00640 1 -- 172.21.15.53:0/4215997294 >> 172.21.15.53:0/4215997294 conn(0x7f1b440ff740 msgr2=0x7f1b44101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:11.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.202+0000 7f1b48a00640 1 -- 172.21.15.53:0/4215997294 shutdown_connections 2024-04-25T14:38:11.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.202+0000 7f1b48a00640 1 -- 172.21.15.53:0/4215997294 wait complete. 2024-04-25T14:38:11.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.203+0000 7f1b48a00640 1 Processor -- start 2024-04-25T14:38:11.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.203+0000 7f1b48a00640 1 -- start start 2024-04-25T14:38:11.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.204+0000 7f1b48a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b44076910 0x7f1b4419e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:11.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.204+0000 7f1b48a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b44077b50 0x7f1b4419ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:11.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.204+0000 7f1b48a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b440784f0 0x7f1b441a5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:11.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.204+0000 7f1b48a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1b44114130 con 0x7f1b44077b50 2024-04-25T14:38:11.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.204+0000 7f1b48a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1b44113fb0 con 0x7f1b440784f0 2024-04-25T14:38:11.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.204+0000 7f1b48a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1b441142b0 con 0x7f1b44076910 2024-04-25T14:38:11.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.204+0000 7f1b41e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b44076910 0x7f1b4419e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:11.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.204+0000 7f1b41e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b44076910 0x7f1b4419e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:41962/0 (socket says 172.21.15.53:41962) 2024-04-25T14:38:11.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.204+0000 7f1b41e00640 1 -- 172.21.15.53:0/3043029369 learned_addr learned my addr 172.21.15.53:0/3043029369 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:38:11.207 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.204+0000 7f1b41400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b44077b50 0x7f1b4419ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:11.207 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.204+0000 7f1b42800640 1 --2- 172.21.15.53:0/3043029369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b440784f0 0x7f1b441a5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:11.207 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.204+0000 7f1b41e00640 1 -- 172.21.15.53:0/3043029369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b440784f0 msgr2=0x7f1b441a5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:11.207 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.204+0000 7f1b41e00640 1 --2- 172.21.15.53:0/3043029369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b440784f0 0x7f1b441a5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:11.207 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.204+0000 7f1b41e00640 1 -- 172.21.15.53:0/3043029369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b44077b50 msgr2=0x7f1b4419ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:11.207 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.204+0000 7f1b41e00640 1 --2- 172.21.15.53:0/3043029369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b44077b50 0x7f1b4419ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:11.207 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.204+0000 7f1b41e00640 1 -- 172.21.15.53:0/3043029369 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b441a63f0 con 0x7f1b44076910 2024-04-25T14:38:11.207 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.205+0000 7f1b41e00640 1 --2- 172.21.15.53:0/3043029369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b44076910 0x7f1b4419e7d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1b3400d950 tx=0x7f1b3400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:11.207 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.205+0000 7f1b3aa00640 1 -- 172.21.15.53:0/3043029369 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b34017070 con 0x7f1b44076910 2024-04-25T14:38:11.207 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.205+0000 7f1b48a00640 1 -- 172.21.15.53:0/3043029369 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b441a66e0 con 0x7f1b44076910 2024-04-25T14:38:11.207 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.205+0000 7f1b48a00640 1 -- 172.21.15.53:0/3043029369 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b441a6c50 con 0x7f1b44076910 2024-04-25T14:38:11.207 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.205+0000 7f1b3aa00640 1 -- 172.21.15.53:0/3043029369 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b34009c30 con 0x7f1b44076910 2024-04-25T14:38:11.207 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.205+0000 7f1b3aa00640 1 -- 172.21.15.53:0/3043029369 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b34020060 con 0x7f1b44076910 2024-04-25T14:38:11.208 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.207+0000 7f1b48a00640 1 -- 172.21.15.53:0/3043029369 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b44110d50 con 0x7f1b44076910 2024-04-25T14:38:11.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.210+0000 7f1b3aa00640 1 -- 172.21.15.53:0/3043029369 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1b34010460 con 0x7f1b44076910 2024-04-25T14:38:11.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.212+0000 7f1b3aa00640 1 --2- 172.21.15.53:0/3043029369 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f1b280a3c30 0x7f1b280a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:11.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.212+0000 7f1b3aa00640 1 -- 172.21.15.53:0/3043029369 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f1b340cfae0 con 0x7f1b44076910 2024-04-25T14:38:11.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.213+0000 7f1b41400640 1 --2- 172.21.15.53:0/3043029369 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f1b280a3c30 0x7f1b280a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:11.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.213+0000 7f1b41400640 1 --2- 172.21.15.53:0/3043029369 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f1b280a3c30 0x7f1b280a60f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1b30031500 tx=0x7f1b30005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:11.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.216+0000 7f1b3aa00640 1 -- 172.21.15.53:0/3043029369 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1b3409bf20 con 0x7f1b44076910 2024-04-25T14:38:11.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.448+0000 7f1b48a00640 1 -- 172.21.15.53:0/3043029369 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1b44062b50 con 0x7f1b44076910 2024-04-25T14:38:11.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.450+0000 7f1b3aa00640 1 -- 172.21.15.53:0/3043029369 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22171 (secure 0 0 0) 0x7f1b340a0220 con 0x7f1b44076910 2024-04-25T14:38:11.451 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:38:11.451 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":88,"fsid":"fd30e6ce-030f-11ef-bc93-c7b262605968","created":"2024-04-25T14:28:54.330326+0000","modified":"2024-04-25T14:38:11.415753+0000","last_up_change":"2024-04-25T14:37:10.921148+0000","last_in_change":"2024-04-25T14:36:49.566459+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:33:02.338115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:38:07.517974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:38:08.994122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aecfbaf0-39da-4749-a49b-c7205dabd4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6803","nonce":464176015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6805","nonce":464176015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6809","nonce":464176015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6807","nonce":464176015}]},"public_addr":"172.21.15.53:6803/464176015","cluster_addr":"172.21.15.53:6805/464176015","heartbeat_back_addr":"172.21.15.53:6809/464176015","heartbeat_front_addr":"172.21.15.53:6807/464176015","state":["exists","up"]},{"osd":1,"uuid":"2c4a21ae-c43e-4839-8a56-c3fc63aeb7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6801","nonce":1304621982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6803","nonce":1304621982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6807","nonce":1304621982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6805","nonce":1304621982}]},"public_addr":"172.21.15.150:6801/1304621982","cluster_addr":"172.21.15.150:6803/1304621982","heartbeat_back_addr":"172.21.15.150:6807/1304621982","heartbeat_front_addr":"172.21.15.150:6805/1304621982","state":["exists","up"]},{"osd":2,"uuid":"e8effd43-adc0-472a-9eed-a9273b06dc0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6801","nonce":574509311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6803","nonce":574509311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6807","nonce":574509311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6805","nonce":574509311}]},"public_addr":"172.21.15.183:6801/574509311","cluster_addr":"172.21.15.183:6803/574509311","heartbeat_back_addr":"172.21.15.183:6807/574509311","heartbeat_front_addr":"172.21.15.183:6805/574509311","state":["exists","up"]},{"osd":3,"uuid":"e7cfced4-9b40-40de-a303-75ea8d5df4a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6811","nonce":570672219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6813","nonce":570672219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6817","nonce":570672219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6815","nonce":570672219}]},"public_addr":"172.21.15.53:6811/570672219","cluster_addr":"172.21.15.53:6813/570672219","heartbeat_back_addr":"172.21.15.53:6817/570672219","heartbeat_front_addr":"172.21.15.53:6815/570672219","state":["exists","up"]},{"osd":4,"uuid":"44a9e070-2282-43f1-9f05-b1f8ff81e540","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6809","nonce":1526850103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6811","nonce":1526850103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6815","nonce":1526850103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6813","nonce":1526850103}]},"public_addr":"172.21.15.150:6809/1526850103","cluster_addr":"172.21.15.150:6811/1526850103","heartbeat_back_addr":"172.21.15.150:6815/1526850103","heartbeat_front_addr":"172.21.15.150:6813/1526850103","state":["exists","up"]},{"osd":5,"uuid":"2c5619d2-c0f9-47d6-bf7e-2f4afd1b09f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6809","nonce":773726576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6811","nonce":773726576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6815","nonce":773726576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6813","nonce":773726576}]},"public_addr":"172.21.15.183:6809/773726576","cluster_addr":"172.21.15.183:6811/773726576","heartbeat_back_addr":"172.21.15.183:6815/773726576","heartbeat_front_addr":"172.21.15.183:6813/773726576","state":["exists","up"]},{"osd":6,"uuid":"1cf74d94-c118-4738-b6db-040af458a8eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6819","nonce":355884319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6821","nonce":355884319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6825","nonce":355884319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6823","nonce":355884319}]},"public_addr":"172.21.15.53:6819/355884319","cluster_addr":"172.21.15.53:6821/355884319","heartbeat_back_addr":"172.21.15.53:6825/355884319","heartbeat_front_addr":"172.21.15.53:6823/355884319","state":["exists","up"]},{"osd":7,"uuid":"0ce97c04-cd41-4080-bbbc-be9cf5fadc02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6817","nonce":229022562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6819","nonce":229022562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6823","nonce":229022562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6821","nonce":229022562}]},"public_addr":"172.21.15.150:6817/229022562","cluster_addr":"172.21.15.150:6819/229022562","heartbeat_back_addr":"172.21.15.150:6823/229022562","heartbeat_front_addr":"172.21.15.150:6821/229022562","state":["exists","up"]},{"osd":8,"uuid":"5d57787d-2a7c-4514-bd01-357ced413f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6817","nonce":301003844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6819","nonce":301003844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6823","nonce":301003844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6821","nonce":301003844}]},"public_addr":"172.21.15.183:6817/301003844","cluster_addr":"172.21.15.183:6819/301003844","heartbeat_back_addr":"172.21.15.183:6823/301003844","heartbeat_front_addr":"172.21.15.183:6821/301003844","state":["exists","up"]},{"osd":9,"uuid":"e11dd8eb-c55a-4d70-8c58-457540b7a50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6827","nonce":3449260978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6829","nonce":3449260978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6833","nonce":3449260978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6831","nonce":3449260978}]},"public_addr":"172.21.15.53:6827/3449260978","cluster_addr":"172.21.15.53:6829/3449260978","heartbeat_back_addr":"172.21.15.53:6833/3449260978","heartbeat_front_addr":"172.21.15.53:6831/3449260978","state":["exists","up"]},{"osd":10,"uuid":"5de4803d-ad37-4a90-add5-0048f44fc4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6825","nonce":3435992308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6827","nonce":3435992308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6831","nonce":3435992308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6829","nonce":3435992308}]},"public_addr":"172.21.15.150:6825/3435992308","cluster_addr":"172.21.15.150:6827/3435992308","heartbeat_back_addr":"172.21.15.150:6831/3435992308","heartbeat_front_addr":"172.21.15.150:6829/3435992308","state":["exists","up"]},{"osd":11,"uuid":"0e819a68-ed56-477e-b9c0-81e8cc0bce11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6825","nonce":1957842686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6827","nonce":1957842686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6831","nonce":1957842686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6829","nonce":1957842686}]},"public_addr":"172.21.15.183:6825/1957842686","cluster_addr":"172.21.15.183:6827/1957842686","heartbeat_back_addr":"172.21.15.183:6831/1957842686","heartbeat_front_addr":"172.21.15.183:6829/1957842686","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:09.840629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:34.769537+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:58.112969+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:33:29.656790+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:33:55.181287+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:34:19.772645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:34:53.833921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:35:19.076733+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:35:43.163390+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:36:16.568340+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:36:42.492768+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:37:07.944639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6800/3887954889":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/1014501022":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/2576091441":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/1233527569":"2024-04-26T14:29:38.106242+0000","172.21.15.53:6801/3887954889":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/931889810":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/177196141":"2024-04-26T14:30:14.162383+0000","172.21.15.53:6800/1722497390":"2024-04-26T14:29:38.106242+0000","172.21.15.53:6801/1722497390":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/1927549198":"2024-04-26T14:30:14.162383+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:38:11.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.453+0000 7f1b48a00640 1 -- 172.21.15.53:0/3043029369 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f1b280a3c30 msgr2=0x7f1b280a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:11.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.453+0000 7f1b48a00640 1 --2- 172.21.15.53:0/3043029369 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f1b280a3c30 0x7f1b280a60f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1b30031500 tx=0x7f1b30005a20 comp rx=0 tx=0).stop 2024-04-25T14:38:11.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.453+0000 7f1b48a00640 1 -- 172.21.15.53:0/3043029369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b44076910 msgr2=0x7f1b4419e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:11.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.453+0000 7f1b48a00640 1 --2- 172.21.15.53:0/3043029369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b44076910 0x7f1b4419e7d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1b3400d950 tx=0x7f1b3400de10 comp rx=0 tx=0).stop 2024-04-25T14:38:11.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.453+0000 7f1b48a00640 1 -- 172.21.15.53:0/3043029369 shutdown_connections 2024-04-25T14:38:11.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.453+0000 7f1b48a00640 1 --2- 172.21.15.53:0/3043029369 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f1b280a3c30 0x7f1b280a60f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:11.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.453+0000 7f1b48a00640 1 --2- 172.21.15.53:0/3043029369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b440784f0 0x7f1b441a5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:11.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.453+0000 7f1b48a00640 1 --2- 172.21.15.53:0/3043029369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b44077b50 0x7f1b4419ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:11.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.453+0000 7f1b48a00640 1 --2- 172.21.15.53:0/3043029369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b44076910 0x7f1b4419e7d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:11.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.453+0000 7f1b48a00640 1 -- 172.21.15.53:0/3043029369 >> 172.21.15.53:0/3043029369 conn(0x7f1b440ff740 msgr2=0x7f1b44100ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:11.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.453+0000 7f1b48a00640 1 -- 172.21.15.53:0/3043029369 shutdown_connections 2024-04-25T14:38:11.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.454+0000 7f1b48a00640 1 -- 172.21.15.53:0/3043029369 wait complete. 2024-04-25T14:38:11.472 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T14:38:11.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.659+0000 7f8a99200640 1 -- 172.21.15.53:0/359101865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a941053c0 msgr2=0x7f8a94105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:11.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.659+0000 7f8a99200640 1 --2- 172.21.15.53:0/359101865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a941053c0 0x7f8a94105840 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8a80009900 tx=0x7f8a8002a3f0 comp rx=0 tx=0).stop 2024-04-25T14:38:11.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.659+0000 7f8a99200640 1 -- 172.21.15.53:0/359101865 shutdown_connections 2024-04-25T14:38:11.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.659+0000 7f8a99200640 1 --2- 172.21.15.53:0/359101865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a94105d80 0x7f8a9410c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:11.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.659+0000 7f8a99200640 1 --2- 172.21.15.53:0/359101865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a941053c0 0x7f8a94105840 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:11.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.659+0000 7f8a99200640 1 --2- 172.21.15.53:0/359101865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a941041d0 0x7f8a941045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:11.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.659+0000 7f8a99200640 1 -- 172.21.15.53:0/359101865 >> 172.21.15.53:0/359101865 conn(0x7f8a940ff740 msgr2=0x7f8a94101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:11.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.659+0000 7f8a99200640 1 -- 172.21.15.53:0/359101865 shutdown_connections 2024-04-25T14:38:11.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.659+0000 7f8a99200640 1 -- 172.21.15.53:0/359101865 wait complete. 2024-04-25T14:38:11.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.660+0000 7f8a99200640 1 Processor -- start 2024-04-25T14:38:11.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.660+0000 7f8a99200640 1 -- start start 2024-04-25T14:38:11.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.660+0000 7f8a99200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a941041d0 0x7f8a9419a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:11.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.660+0000 7f8a99200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a941053c0 0x7f8a9419a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:11.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.660+0000 7f8a99200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a94105d80 0x7f8a941a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:11.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.661+0000 7f8a99200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9410fd60 con 0x7f8a94105d80 2024-04-25T14:38:11.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.661+0000 7f8a99200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9410fbe0 con 0x7f8a941041d0 2024-04-25T14:38:11.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.661+0000 7f8a99200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9410fee0 con 0x7f8a941053c0 2024-04-25T14:38:11.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.661+0000 7f8a91400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a941053c0 0x7f8a9419a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:11.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.661+0000 7f8a91e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a941041d0 0x7f8a9419a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:11.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.661+0000 7f8a91400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a941053c0 0x7f8a9419a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:41976/0 (socket says 172.21.15.53:41976) 2024-04-25T14:38:11.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.661+0000 7f8a91400640 1 -- 172.21.15.53:0/1296466863 learned_addr learned my addr 172.21.15.53:0/1296466863 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:38:11.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.661+0000 7f8a92800640 1 --2- 172.21.15.53:0/1296466863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a94105d80 0x7f8a941a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:11.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.661+0000 7f8a91e00640 1 -- 172.21.15.53:0/1296466863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a941053c0 msgr2=0x7f8a9419a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:11.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.661+0000 7f8a91e00640 1 --2- 172.21.15.53:0/1296466863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a941053c0 0x7f8a9419a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:11.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.661+0000 7f8a91e00640 1 -- 172.21.15.53:0/1296466863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a94105d80 msgr2=0x7f8a941a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:11.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.661+0000 7f8a91e00640 1 --2- 172.21.15.53:0/1296466863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a94105d80 0x7f8a941a19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:11.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.661+0000 7f8a91e00640 1 -- 172.21.15.53:0/1296466863 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a941a2010 con 0x7f8a941041d0 2024-04-25T14:38:11.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.661+0000 7f8a92800640 1 --2- 172.21.15.53:0/1296466863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a94105d80 0x7f8a941a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:38:11.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.662+0000 7f8a91e00640 1 --2- 172.21.15.53:0/1296466863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a941041d0 0x7f8a9419a3f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8a8400e9c0 tx=0x7f8a8400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:11.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.662+0000 7f8a8aa00640 1 -- 172.21.15.53:0/1296466863 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8400ccc0 con 0x7f8a941041d0 2024-04-25T14:38:11.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.662+0000 7f8a8aa00640 1 -- 172.21.15.53:0/1296466863 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a840035f0 con 0x7f8a941041d0 2024-04-25T14:38:11.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.662+0000 7f8a8aa00640 1 -- 172.21.15.53:0/1296466863 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8400ce60 con 0x7f8a941041d0 2024-04-25T14:38:11.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.662+0000 7f8a99200640 1 -- 172.21.15.53:0/1296466863 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a941a2300 con 0x7f8a941041d0 2024-04-25T14:38:11.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.662+0000 7f8a99200640 1 -- 172.21.15.53:0/1296466863 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a941a2800 con 0x7f8a941041d0 2024-04-25T14:38:11.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.663+0000 7f8a99200640 1 -- 172.21.15.53:0/1296466863 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a9410b570 con 0x7f8a941041d0 2024-04-25T14:38:11.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.665+0000 7f8a8aa00640 1 -- 172.21.15.53:0/1296466863 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8a84004000 con 0x7f8a941041d0 2024-04-25T14:38:11.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.666+0000 7f8a8aa00640 1 --2- 172.21.15.53:0/1296466863 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f8a780a3c30 0x7f8a780a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:11.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.666+0000 7f8a8aa00640 1 -- 172.21.15.53:0/1296466863 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f8a840d06b0 con 0x7f8a941041d0 2024-04-25T14:38:11.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.667+0000 7f8a91400640 1 --2- 172.21.15.53:0/1296466863 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f8a780a3c30 0x7f8a780a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:11.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.667+0000 7f8a91400640 1 --2- 172.21.15.53:0/1296466863 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f8a780a3c30 0x7f8a780a60f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8a9419b900 tx=0x7f8a80005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:11.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.670+0000 7f8a8aa00640 1 -- 172.21.15.53:0/1296466863 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8a8409d140 con 0x7f8a941041d0 2024-04-25T14:38:11.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:11 smithi053 ceph-mon[27773]: pgmap v314: 97 pgs: 10 creating+peering, 4 active+clean, 83 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:38:11.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:11 smithi053 ceph-mon[27773]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T14:38:11.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:11 smithi053 ceph-mon[27773]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T14:38:11.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:11 smithi053 ceph-mon[27773]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T14:38:11.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:11 smithi053 ceph-mon[27773]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T14:38:11.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:11 smithi053 ceph-mon[27773]: mds.? [v2:172.21.15.150:6838/149573616,v1:172.21.15.150:6839/149573616] up:active 2024-04-25T14:38:11.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:11 smithi053 ceph-mon[27773]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T14:38:11.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:11 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1809570553' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:38:11.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:11 smithi053 ceph-mon[27773]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T14:38:11.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:11 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/3043029369' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:38:11.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:11.936+0000 7f8a99200640 1 -- 172.21.15.53:0/1296466863 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f8a94062b80 con 0x7f8a941041d0 2024-04-25T14:38:11.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:11 smithi183 ceph-mon[32584]: pgmap v314: 97 pgs: 10 creating+peering, 4 active+clean, 83 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:38:11.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:11 smithi183 ceph-mon[32584]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T14:38:11.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:11 smithi183 ceph-mon[32584]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T14:38:11.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:11 smithi183 ceph-mon[32584]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T14:38:11.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:11 smithi183 ceph-mon[32584]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T14:38:11.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:11 smithi183 ceph-mon[32584]: mds.? [v2:172.21.15.150:6838/149573616,v1:172.21.15.150:6839/149573616] up:active 2024-04-25T14:38:11.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:11 smithi183 ceph-mon[32584]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T14:38:11.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:11 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/1809570553' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:38:11.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:11 smithi183 ceph-mon[32584]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T14:38:11.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:11 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/3043029369' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:38:11.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:11 smithi150 ceph-mon[32504]: pgmap v314: 97 pgs: 10 creating+peering, 4 active+clean, 83 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:38:11.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:11 smithi150 ceph-mon[32504]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T14:38:11.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:11 smithi150 ceph-mon[32504]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T14:38:11.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:11 smithi150 ceph-mon[32504]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T14:38:11.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:11 smithi150 ceph-mon[32504]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T14:38:11.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:11 smithi150 ceph-mon[32504]: mds.? [v2:172.21.15.150:6838/149573616,v1:172.21.15.150:6839/149573616] up:active 2024-04-25T14:38:11.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:11 smithi150 ceph-mon[32504]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T14:38:11.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:11 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/1809570553' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:38:11.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:11 smithi150 ceph-mon[32504]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T14:38:11.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:11 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/3043029369' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:38:12.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.559+0000 7f8a8aa00640 1 -- 172.21.15.53:0/1296466863 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7f8a8409ca40 con 0x7f8a941041d0 2024-04-25T14:38:12.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.562+0000 7f8a99200640 1 -- 172.21.15.53:0/1296466863 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f8a780a3c30 msgr2=0x7f8a780a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:12.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.562+0000 7f8a99200640 1 --2- 172.21.15.53:0/1296466863 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f8a780a3c30 0x7f8a780a60f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8a9419b900 tx=0x7f8a80005a20 comp rx=0 tx=0).stop 2024-04-25T14:38:12.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.562+0000 7f8a99200640 1 -- 172.21.15.53:0/1296466863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a941041d0 msgr2=0x7f8a9419a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:12.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.562+0000 7f8a99200640 1 --2- 172.21.15.53:0/1296466863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a941041d0 0x7f8a9419a3f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8a8400e9c0 tx=0x7f8a8400ee80 comp rx=0 tx=0).stop 2024-04-25T14:38:12.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.562+0000 7f8a99200640 1 -- 172.21.15.53:0/1296466863 shutdown_connections 2024-04-25T14:38:12.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.562+0000 7f8a99200640 1 --2- 172.21.15.53:0/1296466863 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f8a780a3c30 0x7f8a780a60f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:12.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.562+0000 7f8a99200640 1 --2- 172.21.15.53:0/1296466863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a94105d80 0x7f8a941a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:12.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.563+0000 7f8a99200640 1 --2- 172.21.15.53:0/1296466863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a941053c0 0x7f8a9419a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:12.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.563+0000 7f8a99200640 1 --2- 172.21.15.53:0/1296466863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a941041d0 0x7f8a9419a3f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:12.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.563+0000 7f8a99200640 1 -- 172.21.15.53:0/1296466863 >> 172.21.15.53:0/1296466863 conn(0x7f8a940ff740 msgr2=0x7f8a94108740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:12.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.563+0000 7f8a99200640 1 -- 172.21.15.53:0/1296466863 shutdown_connections 2024-04-25T14:38:12.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.563+0000 7f8a99200640 1 -- 172.21.15.53:0/1296466863 wait complete. 2024-04-25T14:38:12.586 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-04-25T14:38:12.586 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-04-25T14:38:12.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.792+0000 7f8240c00640 1 -- 172.21.15.53:0/3567575448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f823c109680 msgr2=0x7f823c109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:12.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.792+0000 7f8240c00640 1 --2- 172.21.15.53:0/3567575448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f823c109680 0x7f823c109b00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8224009900 tx=0x7f822402a3f0 comp rx=0 tx=0).stop 2024-04-25T14:38:12.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.793+0000 7f8240c00640 1 -- 172.21.15.53:0/3567575448 shutdown_connections 2024-04-25T14:38:12.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.793+0000 7f8240c00640 1 --2- 172.21.15.53:0/3567575448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f823c10a040 0x7f823c1108d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:12.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.793+0000 7f8240c00640 1 --2- 172.21.15.53:0/3567575448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f823c109680 0x7f823c109b00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:12.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.793+0000 7f8240c00640 1 --2- 172.21.15.53:0/3567575448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f823c108490 0x7f823c1088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:12.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.793+0000 7f8240c00640 1 -- 172.21.15.53:0/3567575448 >> 172.21.15.53:0/3567575448 conn(0x7f823c077420 msgr2=0x7f823c079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:12.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.793+0000 7f8240c00640 1 -- 172.21.15.53:0/3567575448 shutdown_connections 2024-04-25T14:38:12.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.793+0000 7f8240c00640 1 -- 172.21.15.53:0/3567575448 wait complete. 2024-04-25T14:38:12.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.794+0000 7f8240c00640 1 Processor -- start 2024-04-25T14:38:12.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.794+0000 7f8240c00640 1 -- start start 2024-04-25T14:38:12.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.795+0000 7f8240c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f823c108490 0x7f823c19e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:12.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.795+0000 7f8240c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f823c109680 0x7f823c19eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:12.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.795+0000 7f8240c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f823c10a040 0x7f823c1a5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:12.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.795+0000 7f8240c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f823c114020 con 0x7f823c10a040 2024-04-25T14:38:12.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.795+0000 7f8240c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f823c113ea0 con 0x7f823c108490 2024-04-25T14:38:12.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.795+0000 7f8240c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f823c1141a0 con 0x7f823c109680 2024-04-25T14:38:12.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.795+0000 7f823be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f823c10a040 0x7f823c1a5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:12.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.795+0000 7f8233e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f823c109680 0x7f823c19eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:12.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.795+0000 7f8233e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f823c109680 0x7f823c19eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:38926/0 (socket says 172.21.15.53:38926) 2024-04-25T14:38:12.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.795+0000 7f8233e00640 1 -- 172.21.15.53:0/3284514966 learned_addr learned my addr 172.21.15.53:0/3284514966 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:38:12.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.795+0000 7f823be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f823c10a040 0x7f823c1a5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36046/0 (socket says 172.21.15.53:36046) 2024-04-25T14:38:12.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.795+0000 7f823b400640 1 --2- 172.21.15.53:0/3284514966 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f823c108490 0x7f823c19e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:12.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.796+0000 7f823be00640 1 -- 172.21.15.53:0/3284514966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f823c109680 msgr2=0x7f823c19eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:12.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.796+0000 7f823be00640 1 --2- 172.21.15.53:0/3284514966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f823c109680 0x7f823c19eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:12.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.796+0000 7f823be00640 1 -- 172.21.15.53:0/3284514966 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f823c108490 msgr2=0x7f823c19e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:12.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.796+0000 7f823be00640 1 --2- 172.21.15.53:0/3284514966 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f823c108490 0x7f823c19e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:12.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.796+0000 7f823be00640 1 -- 172.21.15.53:0/3284514966 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f823c1a6270 con 0x7f823c10a040 2024-04-25T14:38:12.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.796+0000 7f823b400640 1 --2- 172.21.15.53:0/3284514966 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f823c108490 0x7f823c19e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:38:12.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.796+0000 7f823be00640 1 --2- 172.21.15.53:0/3284514966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f823c10a040 0x7f823c1a5c00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8234009fd0 tx=0x7f823400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:12.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.796+0000 7f8238c00640 1 -- 172.21.15.53:0/3284514966 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823400ce20 con 0x7f823c10a040 2024-04-25T14:38:12.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.796+0000 7f8240c00640 1 -- 172.21.15.53:0/3284514966 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f823c1a6560 con 0x7f823c10a040 2024-04-25T14:38:12.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.796+0000 7f8238c00640 1 -- 172.21.15.53:0/3284514966 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8234005020 con 0x7f823c10a040 2024-04-25T14:38:12.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.796+0000 7f8238c00640 1 -- 172.21.15.53:0/3284514966 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8234010690 con 0x7f823c10a040 2024-04-25T14:38:12.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.797+0000 7f8240c00640 1 -- 172.21.15.53:0/3284514966 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f823c1a6ab0 con 0x7f823c10a040 2024-04-25T14:38:12.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.798+0000 7f8240c00640 1 -- 172.21.15.53:0/3284514966 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f823c10f7b0 con 0x7f823c10a040 2024-04-25T14:38:12.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.801+0000 7f8238c00640 1 -- 172.21.15.53:0/3284514966 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f82340054d0 con 0x7f823c10a040 2024-04-25T14:38:12.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.802+0000 7f8238c00640 1 --2- 172.21.15.53:0/3284514966 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f82200a3bb0 0x7f82200a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:12.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.802+0000 7f8238c00640 1 -- 172.21.15.53:0/3284514966 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f82340d09f0 con 0x7f823c10a040 2024-04-25T14:38:12.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.802+0000 7f823b400640 1 --2- 172.21.15.53:0/3284514966 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f82200a3bb0 0x7f82200a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:12.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.803+0000 7f823b400640 1 --2- 172.21.15.53:0/3284514966 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f82200a3bb0 0x7f82200a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f823c1094e0 tx=0x7f822c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:12.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:12.804+0000 7f8238c00640 1 -- 172.21.15.53:0/3284514966 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f823409d480 con 0x7f823c10a040 2024-04-25T14:38:12.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:12 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/1296466863' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T14:38:12.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:12 smithi183 ceph-mon[32584]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T14:38:12.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:12 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:12.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:12 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:12.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:12 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:38:12.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:12 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:38:12.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:12 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:12.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:12 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/1296466863' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T14:38:12.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:12 smithi150 ceph-mon[32504]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T14:38:12.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:12 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:12.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:12 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:12.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:12 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:38:12.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:12 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:38:12.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:12 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:13.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:12 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1296466863' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T14:38:13.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:12 smithi053 ceph-mon[27773]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T14:38:13.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:12 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:13.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:12 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:13.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:12 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:38:13.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:12 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:38:13.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:12 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:38:13.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:13.055+0000 7f8240c00640 1 -- 172.21.15.53:0/3284514966 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f823c02d5c0 con 0x7f823c10a040 2024-04-25T14:38:13.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:13 smithi183 ceph-mon[32584]: pgmap v317: 97 pgs: 18 creating+peering, 67 active+clean, 12 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s 2024-04-25T14:38:13.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:13 smithi183 ceph-mon[32584]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T14:38:13.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:13 smithi183 ceph-mon[32584]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T14:38:13.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:13 smithi183 ceph-mon[32584]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T14:38:13.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:13 smithi183 ceph-mon[32584]: Cluster is now healthy 2024-04-25T14:38:13.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:13 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/3284514966' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T14:38:13.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:13 smithi150 ceph-mon[32504]: pgmap v317: 97 pgs: 18 creating+peering, 67 active+clean, 12 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s 2024-04-25T14:38:13.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:13 smithi150 ceph-mon[32504]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T14:38:13.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:13 smithi150 ceph-mon[32504]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T14:38:13.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:13 smithi150 ceph-mon[32504]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T14:38:13.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:13 smithi150 ceph-mon[32504]: Cluster is now healthy 2024-04-25T14:38:13.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:13 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/3284514966' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T14:38:13.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:13.982+0000 7f8238c00640 1 -- 172.21.15.53:0/3284514966 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v17) ==== 105+0+0 (secure 0 0 0) 0x7f823409cd80 con 0x7f823c10a040 2024-04-25T14:38:13.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:13.985+0000 7f8240c00640 1 -- 172.21.15.53:0/3284514966 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f82200a3bb0 msgr2=0x7f82200a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:13.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:13.985+0000 7f8240c00640 1 --2- 172.21.15.53:0/3284514966 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f82200a3bb0 0x7f82200a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f823c1094e0 tx=0x7f822c005f90 comp rx=0 tx=0).stop 2024-04-25T14:38:13.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:13.985+0000 7f8240c00640 1 -- 172.21.15.53:0/3284514966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f823c10a040 msgr2=0x7f823c1a5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:13.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:13.985+0000 7f8240c00640 1 --2- 172.21.15.53:0/3284514966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f823c10a040 0x7f823c1a5c00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8234009fd0 tx=0x7f823400ef10 comp rx=0 tx=0).stop 2024-04-25T14:38:13.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:13.985+0000 7f8240c00640 1 -- 172.21.15.53:0/3284514966 shutdown_connections 2024-04-25T14:38:13.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:13.985+0000 7f8240c00640 1 --2- 172.21.15.53:0/3284514966 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f82200a3bb0 0x7f82200a6070 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:13.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:13.985+0000 7f8240c00640 1 --2- 172.21.15.53:0/3284514966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f823c10a040 0x7f823c1a5c00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:13.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:13.985+0000 7f8240c00640 1 --2- 172.21.15.53:0/3284514966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f823c109680 0x7f823c19eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:13.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:13.985+0000 7f8240c00640 1 --2- 172.21.15.53:0/3284514966 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f823c108490 0x7f823c19e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:13.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:13.985+0000 7f8240c00640 1 -- 172.21.15.53:0/3284514966 >> 172.21.15.53:0/3284514966 conn(0x7f823c077420 msgr2=0x7f823c10c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:13.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:13.986+0000 7f8240c00640 1 -- 172.21.15.53:0/3284514966 shutdown_connections 2024-04-25T14:38:13.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:13.986+0000 7f8240c00640 1 -- 172.21.15.53:0/3284514966 wait complete. 2024-04-25T14:38:14.010 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:13 smithi053 ceph-mon[27773]: pgmap v317: 97 pgs: 18 creating+peering, 67 active+clean, 12 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s 2024-04-25T14:38:14.010 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:13 smithi053 ceph-mon[27773]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T14:38:14.010 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:13 smithi053 ceph-mon[27773]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T14:38:14.010 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:13 smithi053 ceph-mon[27773]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T14:38:14.010 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:13 smithi053 ceph-mon[27773]: Cluster is now healthy 2024-04-25T14:38:14.010 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:13 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/3284514966' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T14:38:14.013 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-25T14:38:14.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:14.213+0000 7f17b8e00640 1 -- 172.21.15.53:0/2811472040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17b41041b0 msgr2=0x7f17b41045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:14.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:14.213+0000 7f17b8e00640 1 --2- 172.21.15.53:0/2811472040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17b41041b0 0x7f17b41045d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f17a4009900 tx=0x7f17a402a3f0 comp rx=0 tx=0).stop 2024-04-25T14:38:14.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:14.213+0000 7f17b8e00640 1 -- 172.21.15.53:0/2811472040 shutdown_connections 2024-04-25T14:38:14.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:14.213+0000 7f17b8e00640 1 --2- 172.21.15.53:0/2811472040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17b4105d60 0x7f17b410c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:14.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:14.213+0000 7f17b8e00640 1 --2- 172.21.15.53:0/2811472040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17b41053a0 0x7f17b4105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:14.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:14.213+0000 7f17b8e00640 1 --2- 172.21.15.53:0/2811472040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17b41041b0 0x7f17b41045d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:14.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:14.213+0000 7f17b8e00640 1 -- 172.21.15.53:0/2811472040 >> 172.21.15.53:0/2811472040 conn(0x7f17b40ff700 msgr2=0x7f17b4101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:14.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:14.213+0000 7f17b8e00640 1 -- 172.21.15.53:0/2811472040 shutdown_connections 2024-04-25T14:38:14.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:14.213+0000 7f17b8e00640 1 -- 172.21.15.53:0/2811472040 wait complete. 2024-04-25T14:38:14.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:14.214+0000 7f17b8e00640 1 Processor -- start 2024-04-25T14:38:14.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:14.214+0000 7f17b8e00640 1 -- start start 2024-04-25T14:38:14.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:14.214+0000 7f17b8e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17b41041b0 0x7f17b419a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:14.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:14.214+0000 7f17b8e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17b41053a0 0x7f17b419a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:14.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:14.214+0000 7f17b8e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17b4105d60 0x7f17b41a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:14.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:14.214+0000 7f17b8e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f17b410fd10 con 0x7f17b4105d60 2024-04-25T14:38:14.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:14.214+0000 7f17b8e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f17b410fb90 con 0x7f17b41041b0 2024-04-25T14:38:14.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:14.214+0000 7f17b8e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f17b410fe90 con 0x7f17b41053a0 2024-04-25T14:38:14.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:14.215+0000 7f17b1e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17b41041b0 0x7f17b419a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:14.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:14.215+0000 7f17b1e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17b41041b0 0x7f17b419a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.53:38220/0 (socket says 172.21.15.53:38220) 2024-04-25T14:38:14.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:14.215+0000 7f17b1e00640 1 -- 172.21.15.53:0/1762755860 learned_addr learned my addr 172.21.15.53:0/1762755860 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:38:14.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:14.215+0000 7f17b2800640 1 --2- 172.21.15.53:0/1762755860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17b4105d60 0x7f17b41a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:14.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:14.215+0000 7f17abe00640 1 --2- 172.21.15.53:0/1762755860 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17b41053a0 0x7f17b419a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:14.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:14.215+0000 7f17b1e00640 1 -- 172.21.15.53:0/1762755860 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17b41053a0 msgr2=0x7f17b419a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:14.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:14.215+0000 7f17b1e00640 1 --2- 172.21.15.53:0/1762755860 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17b41053a0 0x7f17b419a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:14.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:14.215+0000 7f17b1e00640 1 -- 172.21.15.53:0/1762755860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17b4105d60 msgr2=0x7f17b41a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:14.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:14.215+0000 7f17b1e00640 1 --2- 172.21.15.53:0/1762755860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17b4105d60 0x7f17b41a19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:14.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:14.215+0000 7f17b1e00640 1 -- 172.21.15.53:0/1762755860 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17b41a2010 con 0x7f17b41041b0 2024-04-25T14:38:14.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:14.216+0000 7f17b1e00640 1 --2- 172.21.15.53:0/1762755860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17b41041b0 0x7f17b419a3f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f17a4031500 tx=0x7f17a4003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:14.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:14.216+0000 7f17aaa00640 1 -- 172.21.15.53:0/1762755860 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a403f070 con 0x7f17b41041b0 2024-04-25T14:38:14.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:14.216+0000 7f17aaa00640 1 -- 172.21.15.53:0/1762755860 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17a4031c50 con 0x7f17b41041b0 2024-04-25T14:38:14.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:14.216+0000 7f17aaa00640 1 -- 172.21.15.53:0/1762755860 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a4035030 con 0x7f17b41041b0 2024-04-25T14:38:14.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:14.216+0000 7f17b8e00640 1 -- 172.21.15.53:0/1762755860 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17b41a22a0 con 0x7f17b41041b0 2024-04-25T14:38:14.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:14.216+0000 7f17b8e00640 1 -- 172.21.15.53:0/1762755860 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17b41a2790 con 0x7f17b41041b0 2024-04-25T14:38:14.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:14.217+0000 7f17b8e00640 1 -- 172.21.15.53:0/1762755860 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17b410b550 con 0x7f17b41041b0 2024-04-25T14:38:14.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:14.219+0000 7f17aaa00640 1 -- 172.21.15.53:0/1762755860 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f17a4042070 con 0x7f17b41041b0 2024-04-25T14:38:14.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:14.221+0000 7f17aaa00640 1 --2- 172.21.15.53:0/1762755860 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f17980a3bb0 0x7f17980a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:14.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:14.221+0000 7f17aaa00640 1 -- 172.21.15.53:0/1762755860 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f17a40f10b0 con 0x7f17b41041b0 2024-04-25T14:38:14.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:14.221+0000 7f17abe00640 1 --2- 172.21.15.53:0/1762755860 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f17980a3bb0 0x7f17980a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:14.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:14.222+0000 7f17abe00640 1 --2- 172.21.15.53:0/1762755860 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f17980a3bb0 0x7f17980a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f17b419b900 tx=0x7f179c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:14.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:14.223+0000 7f17aaa00640 1 -- 172.21.15.53:0/1762755860 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f17a40bd4f0 con 0x7f17b41041b0 2024-04-25T14:38:14.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:14.474+0000 7f17b8e00640 1 -- 172.21.15.53:0/1762755860 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f17b402d5c0 con 0x7f17b41041b0 2024-04-25T14:38:14.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:14 smithi183 ceph-mon[32584]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T14:38:14.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:14 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/3284514966' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-25T14:38:14.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:14 smithi183 ceph-mon[32584]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T14:38:14.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:14 smithi183 ceph-mon[32584]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T14:38:14.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:14 smithi183 ceph-mon[32584]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-25T14:38:14.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:14 smithi183 ceph-mon[32584]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-25T14:38:14.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:14 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/1762755860' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T14:38:14.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:14 smithi183 ceph-mon[32584]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T14:38:14.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:14 smithi150 ceph-mon[32504]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T14:38:14.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:14 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/3284514966' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-25T14:38:14.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:14 smithi150 ceph-mon[32504]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T14:38:14.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:14 smithi150 ceph-mon[32504]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T14:38:14.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:14 smithi150 ceph-mon[32504]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-25T14:38:14.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:14 smithi150 ceph-mon[32504]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-25T14:38:14.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:14 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/1762755860' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T14:38:14.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:14 smithi150 ceph-mon[32504]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T14:38:15.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.004+0000 7f17aaa00640 1 -- 172.21.15.53:0/1762755860 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v19) ==== 121+0+0 (secure 0 0 0) 0x7f17a40c17f0 con 0x7f17b41041b0 2024-04-25T14:38:15.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.009+0000 7f17b8e00640 1 -- 172.21.15.53:0/1762755860 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f17980a3bb0 msgr2=0x7f17980a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:15.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.009+0000 7f17b8e00640 1 --2- 172.21.15.53:0/1762755860 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f17980a3bb0 0x7f17980a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f17b419b900 tx=0x7f179c005f90 comp rx=0 tx=0).stop 2024-04-25T14:38:15.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.009+0000 7f17b8e00640 1 -- 172.21.15.53:0/1762755860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17b41041b0 msgr2=0x7f17b419a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:15.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.009+0000 7f17b8e00640 1 --2- 172.21.15.53:0/1762755860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17b41041b0 0x7f17b419a3f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f17a4031500 tx=0x7f17a4003c10 comp rx=0 tx=0).stop 2024-04-25T14:38:15.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.010+0000 7f17b8e00640 1 -- 172.21.15.53:0/1762755860 shutdown_connections 2024-04-25T14:38:15.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.010+0000 7f17b8e00640 1 --2- 172.21.15.53:0/1762755860 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f17980a3bb0 0x7f17980a6070 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:15.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.010+0000 7f17b8e00640 1 --2- 172.21.15.53:0/1762755860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17b4105d60 0x7f17b41a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:15.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.010+0000 7f17b8e00640 1 --2- 172.21.15.53:0/1762755860 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17b41053a0 0x7f17b419a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:15.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.010+0000 7f17b8e00640 1 --2- 172.21.15.53:0/1762755860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17b41041b0 0x7f17b419a3f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:15.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.010+0000 7f17b8e00640 1 -- 172.21.15.53:0/1762755860 >> 172.21.15.53:0/1762755860 conn(0x7f17b40ff700 msgr2=0x7f17b4108720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:15.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.011+0000 7f17b8e00640 1 -- 172.21.15.53:0/1762755860 shutdown_connections 2024-04-25T14:38:15.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.011+0000 7f17b8e00640 1 -- 172.21.15.53:0/1762755860 wait complete. 2024-04-25T14:38:15.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:14 smithi053 ceph-mon[27773]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T14:38:15.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:14 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/3284514966' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-25T14:38:15.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:14 smithi053 ceph-mon[27773]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T14:38:15.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:14 smithi053 ceph-mon[27773]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T14:38:15.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:14 smithi053 ceph-mon[27773]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-25T14:38:15.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:14 smithi053 ceph-mon[27773]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-25T14:38:15.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:14 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1762755860' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T14:38:15.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:14 smithi053 ceph-mon[27773]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T14:38:15.038 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-25T14:38:15.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.237+0000 7f5b71600640 1 -- 172.21.15.53:0/3910848686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b6c105d60 msgr2=0x7f5b6c10c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:15.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.237+0000 7f5b71600640 1 --2- 172.21.15.53:0/3910848686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b6c105d60 0x7f5b6c10c5f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5b64009ca0 tx=0x7f5b6402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:38:15.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.238+0000 7f5b71600640 1 -- 172.21.15.53:0/3910848686 shutdown_connections 2024-04-25T14:38:15.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.238+0000 7f5b71600640 1 --2- 172.21.15.53:0/3910848686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b6c105d60 0x7f5b6c10c5f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:15.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.238+0000 7f5b71600640 1 --2- 172.21.15.53:0/3910848686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b6c1053a0 0x7f5b6c105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:15.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.238+0000 7f5b71600640 1 --2- 172.21.15.53:0/3910848686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b6c1041b0 0x7f5b6c1045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:15.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.238+0000 7f5b71600640 1 -- 172.21.15.53:0/3910848686 >> 172.21.15.53:0/3910848686 conn(0x7f5b6c0ff740 msgr2=0x7f5b6c101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:15.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.238+0000 7f5b71600640 1 -- 172.21.15.53:0/3910848686 shutdown_connections 2024-04-25T14:38:15.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.238+0000 7f5b71600640 1 -- 172.21.15.53:0/3910848686 wait complete. 2024-04-25T14:38:15.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.239+0000 7f5b71600640 1 Processor -- start 2024-04-25T14:38:15.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.239+0000 7f5b71600640 1 -- start start 2024-04-25T14:38:15.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.239+0000 7f5b71600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b6c1041b0 0x7f5b6c19a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:15.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.239+0000 7f5b71600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b6c1053a0 0x7f5b6c19a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:15.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.239+0000 7f5b71600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b6c105d60 0x7f5b6c1a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:15.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.239+0000 7f5b71600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c10fd10 con 0x7f5b6c1041b0 2024-04-25T14:38:15.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.239+0000 7f5b71600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c10fb90 con 0x7f5b6c1053a0 2024-04-25T14:38:15.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.239+0000 7f5b71600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c10fe90 con 0x7f5b6c105d60 2024-04-25T14:38:15.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.240+0000 7f5b63e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b6c1053a0 0x7f5b6c19a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:15.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.240+0000 7f5b63e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b6c1053a0 0x7f5b6c19a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.53:38244/0 (socket says 172.21.15.53:38244) 2024-04-25T14:38:15.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.240+0000 7f5b63e00640 1 -- 172.21.15.53:0/2377205347 learned_addr learned my addr 172.21.15.53:0/2377205347 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:38:15.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.240+0000 7f5b6aa00640 1 --2- 172.21.15.53:0/2377205347 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b6c1041b0 0x7f5b6c19a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:15.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.240+0000 7f5b6b400640 1 --2- 172.21.15.53:0/2377205347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b6c105d60 0x7f5b6c1a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:15.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.240+0000 7f5b63e00640 1 -- 172.21.15.53:0/2377205347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b6c105d60 msgr2=0x7f5b6c1a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:15.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.240+0000 7f5b63e00640 1 --2- 172.21.15.53:0/2377205347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b6c105d60 0x7f5b6c1a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:15.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.240+0000 7f5b63e00640 1 -- 172.21.15.53:0/2377205347 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b6c1041b0 msgr2=0x7f5b6c19a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:15.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.240+0000 7f5b63e00640 1 --2- 172.21.15.53:0/2377205347 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b6c1041b0 0x7f5b6c19a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:15.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.240+0000 7f5b63e00640 1 -- 172.21.15.53:0/2377205347 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b6c1a1fb0 con 0x7f5b6c1053a0 2024-04-25T14:38:15.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.240+0000 7f5b6aa00640 1 --2- 172.21.15.53:0/2377205347 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b6c1041b0 0x7f5b6c19a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:38:15.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.240+0000 7f5b63e00640 1 --2- 172.21.15.53:0/2377205347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b6c1053a0 0x7f5b6c19a8d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5b5400e9f0 tx=0x7f5b5400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:15.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.240+0000 7f5b63400640 1 -- 172.21.15.53:0/2377205347 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5400cd80 con 0x7f5b6c1053a0 2024-04-25T14:38:15.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.241+0000 7f5b71600640 1 -- 172.21.15.53:0/2377205347 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b6c1a22a0 con 0x7f5b6c1053a0 2024-04-25T14:38:15.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.241+0000 7f5b63400640 1 -- 172.21.15.53:0/2377205347 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b540108e0 con 0x7f5b6c1053a0 2024-04-25T14:38:15.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.241+0000 7f5b63400640 1 -- 172.21.15.53:0/2377205347 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b54018980 con 0x7f5b6c1053a0 2024-04-25T14:38:15.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.241+0000 7f5b71600640 1 -- 172.21.15.53:0/2377205347 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b6c1a27f0 con 0x7f5b6c1053a0 2024-04-25T14:38:15.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.242+0000 7f5b71600640 1 -- 172.21.15.53:0/2377205347 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b6c10b4f0 con 0x7f5b6c1053a0 2024-04-25T14:38:15.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.244+0000 7f5b63400640 1 -- 172.21.15.53:0/2377205347 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5b54010430 con 0x7f5b6c1053a0 2024-04-25T14:38:15.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.245+0000 7f5b63400640 1 --2- 172.21.15.53:0/2377205347 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5b500a3bb0 0x7f5b500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:15.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.245+0000 7f5b63400640 1 -- 172.21.15.53:0/2377205347 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f5b540d0780 con 0x7f5b6c1053a0 2024-04-25T14:38:15.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.245+0000 7f5b6aa00640 1 --2- 172.21.15.53:0/2377205347 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5b500a3bb0 0x7f5b500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:15.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.246+0000 7f5b6aa00640 1 --2- 172.21.15.53:0/2377205347 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5b500a3bb0 0x7f5b500a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c105200 tx=0x7f5b5c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:15.250 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.249+0000 7f5b63400640 1 -- 172.21.15.53:0/2377205347 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5b5409d210 con 0x7f5b6c1053a0 2024-04-25T14:38:15.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.496+0000 7f5b71600640 1 -- 172.21.15.53:0/2377205347 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f5b6c02d5c0 con 0x7f5b6c1053a0 2024-04-25T14:38:15.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.636+0000 7f5b63400640 1 -- 172.21.15.53:0/2377205347 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v21) ==== 115+0+0 (secure 0 0 0) 0x7f5b5409cb10 con 0x7f5b6c1053a0 2024-04-25T14:38:15.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.640+0000 7f5b71600640 1 -- 172.21.15.53:0/2377205347 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5b500a3bb0 msgr2=0x7f5b500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:15.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.640+0000 7f5b71600640 1 --2- 172.21.15.53:0/2377205347 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5b500a3bb0 0x7f5b500a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c105200 tx=0x7f5b5c005f90 comp rx=0 tx=0).stop 2024-04-25T14:38:15.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.640+0000 7f5b71600640 1 -- 172.21.15.53:0/2377205347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b6c1053a0 msgr2=0x7f5b6c19a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:15.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.640+0000 7f5b71600640 1 --2- 172.21.15.53:0/2377205347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b6c1053a0 0x7f5b6c19a8d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5b5400e9f0 tx=0x7f5b5400eeb0 comp rx=0 tx=0).stop 2024-04-25T14:38:15.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.641+0000 7f5b71600640 1 -- 172.21.15.53:0/2377205347 shutdown_connections 2024-04-25T14:38:15.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.641+0000 7f5b71600640 1 --2- 172.21.15.53:0/2377205347 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5b500a3bb0 0x7f5b500a6070 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:15.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.641+0000 7f5b71600640 1 --2- 172.21.15.53:0/2377205347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b6c105d60 0x7f5b6c1a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:15.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.641+0000 7f5b71600640 1 --2- 172.21.15.53:0/2377205347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b6c1053a0 0x7f5b6c19a8d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:15.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.641+0000 7f5b71600640 1 --2- 172.21.15.53:0/2377205347 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b6c1041b0 0x7f5b6c19a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:15.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.641+0000 7f5b71600640 1 -- 172.21.15.53:0/2377205347 >> 172.21.15.53:0/2377205347 conn(0x7f5b6c0ff740 msgr2=0x7f5b6c1086c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:15.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.641+0000 7f5b71600640 1 -- 172.21.15.53:0/2377205347 shutdown_connections 2024-04-25T14:38:15.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.642+0000 7f5b71600640 1 -- 172.21.15.53:0/2377205347 wait complete. 2024-04-25T14:38:15.664 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-04-25T14:38:15.664 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-25T14:38:15.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.874+0000 7f243f000640 1 -- 172.21.15.53:0/2750224939 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2438108490 msgr2=0x7f24381088b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:15.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.874+0000 7f243f000640 1 --2- 172.21.15.53:0/2750224939 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2438108490 0x7f24381088b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f242c009900 tx=0x7f242c02a3f0 comp rx=0 tx=0).stop 2024-04-25T14:38:15.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.874+0000 7f243f000640 1 -- 172.21.15.53:0/2750224939 shutdown_connections 2024-04-25T14:38:15.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.874+0000 7f243f000640 1 --2- 172.21.15.53:0/2750224939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f243810a040 0x7f24381108d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:15.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.874+0000 7f243f000640 1 --2- 172.21.15.53:0/2750224939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2438109680 0x7f2438109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:15.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.874+0000 7f243f000640 1 --2- 172.21.15.53:0/2750224939 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2438108490 0x7f24381088b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:15.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.874+0000 7f243f000640 1 -- 172.21.15.53:0/2750224939 >> 172.21.15.53:0/2750224939 conn(0x7f2438077420 msgr2=0x7f2438079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:15.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.875+0000 7f243f000640 1 -- 172.21.15.53:0/2750224939 shutdown_connections 2024-04-25T14:38:15.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.875+0000 7f243f000640 1 -- 172.21.15.53:0/2750224939 wait complete. 2024-04-25T14:38:15.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.876+0000 7f243f000640 1 Processor -- start 2024-04-25T14:38:15.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.876+0000 7f243f000640 1 -- start start 2024-04-25T14:38:15.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.876+0000 7f243f000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2438108490 0x7f243819e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:15.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.876+0000 7f243f000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2438109680 0x7f243819ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:15.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.877+0000 7f243f000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f243810a040 0x7f24381a5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:15.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.877+0000 7f243f000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2438113ff0 con 0x7f243810a040 2024-04-25T14:38:15.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.877+0000 7f243f000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2438113e70 con 0x7f2438108490 2024-04-25T14:38:15.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.877+0000 7f243f000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2438114170 con 0x7f2438109680 2024-04-25T14:38:15.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.877+0000 7f243dc00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2438108490 0x7f243819e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:15.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.877+0000 7f243dc00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2438108490 0x7f243819e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.53:38274/0 (socket says 172.21.15.53:38274) 2024-04-25T14:38:15.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.877+0000 7f243dc00640 1 -- 172.21.15.53:0/2034841998 learned_addr learned my addr 172.21.15.53:0/2034841998 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:38:15.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.877+0000 7f243d200640 1 --2- 172.21.15.53:0/2034841998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2438109680 0x7f243819ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:15.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.877+0000 7f243e600640 1 --2- 172.21.15.53:0/2034841998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f243810a040 0x7f24381a5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:15.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.877+0000 7f243dc00640 1 -- 172.21.15.53:0/2034841998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2438109680 msgr2=0x7f243819ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:15.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.877+0000 7f243dc00640 1 --2- 172.21.15.53:0/2034841998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2438109680 0x7f243819ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:15.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.877+0000 7f243dc00640 1 -- 172.21.15.53:0/2034841998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f243810a040 msgr2=0x7f24381a5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:15.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.877+0000 7f243dc00640 1 --2- 172.21.15.53:0/2034841998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f243810a040 0x7f24381a5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:15.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.877+0000 7f243dc00640 1 -- 172.21.15.53:0/2034841998 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24381a62d0 con 0x7f2438108490 2024-04-25T14:38:15.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.877+0000 7f243e600640 1 --2- 172.21.15.53:0/2034841998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f243810a040 0x7f24381a5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:38:15.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.877+0000 7f243dc00640 1 --2- 172.21.15.53:0/2034841998 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2438108490 0x7f243819e6b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f242c031500 tx=0x7f242c003c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:15.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.878+0000 7f2436000640 1 -- 172.21.15.53:0/2034841998 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242c03f070 con 0x7f2438108490 2024-04-25T14:38:15.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.878+0000 7f2436000640 1 -- 172.21.15.53:0/2034841998 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f242c003e70 con 0x7f2438108490 2024-04-25T14:38:15.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.878+0000 7f243f000640 1 -- 172.21.15.53:0/2034841998 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24381a6560 con 0x7f2438108490 2024-04-25T14:38:15.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.878+0000 7f2436000640 1 -- 172.21.15.53:0/2034841998 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242c035030 con 0x7f2438108490 2024-04-25T14:38:15.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.878+0000 7f243f000640 1 -- 172.21.15.53:0/2034841998 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24381a6aa0 con 0x7f2438108490 2024-04-25T14:38:15.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.879+0000 7f243f000640 1 -- 172.21.15.53:0/2034841998 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f243810f7b0 con 0x7f2438108490 2024-04-25T14:38:15.882 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.881+0000 7f2436000640 1 -- 172.21.15.53:0/2034841998 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f242c042070 con 0x7f2438108490 2024-04-25T14:38:15.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.882+0000 7f2436000640 1 --2- 172.21.15.53:0/2034841998 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f24200a3c30 0x7f24200a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:15.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.882+0000 7f2436000640 1 -- 172.21.15.53:0/2034841998 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f242c0f0f00 con 0x7f2438108490 2024-04-25T14:38:15.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.883+0000 7f243d200640 1 --2- 172.21.15.53:0/2034841998 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f24200a3c30 0x7f24200a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:15.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.883+0000 7f243d200640 1 --2- 172.21.15.53:0/2034841998 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f24200a3c30 0x7f24200a60f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f243819fbc0 tx=0x7f242800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:15.887 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:15.886+0000 7f2436000640 1 -- 172.21.15.53:0/2034841998 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f242c0bd990 con 0x7f2438108490 2024-04-25T14:38:15.960 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 25 14:38:15 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-b[62911]: 2024-04-25T14:38:15.501+0000 7fb568e9d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T14:38:15.960 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 25 14:38:15 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-b[62911]: 2024-04-25T14:38:15.501+0000 7fb568e9d700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-25T14:38:16.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.119+0000 7f243f000640 1 -- 172.21.15.53:0/2034841998 --> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f2438062390 con 0x7f24200a3c30 2024-04-25T14:38:16.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.152+0000 7f2436000640 1 -- 172.21.15.53:0/2034841998 <== mgr.14152 v2:172.21.15.53:6800/4154933108 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2438062390 con 0x7f24200a3c30 2024-04-25T14:38:16.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.154+0000 7f243f000640 1 -- 172.21.15.53:0/2034841998 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f24200a3c30 msgr2=0x7f24200a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:16.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.154+0000 7f243f000640 1 --2- 172.21.15.53:0/2034841998 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f24200a3c30 0x7f24200a60f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f243819fbc0 tx=0x7f242800b410 comp rx=0 tx=0).stop 2024-04-25T14:38:16.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.154+0000 7f243f000640 1 -- 172.21.15.53:0/2034841998 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2438108490 msgr2=0x7f243819e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:16.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.154+0000 7f243f000640 1 --2- 172.21.15.53:0/2034841998 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2438108490 0x7f243819e6b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f242c031500 tx=0x7f242c003c80 comp rx=0 tx=0).stop 2024-04-25T14:38:16.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.154+0000 7f243f000640 1 -- 172.21.15.53:0/2034841998 shutdown_connections 2024-04-25T14:38:16.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.154+0000 7f243f000640 1 --2- 172.21.15.53:0/2034841998 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f24200a3c30 0x7f24200a60f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:16.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.154+0000 7f243f000640 1 --2- 172.21.15.53:0/2034841998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f243810a040 0x7f24381a5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:16.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.155+0000 7f243f000640 1 --2- 172.21.15.53:0/2034841998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2438109680 0x7f243819ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:16.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.155+0000 7f243f000640 1 --2- 172.21.15.53:0/2034841998 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2438108490 0x7f243819e6b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:16.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.155+0000 7f243f000640 1 -- 172.21.15.53:0/2034841998 >> 172.21.15.53:0/2034841998 conn(0x7f2438077420 msgr2=0x7f243810c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:16.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.155+0000 7f243f000640 1 -- 172.21.15.53:0/2034841998 shutdown_connections 2024-04-25T14:38:16.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.155+0000 7f243f000640 1 -- 172.21.15.53:0/2034841998 wait complete. 2024-04-25T14:38:16.174 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-25T14:38:16.283 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:38:16 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:38:16.127+0000 7fd7c9c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:38:16.283 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:38:16 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:38:16.127+0000 7fd7c9c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:38:16.283 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:38:16 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:38:16.127+0000 7fd7c9c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:38:16.283 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:38:16 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:38:16.127+0000 7fd7c9c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:38:16.283 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:38:16 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:38:16.127+0000 7fd7c9c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:38:16.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:16 smithi053 ceph-mon[27773]: pgmap v318: 97 pgs: 15 creating+peering, 80 active+clean, 2 unknown; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 6 op/s 2024-04-25T14:38:16.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:16 smithi053 ceph-mon[27773]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T14:38:16.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:16 smithi053 ceph-mon[27773]: mds.? [v2:172.21.15.183:6838/1554258915,v1:172.21.15.183:6839/1554258915] up:active 2024-04-25T14:38:16.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:16 smithi053 ceph-mon[27773]: daemon mds.g assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T14:38:16.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:16 smithi053 ceph-mon[27773]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T14:38:16.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:16 smithi053 ceph-mon[27773]: Cluster is now healthy 2024-04-25T14:38:16.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:16 smithi053 ceph-mon[27773]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-25T14:38:16.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:16 smithi053 ceph-mon[27773]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=g=up:creating} 9 up:standby 2024-04-25T14:38:16.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:16 smithi053 ceph-mon[27773]: daemon mds.g is now active in filesystem cephfs as rank 2 2024-04-25T14:38:16.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:16 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/2377205347' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T14:38:16.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:16 smithi053 ceph-mon[27773]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T14:38:16.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:16 smithi053 ceph-mon[27773]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T14:38:16.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:16 smithi053 ceph-mon[27773]: mds.? [v2:172.21.15.53:6838/570000540,v1:172.21.15.53:6839/570000540] up:active 2024-04-25T14:38:16.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:16 smithi053 ceph-mon[27773]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T14:38:16.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.369+0000 7f9868a00640 1 -- 172.21.15.53:0/3700330667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9864076910 msgr2=0x7f9864076d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:16.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.369+0000 7f9868a00640 1 --2- 172.21.15.53:0/3700330667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9864076910 0x7f9864076d30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f985c009900 tx=0x7f985c02a3f0 comp rx=0 tx=0).stop 2024-04-25T14:38:16.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.369+0000 7f9868a00640 1 -- 172.21.15.53:0/3700330667 shutdown_connections 2024-04-25T14:38:16.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.369+0000 7f9868a00640 1 --2- 172.21.15.53:0/3700330667 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98640784f0 0x7f9864078d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:16.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.369+0000 7f9868a00640 1 --2- 172.21.15.53:0/3700330667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9864077b50 0x7f9864077fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:16.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.369+0000 7f9868a00640 1 --2- 172.21.15.53:0/3700330667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9864076910 0x7f9864076d30 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:16.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.369+0000 7f9868a00640 1 -- 172.21.15.53:0/3700330667 >> 172.21.15.53:0/3700330667 conn(0x7f98640ff600 msgr2=0x7f9864101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:16.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.369+0000 7f9868a00640 1 -- 172.21.15.53:0/3700330667 shutdown_connections 2024-04-25T14:38:16.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.369+0000 7f9868a00640 1 -- 172.21.15.53:0/3700330667 wait complete. 2024-04-25T14:38:16.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.370+0000 7f9868a00640 1 Processor -- start 2024-04-25T14:38:16.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.370+0000 7f9868a00640 1 -- start start 2024-04-25T14:38:16.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.371+0000 7f9868a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9864076910 0x7f986419e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:16.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.371+0000 7f9868a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9864077b50 0x7f986419ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:16.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.371+0000 7f9868a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98640784f0 0x7f98641a5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:16.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.371+0000 7f9868a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9864114010 con 0x7f98640784f0 2024-04-25T14:38:16.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.371+0000 7f9868a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9864113e90 con 0x7f9864077b50 2024-04-25T14:38:16.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.371+0000 7f9868a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9864114190 con 0x7f9864076910 2024-04-25T14:38:16.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.371+0000 7f9863200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98640784f0 0x7f98641a5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:16.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.371+0000 7f9862800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9864076910 0x7f986419e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:16.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.371+0000 7f9863200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98640784f0 0x7f98641a5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36138/0 (socket says 172.21.15.53:36138) 2024-04-25T14:38:16.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.371+0000 7f9861e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9864077b50 0x7f986419ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:16.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.371+0000 7f9863200640 1 -- 172.21.15.53:0/3816104327 learned_addr learned my addr 172.21.15.53:0/3816104327 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:38:16.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.372+0000 7f9863200640 1 -- 172.21.15.53:0/3816104327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9864076910 msgr2=0x7f986419e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:16.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.372+0000 7f9863200640 1 --2- 172.21.15.53:0/3816104327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9864076910 0x7f986419e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:16.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.372+0000 7f9863200640 1 -- 172.21.15.53:0/3816104327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9864077b50 msgr2=0x7f986419ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:16.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.372+0000 7f9863200640 1 --2- 172.21.15.53:0/3816104327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9864077b50 0x7f986419ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:16.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.372+0000 7f9863200640 1 -- 172.21.15.53:0/3816104327 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98641a62c0 con 0x7f98640784f0 2024-04-25T14:38:16.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.372+0000 7f9861e00640 1 --2- 172.21.15.53:0/3816104327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9864077b50 0x7f986419ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:38:16.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.372+0000 7f9863200640 1 --2- 172.21.15.53:0/3816104327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98640784f0 0x7f98641a5c50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f985400e970 tx=0x7f985400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:16.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.372+0000 7f985b400640 1 -- 172.21.15.53:0/3816104327 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985400ccc0 con 0x7f98640784f0 2024-04-25T14:38:16.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.372+0000 7f985b400640 1 -- 172.21.15.53:0/3816104327 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f985400ce60 con 0x7f98640784f0 2024-04-25T14:38:16.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.372+0000 7f985b400640 1 -- 172.21.15.53:0/3816104327 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9854010430 con 0x7f98640784f0 2024-04-25T14:38:16.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.372+0000 7f9868a00640 1 -- 172.21.15.53:0/3816104327 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98641a65b0 con 0x7f98640784f0 2024-04-25T14:38:16.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.372+0000 7f9868a00640 1 -- 172.21.15.53:0/3816104327 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98641a6b50 con 0x7f98640784f0 2024-04-25T14:38:16.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.373+0000 7f9868a00640 1 -- 172.21.15.53:0/3816104327 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9864110be0 con 0x7f98640784f0 2024-04-25T14:38:16.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.376+0000 7f985b400640 1 -- 172.21.15.53:0/3816104327 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9854004f70 con 0x7f98640784f0 2024-04-25T14:38:16.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.377+0000 7f985b400640 1 --2- 172.21.15.53:0/3816104327 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f98480a3c30 0x7f98480a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:16.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.377+0000 7f985b400640 1 -- 172.21.15.53:0/3816104327 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f98540d0830 con 0x7f98640784f0 2024-04-25T14:38:16.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.377+0000 7f9862800640 1 --2- 172.21.15.53:0/3816104327 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f98480a3c30 0x7f98480a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:16.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.378+0000 7f9862800640 1 --2- 172.21.15.53:0/3816104327 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f98480a3c30 0x7f98480a60f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f985c031500 tx=0x7f985c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:16.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.380+0000 7f985b400640 1 -- 172.21.15.53:0/3816104327 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9854053ff0 con 0x7f98640784f0 2024-04-25T14:38:16.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:16 smithi183 ceph-mon[32584]: pgmap v318: 97 pgs: 15 creating+peering, 80 active+clean, 2 unknown; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 6 op/s 2024-04-25T14:38:16.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:16 smithi183 ceph-mon[32584]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T14:38:16.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:16 smithi183 ceph-mon[32584]: mds.? [v2:172.21.15.183:6838/1554258915,v1:172.21.15.183:6839/1554258915] up:active 2024-04-25T14:38:16.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:16 smithi183 ceph-mon[32584]: daemon mds.g assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T14:38:16.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:16 smithi183 ceph-mon[32584]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T14:38:16.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:16 smithi183 ceph-mon[32584]: Cluster is now healthy 2024-04-25T14:38:16.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:16 smithi183 ceph-mon[32584]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-25T14:38:16.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:16 smithi183 ceph-mon[32584]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=g=up:creating} 9 up:standby 2024-04-25T14:38:16.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:16 smithi183 ceph-mon[32584]: daemon mds.g is now active in filesystem cephfs as rank 2 2024-04-25T14:38:16.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:16 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/2377205347' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T14:38:16.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:16 smithi183 ceph-mon[32584]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T14:38:16.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:16 smithi183 ceph-mon[32584]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T14:38:16.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:16 smithi183 ceph-mon[32584]: mds.? [v2:172.21.15.53:6838/570000540,v1:172.21.15.53:6839/570000540] up:active 2024-04-25T14:38:16.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:16 smithi183 ceph-mon[32584]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T14:38:16.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:16 smithi150 ceph-mon[32504]: pgmap v318: 97 pgs: 15 creating+peering, 80 active+clean, 2 unknown; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 6 op/s 2024-04-25T14:38:16.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:16 smithi150 ceph-mon[32504]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T14:38:16.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:16 smithi150 ceph-mon[32504]: mds.? [v2:172.21.15.183:6838/1554258915,v1:172.21.15.183:6839/1554258915] up:active 2024-04-25T14:38:16.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:16 smithi150 ceph-mon[32504]: daemon mds.g assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T14:38:16.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:16 smithi150 ceph-mon[32504]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T14:38:16.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:16 smithi150 ceph-mon[32504]: Cluster is now healthy 2024-04-25T14:38:16.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:16 smithi150 ceph-mon[32504]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-25T14:38:16.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:16 smithi150 ceph-mon[32504]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=g=up:creating} 9 up:standby 2024-04-25T14:38:16.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:16 smithi150 ceph-mon[32504]: daemon mds.g is now active in filesystem cephfs as rank 2 2024-04-25T14:38:16.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:16 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/2377205347' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T14:38:16.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:16 smithi150 ceph-mon[32504]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T14:38:16.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:16 smithi150 ceph-mon[32504]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T14:38:16.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:16 smithi150 ceph-mon[32504]: mds.? [v2:172.21.15.53:6838/570000540,v1:172.21.15.53:6839/570000540] up:active 2024-04-25T14:38:16.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:16 smithi150 ceph-mon[32504]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T14:38:16.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.615+0000 7f9868a00640 1 -- 172.21.15.53:0/3816104327 --> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f9864062390 con 0x7f98480a3c30 2024-04-25T14:38:16.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.652+0000 7f985b400640 1 -- 172.21.15.53:0/3816104327 <== mgr.14152 v2:172.21.15.53:6800/4154933108 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9864062390 con 0x7f98480a3c30 2024-04-25T14:38:16.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.654+0000 7f9868a00640 1 -- 172.21.15.53:0/3816104327 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f98480a3c30 msgr2=0x7f98480a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:16.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.654+0000 7f9868a00640 1 --2- 172.21.15.53:0/3816104327 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f98480a3c30 0x7f98480a60f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f985c031500 tx=0x7f985c005a20 comp rx=0 tx=0).stop 2024-04-25T14:38:16.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.654+0000 7f9868a00640 1 -- 172.21.15.53:0/3816104327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98640784f0 msgr2=0x7f98641a5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:16.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.654+0000 7f9868a00640 1 --2- 172.21.15.53:0/3816104327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98640784f0 0x7f98641a5c50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f985400e970 tx=0x7f985400ed70 comp rx=0 tx=0).stop 2024-04-25T14:38:16.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.655+0000 7f9868a00640 1 -- 172.21.15.53:0/3816104327 shutdown_connections 2024-04-25T14:38:16.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.655+0000 7f9868a00640 1 --2- 172.21.15.53:0/3816104327 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f98480a3c30 0x7f98480a60f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:16.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.655+0000 7f9868a00640 1 --2- 172.21.15.53:0/3816104327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98640784f0 0x7f98641a5c50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:16.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.655+0000 7f9868a00640 1 --2- 172.21.15.53:0/3816104327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9864077b50 0x7f986419ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:16.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.655+0000 7f9868a00640 1 --2- 172.21.15.53:0/3816104327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9864076910 0x7f986419e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:16.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.655+0000 7f9868a00640 1 -- 172.21.15.53:0/3816104327 >> 172.21.15.53:0/3816104327 conn(0x7f98640ff600 msgr2=0x7f9864100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:16.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.655+0000 7f9868a00640 1 -- 172.21.15.53:0/3816104327 shutdown_connections 2024-04-25T14:38:16.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.655+0000 7f9868a00640 1 -- 172.21.15.53:0/3816104327 wait complete. 2024-04-25T14:38:16.678 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-25T14:38:16.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.900+0000 7f65bde00640 1 -- 172.21.15.53:0/1674445947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65b8075f90 msgr2=0x7f65b80743e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:16.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.900+0000 7f65bde00640 1 --2- 172.21.15.53:0/1674445947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65b8075f90 0x7f65b80743e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f65a8009900 tx=0x7f65a802a3f0 comp rx=0 tx=0).stop 2024-04-25T14:38:16.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.901+0000 7f65bde00640 1 -- 172.21.15.53:0/1674445947 shutdown_connections 2024-04-25T14:38:16.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.901+0000 7f65bde00640 1 --2- 172.21.15.53:0/1674445947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65b8109d20 0x7f65b810c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:16.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.901+0000 7f65bde00640 1 --2- 172.21.15.53:0/1674445947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65b80749b0 0x7f65b8074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:16.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.901+0000 7f65bde00640 1 --2- 172.21.15.53:0/1674445947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65b8075f90 0x7f65b80743e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:16.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.901+0000 7f65bde00640 1 -- 172.21.15.53:0/1674445947 >> 172.21.15.53:0/1674445947 conn(0x7f65b80fd3f0 msgr2=0x7f65b80ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:16.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.901+0000 7f65bde00640 1 -- 172.21.15.53:0/1674445947 shutdown_connections 2024-04-25T14:38:16.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.901+0000 7f65bde00640 1 -- 172.21.15.53:0/1674445947 wait complete. 2024-04-25T14:38:16.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.902+0000 7f65bde00640 1 Processor -- start 2024-04-25T14:38:16.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.902+0000 7f65bde00640 1 -- start start 2024-04-25T14:38:16.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.903+0000 7f65bde00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65b80749b0 0x7f65b819a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:16.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.903+0000 7f65bde00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65b8075f90 0x7f65b819a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:16.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.903+0000 7f65bde00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65b8109d20 0x7f65b81a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:16.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.903+0000 7f65bde00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f65b810fb90 con 0x7f65b8075f90 2024-04-25T14:38:16.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.903+0000 7f65bde00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f65b810fa10 con 0x7f65b8109d20 2024-04-25T14:38:16.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.903+0000 7f65bde00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f65b810fd10 con 0x7f65b80749b0 2024-04-25T14:38:16.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.903+0000 7f65afe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65b8075f90 0x7f65b819a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:16.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.903+0000 7f65afe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65b8075f90 0x7f65b819a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36166/0 (socket says 172.21.15.53:36166) 2024-04-25T14:38:16.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.903+0000 7f65bd400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65b8109d20 0x7f65b81a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:16.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.903+0000 7f65afe00640 1 -- 172.21.15.53:0/622538664 learned_addr learned my addr 172.21.15.53:0/622538664 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:38:16.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.903+0000 7f65bca00640 1 --2- 172.21.15.53:0/622538664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65b80749b0 0x7f65b819a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:16.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.904+0000 7f65bd400640 1 -- 172.21.15.53:0/622538664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65b80749b0 msgr2=0x7f65b819a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:16.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.904+0000 7f65bd400640 1 --2- 172.21.15.53:0/622538664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65b80749b0 0x7f65b819a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:16.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.904+0000 7f65bd400640 1 -- 172.21.15.53:0/622538664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65b8075f90 msgr2=0x7f65b819a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:16.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.904+0000 7f65bd400640 1 --2- 172.21.15.53:0/622538664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65b8075f90 0x7f65b819a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:16.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.904+0000 7f65bd400640 1 -- 172.21.15.53:0/622538664 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b81a1ea0 con 0x7f65b8109d20 2024-04-25T14:38:16.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.904+0000 7f65bca00640 1 --2- 172.21.15.53:0/622538664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65b80749b0 0x7f65b819a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:38:16.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.904+0000 7f65afe00640 1 --2- 172.21.15.53:0/622538664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65b8075f90 0x7f65b819a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:38:16.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.904+0000 7f65bd400640 1 --2- 172.21.15.53:0/622538664 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65b8109d20 0x7f65b81a1830 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f65b0009fd0 tx=0x7f65b000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:16.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.904+0000 7f65b6000640 1 -- 172.21.15.53:0/622538664 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b000ce20 con 0x7f65b8109d20 2024-04-25T14:38:16.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.904+0000 7f65b6000640 1 -- 172.21.15.53:0/622538664 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65b0005020 con 0x7f65b8109d20 2024-04-25T14:38:16.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.904+0000 7f65b6000640 1 -- 172.21.15.53:0/622538664 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b0010690 con 0x7f65b8109d20 2024-04-25T14:38:16.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.904+0000 7f65bde00640 1 -- 172.21.15.53:0/622538664 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65b81a2190 con 0x7f65b8109d20 2024-04-25T14:38:16.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.904+0000 7f65bde00640 1 -- 172.21.15.53:0/622538664 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65b81a2690 con 0x7f65b8109d20 2024-04-25T14:38:16.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.905+0000 7f65bde00640 1 -- 172.21.15.53:0/622538664 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65b8108a50 con 0x7f65b8109d20 2024-04-25T14:38:16.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.907+0000 7f65b6000640 1 -- 172.21.15.53:0/622538664 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f65b00054d0 con 0x7f65b8109d20 2024-04-25T14:38:16.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.909+0000 7f65b6000640 1 --2- 172.21.15.53:0/622538664 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f659c0a3c30 0x7f659c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:16.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.909+0000 7f65b6000640 1 -- 172.21.15.53:0/622538664 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f65b00d0a40 con 0x7f65b8109d20 2024-04-25T14:38:16.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.909+0000 7f65bca00640 1 --2- 172.21.15.53:0/622538664 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f659c0a3c30 0x7f659c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:16.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.910+0000 7f65bca00640 1 --2- 172.21.15.53:0/622538664 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f659c0a3c30 0x7f659c0a60f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f65a8000c00 tx=0x7f65a80314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:16.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:16.913+0000 7f65b6000640 1 -- 172.21.15.53:0/622538664 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f65b009d4d0 con 0x7f65b8109d20 2024-04-25T14:38:17.032 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.030+0000 7f65b6000640 1 -- 172.21.15.53:0/622538664 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f65b0099440 con 0x7f65b8109d20 2024-04-25T14:38:17.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.160+0000 7f65bde00640 1 -- 172.21.15.53:0/622538664 --> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f65b8062390 con 0x7f659c0a3c30 2024-04-25T14:38:17.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.169+0000 7f65b6000640 1 -- 172.21.15.53:0/622538664 <== mgr.14152 v2:172.21.15.53:6800/4154933108 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f65b8062390 con 0x7f659c0a3c30 2024-04-25T14:38:17.170 INFO:teuthology.orchestra.run.smithi053.stdout:/volumes/qa/sv_0/9b7b93e1-76f5-4c41-ab1f-a88fb67a00e4 2024-04-25T14:38:17.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.171+0000 7f65bde00640 1 -- 172.21.15.53:0/622538664 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f659c0a3c30 msgr2=0x7f659c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:17.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.171+0000 7f65bde00640 1 --2- 172.21.15.53:0/622538664 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f659c0a3c30 0x7f659c0a60f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f65a8000c00 tx=0x7f65a80314a0 comp rx=0 tx=0).stop 2024-04-25T14:38:17.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.172+0000 7f65bde00640 1 -- 172.21.15.53:0/622538664 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65b8109d20 msgr2=0x7f65b81a1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:17.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.172+0000 7f65bde00640 1 --2- 172.21.15.53:0/622538664 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65b8109d20 0x7f65b81a1830 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f65b0009fd0 tx=0x7f65b000ef10 comp rx=0 tx=0).stop 2024-04-25T14:38:17.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.172+0000 7f65bde00640 1 -- 172.21.15.53:0/622538664 shutdown_connections 2024-04-25T14:38:17.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.172+0000 7f65bde00640 1 --2- 172.21.15.53:0/622538664 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f659c0a3c30 0x7f659c0a60f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:17.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.172+0000 7f65bde00640 1 --2- 172.21.15.53:0/622538664 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65b8109d20 0x7f65b81a1830 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:17.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.172+0000 7f65bde00640 1 --2- 172.21.15.53:0/622538664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65b8075f90 0x7f65b819a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:17.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.172+0000 7f65bde00640 1 --2- 172.21.15.53:0/622538664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65b80749b0 0x7f65b819a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:17.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.172+0000 7f65bde00640 1 -- 172.21.15.53:0/622538664 >> 172.21.15.53:0/622538664 conn(0x7f65b80fd3f0 msgr2=0x7f65b8105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:17.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.172+0000 7f65bde00640 1 -- 172.21.15.53:0/622538664 shutdown_connections 2024-04-25T14:38:17.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.172+0000 7f65bde00640 1 -- 172.21.15.53:0/622538664 wait complete. 2024-04-25T14:38:17.192 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-25T14:38:17.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:17 smithi053 ceph-mon[27773]: from='client.24668 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:38:17.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.395+0000 7faeb1000640 1 -- 172.21.15.53:0/2713543919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faeac076910 msgr2=0x7faeac076d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:17.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.395+0000 7faeb1000640 1 --2- 172.21.15.53:0/2713543919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faeac076910 0x7faeac076d30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fae9c009ca0 tx=0x7fae9c02a8b0 comp rx=0 tx=0).stop 2024-04-25T14:38:17.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.395+0000 7faeb1000640 1 -- 172.21.15.53:0/2713543919 shutdown_connections 2024-04-25T14:38:17.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.395+0000 7faeb1000640 1 --2- 172.21.15.53:0/2713543919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faeac0784f0 0x7faeac078d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:17.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.395+0000 7faeb1000640 1 --2- 172.21.15.53:0/2713543919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faeac077b50 0x7faeac077fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:17.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.395+0000 7faeb1000640 1 --2- 172.21.15.53:0/2713543919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faeac076910 0x7faeac076d30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:17.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.395+0000 7faeb1000640 1 -- 172.21.15.53:0/2713543919 >> 172.21.15.53:0/2713543919 conn(0x7faeac0ff740 msgr2=0x7faeac101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:17.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.395+0000 7faeb1000640 1 -- 172.21.15.53:0/2713543919 shutdown_connections 2024-04-25T14:38:17.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.395+0000 7faeb1000640 1 -- 172.21.15.53:0/2713543919 wait complete. 2024-04-25T14:38:17.399 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.397+0000 7faeb1000640 1 Processor -- start 2024-04-25T14:38:17.399 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.397+0000 7faeb1000640 1 -- start start 2024-04-25T14:38:17.399 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.397+0000 7faeb1000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faeac076910 0x7faeac19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:17.399 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.397+0000 7faeb1000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faeac077b50 0x7faeac19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:17.399 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.397+0000 7faeb1000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faeac0784f0 0x7faeac1a5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:17.399 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.397+0000 7faeb1000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7faeac114110 con 0x7faeac077b50 2024-04-25T14:38:17.399 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.397+0000 7faeb1000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7faeac113f90 con 0x7faeac076910 2024-04-25T14:38:17.399 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.397+0000 7faeb1000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faeac114290 con 0x7faeac0784f0 2024-04-25T14:38:17.399 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.398+0000 7faea9400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faeac077b50 0x7faeac19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:17.399 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.398+0000 7faea9400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faeac077b50 0x7faeac19ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36170/0 (socket says 172.21.15.53:36170) 2024-04-25T14:38:17.399 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.398+0000 7faea9400640 1 -- 172.21.15.53:0/3544024318 learned_addr learned my addr 172.21.15.53:0/3544024318 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:38:17.399 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.398+0000 7faea9e00640 1 --2- 172.21.15.53:0/3544024318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faeac076910 0x7faeac19e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:17.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.398+0000 7faeaa800640 1 --2- 172.21.15.53:0/3544024318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faeac0784f0 0x7faeac1a5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:17.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.398+0000 7faea9400640 1 -- 172.21.15.53:0/3544024318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faeac0784f0 msgr2=0x7faeac1a5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:17.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.398+0000 7faea9400640 1 --2- 172.21.15.53:0/3544024318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faeac0784f0 0x7faeac1a5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:17.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.398+0000 7faea9400640 1 -- 172.21.15.53:0/3544024318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faeac076910 msgr2=0x7faeac19e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:17.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.398+0000 7faea9400640 1 --2- 172.21.15.53:0/3544024318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faeac076910 0x7faeac19e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:17.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.398+0000 7faea9400640 1 -- 172.21.15.53:0/3544024318 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeac1a6420 con 0x7faeac077b50 2024-04-25T14:38:17.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.399+0000 7faea9400640 1 --2- 172.21.15.53:0/3544024318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faeac077b50 0x7faeac19ed40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fae9400ea60 tx=0x7fae9400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:17.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.399+0000 7faea2a00640 1 -- 172.21.15.53:0/3544024318 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9400cde0 con 0x7faeac077b50 2024-04-25T14:38:17.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.399+0000 7faea2a00640 1 -- 172.21.15.53:0/3544024318 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae940035f0 con 0x7faeac077b50 2024-04-25T14:38:17.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.399+0000 7faea2a00640 1 -- 172.21.15.53:0/3544024318 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae94010690 con 0x7faeac077b50 2024-04-25T14:38:17.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.399+0000 7faeb1000640 1 -- 172.21.15.53:0/3544024318 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faeac1a6710 con 0x7faeac077b50 2024-04-25T14:38:17.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.399+0000 7faeb1000640 1 -- 172.21.15.53:0/3544024318 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faeac1a6c10 con 0x7faeac077b50 2024-04-25T14:38:17.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.400+0000 7faeb1000640 1 -- 172.21.15.53:0/3544024318 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faeac110b40 con 0x7faeac077b50 2024-04-25T14:38:17.403 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.402+0000 7faea2a00640 1 -- 172.21.15.53:0/3544024318 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fae940037a0 con 0x7faeac077b50 2024-04-25T14:38:17.404 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.404+0000 7faea2a00640 1 --2- 172.21.15.53:0/3544024318 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fae900a3cc0 0x7fae900a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:17.404 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.404+0000 7faea2a00640 1 -- 172.21.15.53:0/3544024318 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fae940d06f0 con 0x7faeac077b50 2024-04-25T14:38:17.404 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.404+0000 7faea9e00640 1 --2- 172.21.15.53:0/3544024318 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fae900a3cc0 0x7fae900a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:17.405 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.405+0000 7faea9e00640 1 --2- 172.21.15.53:0/3544024318 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fae900a3cc0 0x7fae900a6180 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fae9c006010 tx=0x7fae9c005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:17.409 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.409+0000 7faea2a00640 1 -- 172.21.15.53:0/3544024318 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fae9409d020 con 0x7faeac077b50 2024-04-25T14:38:17.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:17 smithi183 ceph-mon[32584]: from='client.24668 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:38:17.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:17 smithi150 ceph-mon[32504]: from='client.24668 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:38:17.649 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.648+0000 7faeb1000640 1 -- 172.21.15.53:0/3544024318 --> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7faeac062390 con 0x7fae900a3cc0 2024-04-25T14:38:17.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.688+0000 7faea2a00640 1 -- 172.21.15.53:0/3544024318 <== mgr.14152 v2:172.21.15.53:6800/4154933108 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7faeac062390 con 0x7fae900a3cc0 2024-04-25T14:38:17.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.691+0000 7faeb1000640 1 -- 172.21.15.53:0/3544024318 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fae900a3cc0 msgr2=0x7fae900a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:17.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.691+0000 7faeb1000640 1 --2- 172.21.15.53:0/3544024318 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fae900a3cc0 0x7fae900a6180 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fae9c006010 tx=0x7fae9c005f50 comp rx=0 tx=0).stop 2024-04-25T14:38:17.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.691+0000 7faeb1000640 1 -- 172.21.15.53:0/3544024318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faeac077b50 msgr2=0x7faeac19ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:17.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.691+0000 7faeb1000640 1 --2- 172.21.15.53:0/3544024318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faeac077b50 0x7faeac19ed40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fae9400ea60 tx=0x7fae9400ef20 comp rx=0 tx=0).stop 2024-04-25T14:38:17.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.691+0000 7faeb1000640 1 -- 172.21.15.53:0/3544024318 shutdown_connections 2024-04-25T14:38:17.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.691+0000 7faeb1000640 1 --2- 172.21.15.53:0/3544024318 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fae900a3cc0 0x7fae900a6180 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:17.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.691+0000 7faeb1000640 1 --2- 172.21.15.53:0/3544024318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faeac0784f0 0x7faeac1a5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:17.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.691+0000 7faeb1000640 1 --2- 172.21.15.53:0/3544024318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faeac077b50 0x7faeac19ed40 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:17.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.691+0000 7faeb1000640 1 --2- 172.21.15.53:0/3544024318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faeac076910 0x7faeac19e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:17.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.691+0000 7faeb1000640 1 -- 172.21.15.53:0/3544024318 >> 172.21.15.53:0/3544024318 conn(0x7faeac0ff740 msgr2=0x7faeac102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:17.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.691+0000 7faeb1000640 1 -- 172.21.15.53:0/3544024318 shutdown_connections 2024-04-25T14:38:17.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.691+0000 7faeb1000640 1 -- 172.21.15.53:0/3544024318 wait complete. 2024-04-25T14:38:17.712 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-25T14:38:17.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.928+0000 7f21eec00640 1 -- 172.21.15.53:0/2907954355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21e8104090 msgr2=0x7f21e81044b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:17.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.928+0000 7f21eec00640 1 --2- 172.21.15.53:0/2907954355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21e8104090 0x7f21e81044b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f21dc009860 tx=0x7f21dc02a340 comp rx=0 tx=0).stop 2024-04-25T14:38:17.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.928+0000 7f21eec00640 1 -- 172.21.15.53:0/2907954355 shutdown_connections 2024-04-25T14:38:17.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.928+0000 7f21eec00640 1 --2- 172.21.15.53:0/2907954355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21e8105c40 0x7f21e810c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:17.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.928+0000 7f21eec00640 1 --2- 172.21.15.53:0/2907954355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21e8105280 0x7f21e8105700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:17.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.928+0000 7f21eec00640 1 --2- 172.21.15.53:0/2907954355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21e8104090 0x7f21e81044b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:17.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.928+0000 7f21eec00640 1 -- 172.21.15.53:0/2907954355 >> 172.21.15.53:0/2907954355 conn(0x7f21e80ff600 msgr2=0x7f21e8101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:17.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.928+0000 7f21eec00640 1 -- 172.21.15.53:0/2907954355 shutdown_connections 2024-04-25T14:38:17.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.929+0000 7f21eec00640 1 -- 172.21.15.53:0/2907954355 wait complete. 2024-04-25T14:38:17.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.930+0000 7f21eec00640 1 Processor -- start 2024-04-25T14:38:17.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.930+0000 7f21eec00640 1 -- start start 2024-04-25T14:38:17.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.930+0000 7f21eec00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21e8104090 0x7f21e819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:17.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.930+0000 7f21eec00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21e8105280 0x7f21e819a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:17.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.930+0000 7f21eec00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21e8105c40 0x7f21e81a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:17.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.930+0000 7f21eec00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f21e810fbd0 con 0x7f21e8104090 2024-04-25T14:38:17.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.930+0000 7f21eec00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f21e810fa50 con 0x7f21e8105c40 2024-04-25T14:38:17.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.930+0000 7f21eec00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f21e810fd50 con 0x7f21e8105280 2024-04-25T14:38:17.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.931+0000 7f21ed800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21e8104090 0x7f21e819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:17.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.931+0000 7f21ee200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21e8105c40 0x7f21e81a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:17.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.931+0000 7f21ed800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21e8104090 0x7f21e819a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36190/0 (socket says 172.21.15.53:36190) 2024-04-25T14:38:17.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.931+0000 7f21ed800640 1 -- 172.21.15.53:0/3923483528 learned_addr learned my addr 172.21.15.53:0/3923483528 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:38:17.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.931+0000 7f21ece00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21e8105280 0x7f21e819a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:17.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.931+0000 7f21ece00640 1 -- 172.21.15.53:0/3923483528 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21e8105c40 msgr2=0x7f21e81a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:17.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.931+0000 7f21ece00640 1 --2- 172.21.15.53:0/3923483528 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21e8105c40 0x7f21e81a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:17.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.931+0000 7f21ece00640 1 -- 172.21.15.53:0/3923483528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21e8104090 msgr2=0x7f21e819a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:17.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.931+0000 7f21ece00640 1 --2- 172.21.15.53:0/3923483528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21e8104090 0x7f21e819a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:17.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.931+0000 7f21ece00640 1 -- 172.21.15.53:0/3923483528 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e81a1f20 con 0x7f21e8105280 2024-04-25T14:38:17.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.932+0000 7f21ece00640 1 --2- 172.21.15.53:0/3923483528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21e8105280 0x7f21e819a840 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f21d4005fd0 tx=0x7f21d400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:17.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.932+0000 7f21e6000640 1 -- 172.21.15.53:0/3923483528 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d400cd00 con 0x7f21e8105280 2024-04-25T14:38:17.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.932+0000 7f21eec00640 1 -- 172.21.15.53:0/3923483528 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21e81a2210 con 0x7f21e8105280 2024-04-25T14:38:17.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.932+0000 7f21e6000640 1 -- 172.21.15.53:0/3923483528 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21d40035f0 con 0x7f21e8105280 2024-04-25T14:38:17.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.932+0000 7f21e6000640 1 -- 172.21.15.53:0/3923483528 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d4010690 con 0x7f21e8105280 2024-04-25T14:38:17.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.932+0000 7f21eec00640 1 -- 172.21.15.53:0/3923483528 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21e81a27b0 con 0x7f21e8105280 2024-04-25T14:38:17.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.933+0000 7f21eec00640 1 -- 172.21.15.53:0/3923483528 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21e810b470 con 0x7f21e8105280 2024-04-25T14:38:17.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.936+0000 7f21e6000640 1 -- 172.21.15.53:0/3923483528 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f21d4003a80 con 0x7f21e8105280 2024-04-25T14:38:17.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.938+0000 7f21e6000640 1 --2- 172.21.15.53:0/3923483528 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f21d00a3c40 0x7f21d00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:17.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.938+0000 7f21e6000640 1 -- 172.21.15.53:0/3923483528 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f21d40d0830 con 0x7f21e8105280 2024-04-25T14:38:17.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.938+0000 7f21ed800640 1 --2- 172.21.15.53:0/3923483528 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f21d00a3c40 0x7f21d00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:17.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.939+0000 7f21ed800640 1 --2- 172.21.15.53:0/3923483528 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f21d00a3c40 0x7f21d00a6100 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f21dc0095f0 tx=0x7f21dc030450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:17.940 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:17.940+0000 7f21e6000640 1 -- 172.21.15.53:0/3923483528 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f21d4009510 con 0x7f21e8105280 2024-04-25T14:38:18.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.182+0000 7f21eec00640 1 -- 172.21.15.53:0/3923483528 --> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f21e8062390 con 0x7f21d00a3c40 2024-04-25T14:38:18.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.195+0000 7f21e6000640 1 -- 172.21.15.53:0/3923483528 <== mgr.14152 v2:172.21.15.53:6800/4154933108 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f21e8062390 con 0x7f21d00a3c40 2024-04-25T14:38:18.196 INFO:teuthology.orchestra.run.smithi053.stdout:/volumes/qa/sv_1/df7995a7-188f-4b69-96c8-3b33c8c560c6 2024-04-25T14:38:18.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.197+0000 7f21eec00640 1 -- 172.21.15.53:0/3923483528 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f21d00a3c40 msgr2=0x7f21d00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:18.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.197+0000 7f21eec00640 1 --2- 172.21.15.53:0/3923483528 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f21d00a3c40 0x7f21d00a6100 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f21dc0095f0 tx=0x7f21dc030450 comp rx=0 tx=0).stop 2024-04-25T14:38:18.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.198+0000 7f21eec00640 1 -- 172.21.15.53:0/3923483528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21e8105280 msgr2=0x7f21e819a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:18.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.198+0000 7f21eec00640 1 --2- 172.21.15.53:0/3923483528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21e8105280 0x7f21e819a840 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f21d4005fd0 tx=0x7f21d400ee50 comp rx=0 tx=0).stop 2024-04-25T14:38:18.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.198+0000 7f21eec00640 1 -- 172.21.15.53:0/3923483528 shutdown_connections 2024-04-25T14:38:18.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.198+0000 7f21eec00640 1 --2- 172.21.15.53:0/3923483528 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f21d00a3c40 0x7f21d00a6100 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:18.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.198+0000 7f21eec00640 1 --2- 172.21.15.53:0/3923483528 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21e8105c40 0x7f21e81a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:18.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.198+0000 7f21eec00640 1 --2- 172.21.15.53:0/3923483528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21e8105280 0x7f21e819a840 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:18.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.198+0000 7f21eec00640 1 --2- 172.21.15.53:0/3923483528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21e8104090 0x7f21e819a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:18.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.198+0000 7f21eec00640 1 -- 172.21.15.53:0/3923483528 >> 172.21.15.53:0/3923483528 conn(0x7f21e80ff600 msgr2=0x7f21e8108640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:18.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.198+0000 7f21eec00640 1 -- 172.21.15.53:0/3923483528 shutdown_connections 2024-04-25T14:38:18.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.198+0000 7f21eec00640 1 -- 172.21.15.53:0/3923483528 wait complete. 2024-04-25T14:38:18.217 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:38:18.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:18 smithi053 ceph-mon[27773]: pgmap v319: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 151 B/s rd, 3.4 KiB/s wr, 12 op/s 2024-04-25T14:38:18.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:18 smithi053 ceph-mon[27773]: from='client.14562 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:38:18.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:18 smithi053 ceph-mon[27773]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-25T14:38:18.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.438+0000 7f261a000640 1 -- 172.21.15.53:0/1860849670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2614105ba0 msgr2=0x7f261410c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:18.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.438+0000 7f261a000640 1 --2- 172.21.15.53:0/1860849670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2614105ba0 0x7f261410c430 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f260c009ca0 tx=0x7f260c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:38:18.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.438+0000 7f261a000640 1 -- 172.21.15.53:0/1860849670 shutdown_connections 2024-04-25T14:38:18.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.438+0000 7f261a000640 1 --2- 172.21.15.53:0/1860849670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2614105ba0 0x7f261410c430 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:18.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.438+0000 7f261a000640 1 --2- 172.21.15.53:0/1860849670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26141051e0 0x7f2614105660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:18.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.438+0000 7f261a000640 1 --2- 172.21.15.53:0/1860849670 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2614103ff0 0x7f2614104410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:18.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.438+0000 7f261a000640 1 -- 172.21.15.53:0/1860849670 >> 172.21.15.53:0/1860849670 conn(0x7f26140ff560 msgr2=0x7f26141019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:18.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.438+0000 7f261a000640 1 -- 172.21.15.53:0/1860849670 shutdown_connections 2024-04-25T14:38:18.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.439+0000 7f261a000640 1 -- 172.21.15.53:0/1860849670 wait complete. 2024-04-25T14:38:18.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.440+0000 7f261a000640 1 Processor -- start 2024-04-25T14:38:18.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.440+0000 7f261a000640 1 -- start start 2024-04-25T14:38:18.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.440+0000 7f261a000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2614103ff0 0x7f261419a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:18.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.440+0000 7f261a000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26141051e0 0x7f261419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:18.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.440+0000 7f261a000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2614105ba0 0x7f26141a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:18.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.440+0000 7f261a000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f261410fb80 con 0x7f2614105ba0 2024-04-25T14:38:18.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.440+0000 7f261a000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f261410fa00 con 0x7f26141051e0 2024-04-25T14:38:18.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.440+0000 7f261a000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f261410fd00 con 0x7f2614103ff0 2024-04-25T14:38:18.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.440+0000 7f2613e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26141051e0 0x7f261419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:18.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.440+0000 7f2613e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26141051e0 0x7f261419a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.53:38382/0 (socket says 172.21.15.53:38382) 2024-04-25T14:38:18.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.440+0000 7f2619600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2614105ba0 0x7f26141a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:18.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.440+0000 7f2613e00640 1 -- 172.21.15.53:0/1886342653 learned_addr learned my addr 172.21.15.53:0/1886342653 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:38:18.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.440+0000 7f2618c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2614103ff0 0x7f261419a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:18.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.441+0000 7f2613e00640 1 -- 172.21.15.53:0/1886342653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2614103ff0 msgr2=0x7f261419a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:18.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.441+0000 7f2613e00640 1 --2- 172.21.15.53:0/1886342653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2614103ff0 0x7f261419a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:18.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.441+0000 7f2613e00640 1 -- 172.21.15.53:0/1886342653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2614105ba0 msgr2=0x7f26141a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:18.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.441+0000 7f2613e00640 1 --2- 172.21.15.53:0/1886342653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2614105ba0 0x7f26141a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:18.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.441+0000 7f2613e00640 1 -- 172.21.15.53:0/1886342653 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26141a1df0 con 0x7f26141051e0 2024-04-25T14:38:18.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.441+0000 7f2618c00640 1 --2- 172.21.15.53:0/1886342653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2614103ff0 0x7f261419a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:38:18.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.441+0000 7f2613e00640 1 --2- 172.21.15.53:0/1886342653 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26141051e0 0x7f261419a710 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f260400e9f0 tx=0x7f260400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:18.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.441+0000 7f2611600640 1 -- 172.21.15.53:0/1886342653 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260400cd80 con 0x7f26141051e0 2024-04-25T14:38:18.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.441+0000 7f261a000640 1 -- 172.21.15.53:0/1886342653 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26141a20e0 con 0x7f26141051e0 2024-04-25T14:38:18.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.441+0000 7f2611600640 1 -- 172.21.15.53:0/1886342653 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26040108e0 con 0x7f26141051e0 2024-04-25T14:38:18.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.441+0000 7f2611600640 1 -- 172.21.15.53:0/1886342653 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2604018980 con 0x7f26141051e0 2024-04-25T14:38:18.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.442+0000 7f261a000640 1 -- 172.21.15.53:0/1886342653 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26141a2630 con 0x7f26141051e0 2024-04-25T14:38:18.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.443+0000 7f261a000640 1 -- 172.21.15.53:0/1886342653 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f261410b400 con 0x7f26141051e0 2024-04-25T14:38:18.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.445+0000 7f2611600640 1 -- 172.21.15.53:0/1886342653 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2604010430 con 0x7f26141051e0 2024-04-25T14:38:18.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.446+0000 7f2611600640 1 --2- 172.21.15.53:0/1886342653 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f25f80a3c40 0x7f25f80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:18.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.447+0000 7f2611600640 1 -- 172.21.15.53:0/1886342653 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f26040d07b0 con 0x7f26141051e0 2024-04-25T14:38:18.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.447+0000 7f2618c00640 1 --2- 172.21.15.53:0/1886342653 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f25f80a3c40 0x7f25f80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:18.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.447+0000 7f2618c00640 1 --2- 172.21.15.53:0/1886342653 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f25f80a3c40 0x7f25f80a6100 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2614105040 tx=0x7f2600005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:18.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.451+0000 7f2611600640 1 -- 172.21.15.53:0/1886342653 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f260409ca90 con 0x7f26141051e0 2024-04-25T14:38:18.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:18 smithi183 ceph-mon[32584]: pgmap v319: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 151 B/s rd, 3.4 KiB/s wr, 12 op/s 2024-04-25T14:38:18.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:18 smithi183 ceph-mon[32584]: from='client.14562 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:38:18.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:18 smithi183 ceph-mon[32584]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-25T14:38:18.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:18 smithi150 ceph-mon[32504]: pgmap v319: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 151 B/s rd, 3.4 KiB/s wr, 12 op/s 2024-04-25T14:38:18.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:18 smithi150 ceph-mon[32504]: from='client.14562 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:38:18.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:18 smithi150 ceph-mon[32504]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-25T14:38:18.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.700+0000 7f261a000640 1 -- 172.21.15.53:0/1886342653 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f261402d810 con 0x7f26141051e0 2024-04-25T14:38:18.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.701+0000 7f2611600640 1 -- 172.21.15.53:0/1886342653 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11862 (secure 0 0 0) 0x7f26040a0d90 con 0x7f26141051e0 2024-04-25T14:38:18.702 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:38:18.703 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14460,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6841/41110727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6840","nonce":41110727},{"type":"v1","addr":"172.21.15.53:6841","nonce":41110727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2966992472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2966992472},{"type":"v1","addr":"172.21.15.150:6833","nonce":2966992472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/94143643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":94143643},{"type":"v1","addr":"172.21.15.183:6833","nonce":94143643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24470,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/506682943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":506682943},{"type":"v1","addr":"172.21.15.53:6835","nonce":506682943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1369488844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1369488844},{"type":"v1","addr":"172.21.15.150:6835","nonce":1369488844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3786232682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3786232682},{"type":"v1","addr":"172.21.15.53:6837","nonce":3786232682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T14:38:10.037451+0000","modified":"2024-04-25T14:38:15.629854+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24584,"mds_1":24565,"mds_2":24560},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24532":{"gid":24532,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1189328602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1189328602},{"type":"v1","addr":"172.21.15.183:6835","nonce":1189328602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/2344343627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2344343627},{"type":"v1","addr":"172.21.15.150:6837","nonce":2344343627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24544":{"gid":24544,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1163336384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1163336384},{"type":"v1","addr":"172.21.15.183:6837","nonce":1163336384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6839/570000540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":570000540},{"type":"v1","addr":"172.21.15.53:6839","nonce":570000540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.183:6839/1554258915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1554258915},{"type":"v1","addr":"172.21.15.183:6839","nonce":1554258915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/149573616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":149573616},{"type":"v1","addr":"172.21.15.150:6839","nonce":149573616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24565,24584,24560]},"id":1}]} 2024-04-25T14:38:18.703 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 21 2024-04-25T14:38:18.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.704+0000 7f261a000640 1 -- 172.21.15.53:0/1886342653 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f25f80a3c40 msgr2=0x7f25f80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:18.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.704+0000 7f261a000640 1 --2- 172.21.15.53:0/1886342653 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f25f80a3c40 0x7f25f80a6100 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2614105040 tx=0x7f2600005f90 comp rx=0 tx=0).stop 2024-04-25T14:38:18.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.704+0000 7f261a000640 1 -- 172.21.15.53:0/1886342653 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26141051e0 msgr2=0x7f261419a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:18.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.704+0000 7f261a000640 1 --2- 172.21.15.53:0/1886342653 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26141051e0 0x7f261419a710 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f260400e9f0 tx=0x7f260400eeb0 comp rx=0 tx=0).stop 2024-04-25T14:38:18.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.704+0000 7f261a000640 1 -- 172.21.15.53:0/1886342653 shutdown_connections 2024-04-25T14:38:18.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.704+0000 7f261a000640 1 --2- 172.21.15.53:0/1886342653 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f25f80a3c40 0x7f25f80a6100 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:18.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.704+0000 7f261a000640 1 --2- 172.21.15.53:0/1886342653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2614105ba0 0x7f26141a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:18.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.704+0000 7f261a000640 1 --2- 172.21.15.53:0/1886342653 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26141051e0 0x7f261419a710 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:18.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.704+0000 7f261a000640 1 --2- 172.21.15.53:0/1886342653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2614103ff0 0x7f261419a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:18.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.704+0000 7f261a000640 1 -- 172.21.15.53:0/1886342653 >> 172.21.15.53:0/1886342653 conn(0x7f26140ff560 msgr2=0x7f26141085d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:18.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.704+0000 7f261a000640 1 -- 172.21.15.53:0/1886342653 shutdown_connections 2024-04-25T14:38:18.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.704+0000 7f261a000640 1 -- 172.21.15.53:0/1886342653 wait complete. 2024-04-25T14:38:18.726 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:38:18.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.931+0000 7f8f1f200640 1 -- 172.21.15.53:0/3977847207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f20109f80 msgr2=0x7f8f20110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:18.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.931+0000 7f8f1f200640 1 --2- 172.21.15.53:0/3977847207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f20109f80 0x7f8f20110810 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8f18009ca0 tx=0x7f8f1802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:38:18.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.931+0000 7f8f1f200640 1 -- 172.21.15.53:0/3977847207 shutdown_connections 2024-04-25T14:38:18.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.931+0000 7f8f1f200640 1 --2- 172.21.15.53:0/3977847207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f20109f80 0x7f8f20110810 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:18.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.931+0000 7f8f1f200640 1 --2- 172.21.15.53:0/3977847207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f201095c0 0x7f8f20109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:18.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.931+0000 7f8f1f200640 1 --2- 172.21.15.53:0/3977847207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f201083d0 0x7f8f201087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:18.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.931+0000 7f8f1f200640 1 -- 172.21.15.53:0/3977847207 >> 172.21.15.53:0/3977847207 conn(0x7f8f20077390 msgr2=0x7f8f200797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:18.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.931+0000 7f8f1f200640 1 -- 172.21.15.53:0/3977847207 shutdown_connections 2024-04-25T14:38:18.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.932+0000 7f8f1f200640 1 -- 172.21.15.53:0/3977847207 wait complete. 2024-04-25T14:38:18.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.932+0000 7f8f1f200640 1 Processor -- start 2024-04-25T14:38:18.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.933+0000 7f8f1f200640 1 -- start start 2024-04-25T14:38:18.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.933+0000 7f8f1f200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f201083d0 0x7f8f2019e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:18.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.933+0000 7f8f1f200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f201095c0 0x7f8f2019eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:18.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.933+0000 7f8f1f200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f20109f80 0x7f8f201a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:18.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.933+0000 7f8f1f200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8f20113f30 con 0x7f8f201095c0 2024-04-25T14:38:18.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.933+0000 7f8f1f200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8f20113db0 con 0x7f8f20109f80 2024-04-25T14:38:18.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.933+0000 7f8f1f200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8f201140b0 con 0x7f8f201083d0 2024-04-25T14:38:18.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.933+0000 7f8f17e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f201095c0 0x7f8f2019eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:18.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.933+0000 7f8f1de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f201083d0 0x7f8f2019e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:18.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.933+0000 7f8f1e800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f20109f80 0x7f8f201a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:18.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.933+0000 7f8f17e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f201095c0 0x7f8f2019eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36210/0 (socket says 172.21.15.53:36210) 2024-04-25T14:38:18.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.933+0000 7f8f17e00640 1 -- 172.21.15.53:0/855048276 learned_addr learned my addr 172.21.15.53:0/855048276 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:38:18.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.934+0000 7f8f17e00640 1 -- 172.21.15.53:0/855048276 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f201083d0 msgr2=0x7f8f2019e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:18.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.934+0000 7f8f17e00640 1 --2- 172.21.15.53:0/855048276 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f201083d0 0x7f8f2019e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:18.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.934+0000 7f8f17e00640 1 -- 172.21.15.53:0/855048276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f20109f80 msgr2=0x7f8f201a5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:18.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.934+0000 7f8f17e00640 1 --2- 172.21.15.53:0/855048276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f20109f80 0x7f8f201a5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:18.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.934+0000 7f8f17e00640 1 -- 172.21.15.53:0/855048276 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f201a61c0 con 0x7f8f201095c0 2024-04-25T14:38:18.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.934+0000 7f8f1e800640 1 --2- 172.21.15.53:0/855048276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f20109f80 0x7f8f201a5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:38:18.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.934+0000 7f8f17e00640 1 --2- 172.21.15.53:0/855048276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f201095c0 0x7f8f2019eae0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8f0800ea60 tx=0x7f8f0800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:18.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.935+0000 7f8f16a00640 1 -- 172.21.15.53:0/855048276 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0800cde0 con 0x7f8f201095c0 2024-04-25T14:38:18.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.935+0000 7f8f16a00640 1 -- 172.21.15.53:0/855048276 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f080035f0 con 0x7f8f201095c0 2024-04-25T14:38:18.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.935+0000 7f8f1f200640 1 -- 172.21.15.53:0/855048276 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f201a64b0 con 0x7f8f201095c0 2024-04-25T14:38:18.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.935+0000 7f8f16a00640 1 -- 172.21.15.53:0/855048276 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08010690 con 0x7f8f201095c0 2024-04-25T14:38:18.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.935+0000 7f8f1f200640 1 -- 172.21.15.53:0/855048276 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f201a6a00 con 0x7f8f201095c0 2024-04-25T14:38:18.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.936+0000 7f8f1f200640 1 -- 172.21.15.53:0/855048276 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f2010f6f0 con 0x7f8f201095c0 2024-04-25T14:38:18.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.938+0000 7f8f16a00640 1 -- 172.21.15.53:0/855048276 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8f080037a0 con 0x7f8f201095c0 2024-04-25T14:38:18.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.940+0000 7f8f16a00640 1 --2- 172.21.15.53:0/855048276 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f8f040a3c40 0x7f8f040a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:18.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.941+0000 7f8f16a00640 1 -- 172.21.15.53:0/855048276 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f8f080d0770 con 0x7f8f201095c0 2024-04-25T14:38:18.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.941+0000 7f8f1de00640 1 --2- 172.21.15.53:0/855048276 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f8f040a3c40 0x7f8f040a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:18.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.941+0000 7f8f1de00640 1 --2- 172.21.15.53:0/855048276 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f8f040a3c40 0x7f8f040a6100 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8f20109420 tx=0x7f8f10005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:18.946 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:18.946+0000 7f8f16a00640 1 -- 172.21.15.53:0/855048276 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8f0809d0a0 con 0x7f8f201095c0 2024-04-25T14:38:19.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.156+0000 7f8f1f200640 1 -- 172.21.15.53:0/855048276 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8f2002d800 con 0x7f8f201095c0 2024-04-25T14:38:19.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.160+0000 7f8f16a00640 1 -- 172.21.15.53:0/855048276 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22171 (secure 0 0 0) 0x7f8f0809c9a0 con 0x7f8f201095c0 2024-04-25T14:38:19.162 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:38:19.162 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":88,"fsid":"fd30e6ce-030f-11ef-bc93-c7b262605968","created":"2024-04-25T14:28:54.330326+0000","modified":"2024-04-25T14:38:11.415753+0000","last_up_change":"2024-04-25T14:37:10.921148+0000","last_in_change":"2024-04-25T14:36:49.566459+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:33:02.338115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:38:07.517974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:38:08.994122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aecfbaf0-39da-4749-a49b-c7205dabd4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6803","nonce":464176015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6805","nonce":464176015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6809","nonce":464176015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6807","nonce":464176015}]},"public_addr":"172.21.15.53:6803/464176015","cluster_addr":"172.21.15.53:6805/464176015","heartbeat_back_addr":"172.21.15.53:6809/464176015","heartbeat_front_addr":"172.21.15.53:6807/464176015","state":["exists","up"]},{"osd":1,"uuid":"2c4a21ae-c43e-4839-8a56-c3fc63aeb7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6801","nonce":1304621982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6803","nonce":1304621982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6807","nonce":1304621982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6805","nonce":1304621982}]},"public_addr":"172.21.15.150:6801/1304621982","cluster_addr":"172.21.15.150:6803/1304621982","heartbeat_back_addr":"172.21.15.150:6807/1304621982","heartbeat_front_addr":"172.21.15.150:6805/1304621982","state":["exists","up"]},{"osd":2,"uuid":"e8effd43-adc0-472a-9eed-a9273b06dc0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6801","nonce":574509311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6803","nonce":574509311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6807","nonce":574509311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6805","nonce":574509311}]},"public_addr":"172.21.15.183:6801/574509311","cluster_addr":"172.21.15.183:6803/574509311","heartbeat_back_addr":"172.21.15.183:6807/574509311","heartbeat_front_addr":"172.21.15.183:6805/574509311","state":["exists","up"]},{"osd":3,"uuid":"e7cfced4-9b40-40de-a303-75ea8d5df4a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6811","nonce":570672219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6813","nonce":570672219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6817","nonce":570672219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6815","nonce":570672219}]},"public_addr":"172.21.15.53:6811/570672219","cluster_addr":"172.21.15.53:6813/570672219","heartbeat_back_addr":"172.21.15.53:6817/570672219","heartbeat_front_addr":"172.21.15.53:6815/570672219","state":["exists","up"]},{"osd":4,"uuid":"44a9e070-2282-43f1-9f05-b1f8ff81e540","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6809","nonce":1526850103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6811","nonce":1526850103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6815","nonce":1526850103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6813","nonce":1526850103}]},"public_addr":"172.21.15.150:6809/1526850103","cluster_addr":"172.21.15.150:6811/1526850103","heartbeat_back_addr":"172.21.15.150:6815/1526850103","heartbeat_front_addr":"172.21.15.150:6813/1526850103","state":["exists","up"]},{"osd":5,"uuid":"2c5619d2-c0f9-47d6-bf7e-2f4afd1b09f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6809","nonce":773726576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6811","nonce":773726576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6815","nonce":773726576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6813","nonce":773726576}]},"public_addr":"172.21.15.183:6809/773726576","cluster_addr":"172.21.15.183:6811/773726576","heartbeat_back_addr":"172.21.15.183:6815/773726576","heartbeat_front_addr":"172.21.15.183:6813/773726576","state":["exists","up"]},{"osd":6,"uuid":"1cf74d94-c118-4738-b6db-040af458a8eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6819","nonce":355884319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6821","nonce":355884319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6825","nonce":355884319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6823","nonce":355884319}]},"public_addr":"172.21.15.53:6819/355884319","cluster_addr":"172.21.15.53:6821/355884319","heartbeat_back_addr":"172.21.15.53:6825/355884319","heartbeat_front_addr":"172.21.15.53:6823/355884319","state":["exists","up"]},{"osd":7,"uuid":"0ce97c04-cd41-4080-bbbc-be9cf5fadc02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6817","nonce":229022562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6819","nonce":229022562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6823","nonce":229022562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6821","nonce":229022562}]},"public_addr":"172.21.15.150:6817/229022562","cluster_addr":"172.21.15.150:6819/229022562","heartbeat_back_addr":"172.21.15.150:6823/229022562","heartbeat_front_addr":"172.21.15.150:6821/229022562","state":["exists","up"]},{"osd":8,"uuid":"5d57787d-2a7c-4514-bd01-357ced413f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6817","nonce":301003844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6819","nonce":301003844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6823","nonce":301003844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6821","nonce":301003844}]},"public_addr":"172.21.15.183:6817/301003844","cluster_addr":"172.21.15.183:6819/301003844","heartbeat_back_addr":"172.21.15.183:6823/301003844","heartbeat_front_addr":"172.21.15.183:6821/301003844","state":["exists","up"]},{"osd":9,"uuid":"e11dd8eb-c55a-4d70-8c58-457540b7a50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6827","nonce":3449260978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6829","nonce":3449260978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6833","nonce":3449260978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6831","nonce":3449260978}]},"public_addr":"172.21.15.53:6827/3449260978","cluster_addr":"172.21.15.53:6829/3449260978","heartbeat_back_addr":"172.21.15.53:6833/3449260978","heartbeat_front_addr":"172.21.15.53:6831/3449260978","state":["exists","up"]},{"osd":10,"uuid":"5de4803d-ad37-4a90-add5-0048f44fc4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6825","nonce":3435992308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6827","nonce":3435992308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6831","nonce":3435992308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6829","nonce":3435992308}]},"public_addr":"172.21.15.150:6825/3435992308","cluster_addr":"172.21.15.150:6827/3435992308","heartbeat_back_addr":"172.21.15.150:6831/3435992308","heartbeat_front_addr":"172.21.15.150:6829/3435992308","state":["exists","up"]},{"osd":11,"uuid":"0e819a68-ed56-477e-b9c0-81e8cc0bce11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6825","nonce":1957842686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6827","nonce":1957842686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6831","nonce":1957842686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6829","nonce":1957842686}]},"public_addr":"172.21.15.183:6825/1957842686","cluster_addr":"172.21.15.183:6827/1957842686","heartbeat_back_addr":"172.21.15.183:6831/1957842686","heartbeat_front_addr":"172.21.15.183:6829/1957842686","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:09.840629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:34.769537+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:58.112969+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:33:29.656790+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:33:55.181287+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:34:19.772645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:34:53.833921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:35:19.076733+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:35:43.163390+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:36:16.568340+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:36:42.492768+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:37:07.944639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6800/3887954889":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/1014501022":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/2576091441":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/1233527569":"2024-04-26T14:29:38.106242+0000","172.21.15.53:6801/3887954889":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/931889810":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/177196141":"2024-04-26T14:30:14.162383+0000","172.21.15.53:6800/1722497390":"2024-04-26T14:29:38.106242+0000","172.21.15.53:6801/1722497390":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/1927549198":"2024-04-26T14:30:14.162383+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:38:19.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.163+0000 7f8f1f200640 1 -- 172.21.15.53:0/855048276 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f8f040a3c40 msgr2=0x7f8f040a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:19.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.163+0000 7f8f1f200640 1 --2- 172.21.15.53:0/855048276 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f8f040a3c40 0x7f8f040a6100 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8f20109420 tx=0x7f8f10005f90 comp rx=0 tx=0).stop 2024-04-25T14:38:19.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.163+0000 7f8f1f200640 1 -- 172.21.15.53:0/855048276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f201095c0 msgr2=0x7f8f2019eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:19.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.163+0000 7f8f1f200640 1 --2- 172.21.15.53:0/855048276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f201095c0 0x7f8f2019eae0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8f0800ea60 tx=0x7f8f0800ef20 comp rx=0 tx=0).stop 2024-04-25T14:38:19.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.163+0000 7f8f1f200640 1 -- 172.21.15.53:0/855048276 shutdown_connections 2024-04-25T14:38:19.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.163+0000 7f8f1f200640 1 --2- 172.21.15.53:0/855048276 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f8f040a3c40 0x7f8f040a6100 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:19.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.163+0000 7f8f1f200640 1 --2- 172.21.15.53:0/855048276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f20109f80 0x7f8f201a5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:19.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.163+0000 7f8f1f200640 1 --2- 172.21.15.53:0/855048276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f201095c0 0x7f8f2019eae0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:19.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.163+0000 7f8f1f200640 1 --2- 172.21.15.53:0/855048276 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f201083d0 0x7f8f2019e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:19.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.163+0000 7f8f1f200640 1 -- 172.21.15.53:0/855048276 >> 172.21.15.53:0/855048276 conn(0x7f8f20077390 msgr2=0x7f8f2010c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:19.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.163+0000 7f8f1f200640 1 -- 172.21.15.53:0/855048276 shutdown_connections 2024-04-25T14:38:19.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.164+0000 7f8f1f200640 1 -- 172.21.15.53:0/855048276 wait complete. 2024-04-25T14:38:19.184 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T14:38:19.185 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:38:19.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:19 smithi053 ceph-mon[27773]: from='client.24686 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:38:19.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:19 smithi053 ceph-mon[27773]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:38:19.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:19 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1886342653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:38:19.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.393+0000 7f791d400640 1 -- 172.21.15.53:0/1150589057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7918105b80 msgr2=0x7f791810c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:19.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.393+0000 7f791d400640 1 --2- 172.21.15.53:0/1150589057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7918105b80 0x7f791810c410 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7910009ca0 tx=0x7f791002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:38:19.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.393+0000 7f791d400640 1 -- 172.21.15.53:0/1150589057 shutdown_connections 2024-04-25T14:38:19.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.393+0000 7f791d400640 1 --2- 172.21.15.53:0/1150589057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7918105b80 0x7f791810c410 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:19.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.393+0000 7f791d400640 1 --2- 172.21.15.53:0/1150589057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79181051c0 0x7f7918105640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:19.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.393+0000 7f791d400640 1 --2- 172.21.15.53:0/1150589057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7918103fd0 0x7f79181043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:19.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.393+0000 7f791d400640 1 -- 172.21.15.53:0/1150589057 >> 172.21.15.53:0/1150589057 conn(0x7f79180ff560 msgr2=0x7f7918101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:19.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.393+0000 7f791d400640 1 -- 172.21.15.53:0/1150589057 shutdown_connections 2024-04-25T14:38:19.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.393+0000 7f791d400640 1 -- 172.21.15.53:0/1150589057 wait complete. 2024-04-25T14:38:19.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.394+0000 7f791d400640 1 Processor -- start 2024-04-25T14:38:19.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.394+0000 7f791d400640 1 -- start start 2024-04-25T14:38:19.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.394+0000 7f791d400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7918103fd0 0x7f791819a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:19.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.394+0000 7f791d400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79181051c0 0x7f791819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:19.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.394+0000 7f791d400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7918105b80 0x7f79181a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:19.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.394+0000 7f791d400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f791810fb60 con 0x7f7918105b80 2024-04-25T14:38:19.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.394+0000 7f791d400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f791810f9e0 con 0x7f79181051c0 2024-04-25T14:38:19.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.394+0000 7f791d400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f791810fce0 con 0x7f7918103fd0 2024-04-25T14:38:19.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.395+0000 7f791ca00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7918105b80 0x7f79181a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:19.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.395+0000 7f791ca00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7918105b80 0x7f79181a1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36226/0 (socket says 172.21.15.53:36226) 2024-04-25T14:38:19.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.395+0000 7f791ca00640 1 -- 172.21.15.53:0/1181309152 learned_addr learned my addr 172.21.15.53:0/1181309152 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:38:19.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.395+0000 7f7917400640 1 --2- 172.21.15.53:0/1181309152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79181051c0 0x7f791819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:19.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.395+0000 7f7917e00640 1 --2- 172.21.15.53:0/1181309152 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7918103fd0 0x7f791819a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:19.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.395+0000 7f791ca00640 1 -- 172.21.15.53:0/1181309152 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7918103fd0 msgr2=0x7f791819a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:19.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.395+0000 7f791ca00640 1 --2- 172.21.15.53:0/1181309152 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7918103fd0 0x7f791819a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:19.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.395+0000 7f791ca00640 1 -- 172.21.15.53:0/1181309152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79181051c0 msgr2=0x7f791819a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:19.398 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.395+0000 7f791ca00640 1 --2- 172.21.15.53:0/1181309152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79181051c0 0x7f791819a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:19.398 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.395+0000 7f791ca00640 1 -- 172.21.15.53:0/1181309152 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79181a1dc0 con 0x7f7918105b80 2024-04-25T14:38:19.398 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.395+0000 7f791ca00640 1 --2- 172.21.15.53:0/1181309152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7918105b80 0x7f79181a1750 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7910033040 tx=0x7f79100317c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:19.398 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.396+0000 7f7914c00640 1 -- 172.21.15.53:0/1181309152 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7910003b70 con 0x7f7918105b80 2024-04-25T14:38:19.398 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.396+0000 7f7914c00640 1 -- 172.21.15.53:0/1181309152 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7910003d10 con 0x7f7918105b80 2024-04-25T14:38:19.398 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.396+0000 7f7914c00640 1 -- 172.21.15.53:0/1181309152 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791002fbd0 con 0x7f7918105b80 2024-04-25T14:38:19.398 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.396+0000 7f791d400640 1 -- 172.21.15.53:0/1181309152 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79181a2050 con 0x7f7918105b80 2024-04-25T14:38:19.398 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.396+0000 7f791d400640 1 -- 172.21.15.53:0/1181309152 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79181a2550 con 0x7f7918105b80 2024-04-25T14:38:19.398 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.397+0000 7f791d400640 1 -- 172.21.15.53:0/1181309152 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f791810b2f0 con 0x7f7918105b80 2024-04-25T14:38:19.399 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.399+0000 7f7914c00640 1 -- 172.21.15.53:0/1181309152 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f791002fd70 con 0x7f7918105b80 2024-04-25T14:38:19.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.401+0000 7f7914c00640 1 --2- 172.21.15.53:0/1181309152 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f78fc0a3cc0 0x7f78fc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:19.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.402+0000 7f7914c00640 1 -- 172.21.15.53:0/1181309152 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f79100f1940 con 0x7f7918105b80 2024-04-25T14:38:19.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.402+0000 7f7917e00640 1 --2- 172.21.15.53:0/1181309152 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f78fc0a3cc0 0x7f78fc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:19.403 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.402+0000 7f7917e00640 1 --2- 172.21.15.53:0/1181309152 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f78fc0a3cc0 0x7f78fc0a6180 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7908005fd0 tx=0x7f790800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:19.406 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.406+0000 7f7914c00640 1 -- 172.21.15.53:0/1181309152 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f79100be380 con 0x7f7918105b80 2024-04-25T14:38:19.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:19 smithi183 ceph-mon[32584]: from='client.24686 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:38:19.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:19 smithi183 ceph-mon[32584]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:38:19.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:19 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/1886342653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:38:19.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:19 smithi150 ceph-mon[32504]: from='client.24686 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:38:19.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:19 smithi150 ceph-mon[32504]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:38:19.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:19 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/1886342653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:38:19.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.608+0000 7f791d400640 1 -- 172.21.15.53:0/1181309152 --> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7918062390 con 0x7f78fc0a3cc0 2024-04-25T14:38:19.623 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.622+0000 7f7914c00640 1 -- 172.21.15.53:0/1181309152 <== mgr.14152 v2:172.21.15.53:6800/4154933108 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258736 (secure 0 0 0) 0x7f7918062390 con 0x7f78fc0a3cc0 2024-04-25T14:38:19.628 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:38:19.629 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-04-25T14:38:19.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.627+0000 7f791d400640 1 -- 172.21.15.53:0/1181309152 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f78fc0a3cc0 msgr2=0x7f78fc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:19.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.627+0000 7f791d400640 1 --2- 172.21.15.53:0/1181309152 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f78fc0a3cc0 0x7f78fc0a6180 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7908005fd0 tx=0x7f790800b410 comp rx=0 tx=0).stop 2024-04-25T14:38:19.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.627+0000 7f791d400640 1 -- 172.21.15.53:0/1181309152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7918105b80 msgr2=0x7f79181a1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:19.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.627+0000 7f791d400640 1 --2- 172.21.15.53:0/1181309152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7918105b80 0x7f79181a1750 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7910033040 tx=0x7f79100317c0 comp rx=0 tx=0).stop 2024-04-25T14:38:19.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.628+0000 7f791d400640 1 -- 172.21.15.53:0/1181309152 shutdown_connections 2024-04-25T14:38:19.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.628+0000 7f791d400640 1 --2- 172.21.15.53:0/1181309152 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f78fc0a3cc0 0x7f78fc0a6180 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:19.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.628+0000 7f791d400640 1 --2- 172.21.15.53:0/1181309152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7918105b80 0x7f79181a1750 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:19.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.628+0000 7f791d400640 1 --2- 172.21.15.53:0/1181309152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79181051c0 0x7f791819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:19.633 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.628+0000 7f791d400640 1 --2- 172.21.15.53:0/1181309152 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7918103fd0 0x7f791819a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:19.633 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.628+0000 7f791d400640 1 -- 172.21.15.53:0/1181309152 >> 172.21.15.53:0/1181309152 conn(0x7f79180ff560 msgr2=0x7f79181084c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:19.633 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.628+0000 7f791d400640 1 -- 172.21.15.53:0/1181309152 shutdown_connections 2024-04-25T14:38:19.633 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.628+0000 7f791d400640 1 -- 172.21.15.53:0/1181309152 wait complete. 2024-04-25T14:38:19.649 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":320,"stamp":"2024-04-25T14:38:18.305302+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595359,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":151,"num_read_kb":132,"num_write":295,"num_write_kb":2975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":236,"ondisk_log_size":236,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374692,"kb_used_data":13044,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567132,"statfs":{"total":1151940427776,"available":1151556743168,"internally_reserved":0,"allocated":13357056,"data_stored":7214587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19137,"internal_metadata":370193727},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4991,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":15,"num_read_kb":13,"num_write":114,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003640"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.214257+0000","last_change":"2024-04-25T14:38:10.435535+0000","last_active":"2024-04-25T14:38:12.214257+0000","last_peered":"2024-04-25T14:38:12.214257+0000","last_clean":"2024-04-25T14:38:12.214257+0000","last_became_active":"2024-04-25T14:38:10.435205+0000","last_became_peered":"2024-04-25T14:38:10.435205+0000","last_unstale":"2024-04-25T14:38:12.214257+0000","last_undegraded":"2024-04-25T14:38:12.214257+0000","last_fullsized":"2024-04-25T14:38:12.214257+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:12.301771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.507996+0000","last_change":"2024-04-25T14:38:10.428216+0000","last_active":"2024-04-25T14:38:11.507996+0000","last_peered":"2024-04-25T14:38:11.507996+0000","last_clean":"2024-04-25T14:38:11.507996+0000","last_became_active":"2024-04-25T14:38:10.427419+0000","last_became_peered":"2024-04-25T14:38:10.427419+0000","last_unstale":"2024-04-25T14:38:11.507996+0000","last_undegraded":"2024-04-25T14:38:11.507996+0000","last_fullsized":"2024-04-25T14:38:11.507996+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:42.580424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.507851+0000","last_change":"2024-04-25T14:38:10.427827+0000","last_active":"2024-04-25T14:38:11.507851+0000","last_peered":"2024-04-25T14:38:11.507851+0000","last_clean":"2024-04-25T14:38:11.507851+0000","last_became_active":"2024-04-25T14:38:10.427435+0000","last_became_peered":"2024-04-25T14:38:10.427435+0000","last_unstale":"2024-04-25T14:38:11.507851+0000","last_undegraded":"2024-04-25T14:38:11.507851+0000","last_fullsized":"2024-04-25T14:38:11.507851+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:15.539399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.420475+0000","last_change":"2024-04-25T14:38:10.423751+0000","last_active":"2024-04-25T14:38:11.420475+0000","last_peered":"2024-04-25T14:38:11.420475+0000","last_clean":"2024-04-25T14:38:11.420475+0000","last_became_active":"2024-04-25T14:38:10.423409+0000","last_became_peered":"2024-04-25T14:38:10.423409+0000","last_unstale":"2024-04-25T14:38:11.420475+0000","last_undegraded":"2024-04-25T14:38:11.420475+0000","last_fullsized":"2024-04-25T14:38:11.420475+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:45.403788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.539992+0000","last_change":"2024-04-25T14:38:10.430000+0000","last_active":"2024-04-25T14:38:11.539992+0000","last_peered":"2024-04-25T14:38:11.539992+0000","last_clean":"2024-04-25T14:38:11.539992+0000","last_became_active":"2024-04-25T14:38:10.429684+0000","last_became_peered":"2024-04-25T14:38:10.429684+0000","last_unstale":"2024-04-25T14:38:11.539992+0000","last_undegraded":"2024-04-25T14:38:11.539992+0000","last_fullsized":"2024-04-25T14:38:11.539992+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:18.421970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.214524+0000","last_change":"2024-04-25T14:38:10.425057+0000","last_active":"2024-04-25T14:38:12.214524+0000","last_peered":"2024-04-25T14:38:12.214524+0000","last_clean":"2024-04-25T14:38:12.214524+0000","last_became_active":"2024-04-25T14:38:10.424463+0000","last_became_peered":"2024-04-25T14:38:10.424463+0000","last_unstale":"2024-04-25T14:38:12.214524+0000","last_undegraded":"2024-04-25T14:38:12.214524+0000","last_fullsized":"2024-04-25T14:38:12.214524+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:56.752189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.213996+0000","last_change":"2024-04-25T14:38:10.420022+0000","last_active":"2024-04-25T14:38:12.213996+0000","last_peered":"2024-04-25T14:38:12.213996+0000","last_clean":"2024-04-25T14:38:12.213996+0000","last_became_active":"2024-04-25T14:38:10.419673+0000","last_became_peered":"2024-04-25T14:38:10.419673+0000","last_unstale":"2024-04-25T14:38:12.213996+0000","last_undegraded":"2024-04-25T14:38:12.213996+0000","last_fullsized":"2024-04-25T14:38:12.213996+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:09.177814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.419909+0000","last_change":"2024-04-25T14:38:10.436154+0000","last_active":"2024-04-25T14:38:11.419909+0000","last_peered":"2024-04-25T14:38:11.419909+0000","last_clean":"2024-04-25T14:38:11.419909+0000","last_became_active":"2024-04-25T14:38:10.435788+0000","last_became_peered":"2024-04-25T14:38:10.435788+0000","last_unstale":"2024-04-25T14:38:11.419909+0000","last_undegraded":"2024-04-25T14:38:11.419909+0000","last_fullsized":"2024-04-25T14:38:11.419909+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:15.317674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.458628+0000","last_change":"2024-04-25T14:38:10.424086+0000","last_active":"2024-04-25T14:38:11.458628+0000","last_peered":"2024-04-25T14:38:11.458628+0000","last_clean":"2024-04-25T14:38:11.458628+0000","last_became_active":"2024-04-25T14:38:10.423769+0000","last_became_peered":"2024-04-25T14:38:10.423769+0000","last_unstale":"2024-04-25T14:38:11.458628+0000","last_undegraded":"2024-04-25T14:38:11.458628+0000","last_fullsized":"2024-04-25T14:38:11.458628+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:19:22.736053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.215123+0000","last_change":"2024-04-25T14:38:10.427049+0000","last_active":"2024-04-25T14:38:12.215123+0000","last_peered":"2024-04-25T14:38:12.215123+0000","last_clean":"2024-04-25T14:38:12.215123+0000","last_became_active":"2024-04-25T14:38:10.426729+0000","last_became_peered":"2024-04-25T14:38:10.426729+0000","last_unstale":"2024-04-25T14:38:12.215123+0000","last_undegraded":"2024-04-25T14:38:12.215123+0000","last_fullsized":"2024-04-25T14:38:12.215123+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:00.849650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.214437+0000","last_change":"2024-04-25T14:38:10.424976+0000","last_active":"2024-04-25T14:38:12.214437+0000","last_peered":"2024-04-25T14:38:12.214437+0000","last_clean":"2024-04-25T14:38:12.214437+0000","last_became_active":"2024-04-25T14:38:10.424585+0000","last_became_peered":"2024-04-25T14:38:10.424585+0000","last_unstale":"2024-04-25T14:38:12.214437+0000","last_undegraded":"2024-04-25T14:38:12.214437+0000","last_fullsized":"2024-04-25T14:38:12.214437+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:54.639471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.420470+0000","last_change":"2024-04-25T14:38:10.418022+0000","last_active":"2024-04-25T14:38:11.420470+0000","last_peered":"2024-04-25T14:38:11.420470+0000","last_clean":"2024-04-25T14:38:11.420470+0000","last_became_active":"2024-04-25T14:38:10.417735+0000","last_became_peered":"2024-04-25T14:38:10.417735+0000","last_unstale":"2024-04-25T14:38:11.420470+0000","last_undegraded":"2024-04-25T14:38:11.420470+0000","last_fullsized":"2024-04-25T14:38:11.420470+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:48.785521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.759636+0000","last_change":"2024-04-25T14:38:10.423464+0000","last_active":"2024-04-25T14:38:11.759636+0000","last_peered":"2024-04-25T14:38:11.759636+0000","last_clean":"2024-04-25T14:38:11.759636+0000","last_became_active":"2024-04-25T14:38:10.423105+0000","last_became_peered":"2024-04-25T14:38:10.423105+0000","last_unstale":"2024-04-25T14:38:11.759636+0000","last_undegraded":"2024-04-25T14:38:11.759636+0000","last_fullsized":"2024-04-25T14:38:11.759636+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:26.568198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.507518+0000","last_change":"2024-04-25T14:38:10.426870+0000","last_active":"2024-04-25T14:38:11.507518+0000","last_peered":"2024-04-25T14:38:11.507518+0000","last_clean":"2024-04-25T14:38:11.507518+0000","last_became_active":"2024-04-25T14:38:10.426506+0000","last_became_peered":"2024-04-25T14:38:10.426506+0000","last_unstale":"2024-04-25T14:38:11.507518+0000","last_undegraded":"2024-04-25T14:38:11.507518+0000","last_fullsized":"2024-04-25T14:38:11.507518+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:16:49.539221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.507428+0000","last_change":"2024-04-25T14:38:10.434618+0000","last_active":"2024-04-25T14:38:11.507428+0000","last_peered":"2024-04-25T14:38:11.507428+0000","last_clean":"2024-04-25T14:38:11.507428+0000","last_became_active":"2024-04-25T14:38:10.430661+0000","last_became_peered":"2024-04-25T14:38:10.430661+0000","last_unstale":"2024-04-25T14:38:11.507428+0000","last_undegraded":"2024-04-25T14:38:11.507428+0000","last_fullsized":"2024-04-25T14:38:11.507428+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:44.548788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.458022+0000","last_change":"2024-04-25T14:38:10.429898+0000","last_active":"2024-04-25T14:38:11.458022+0000","last_peered":"2024-04-25T14:38:11.458022+0000","last_clean":"2024-04-25T14:38:11.458022+0000","last_became_active":"2024-04-25T14:38:10.429557+0000","last_became_peered":"2024-04-25T14:38:10.429557+0000","last_unstale":"2024-04-25T14:38:11.458022+0000","last_undegraded":"2024-04-25T14:38:11.458022+0000","last_fullsized":"2024-04-25T14:38:11.458022+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:54.258526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.214414+0000","last_change":"2024-04-25T14:38:10.435134+0000","last_active":"2024-04-25T14:38:12.214414+0000","last_peered":"2024-04-25T14:38:12.214414+0000","last_clean":"2024-04-25T14:38:12.214414+0000","last_became_active":"2024-04-25T14:38:10.434775+0000","last_became_peered":"2024-04-25T14:38:10.434775+0000","last_unstale":"2024-04-25T14:38:12.214414+0000","last_undegraded":"2024-04-25T14:38:12.214414+0000","last_fullsized":"2024-04-25T14:38:12.214414+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:49.827881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.421743+0000","last_change":"2024-04-25T14:38:10.424929+0000","last_active":"2024-04-25T14:38:11.421743+0000","last_peered":"2024-04-25T14:38:11.421743+0000","last_clean":"2024-04-25T14:38:11.421743+0000","last_became_active":"2024-04-25T14:38:10.424399+0000","last_became_peered":"2024-04-25T14:38:10.424399+0000","last_unstale":"2024-04-25T14:38:11.421743+0000","last_undegraded":"2024-04-25T14:38:11.421743+0000","last_fullsized":"2024-04-25T14:38:11.421743+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:08.834176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.214480+0000","last_change":"2024-04-25T14:38:10.444004+0000","last_active":"2024-04-25T14:38:12.214480+0000","last_peered":"2024-04-25T14:38:12.214480+0000","last_clean":"2024-04-25T14:38:12.214480+0000","last_became_active":"2024-04-25T14:38:10.443669+0000","last_became_peered":"2024-04-25T14:38:10.443669+0000","last_unstale":"2024-04-25T14:38:12.214480+0000","last_undegraded":"2024-04-25T14:38:12.214480+0000","last_fullsized":"2024-04-25T14:38:12.214480+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:02.302057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.421262+0000","last_change":"2024-04-25T14:38:09.545010+0000","last_active":"2024-04-25T14:38:11.421262+0000","last_peered":"2024-04-25T14:38:11.421262+0000","last_clean":"2024-04-25T14:38:11.421262+0000","last_became_active":"2024-04-25T14:38:09.544622+0000","last_became_peered":"2024-04-25T14:38:09.544622+0000","last_unstale":"2024-04-25T14:38:11.421262+0000","last_undegraded":"2024-04-25T14:38:11.421262+0000","last_fullsized":"2024-04-25T14:38:11.421262+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:51.200242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.421654+0000","last_change":"2024-04-25T14:38:10.446273+0000","last_active":"2024-04-25T14:38:11.421654+0000","last_peered":"2024-04-25T14:38:11.421654+0000","last_clean":"2024-04-25T14:38:11.421654+0000","last_became_active":"2024-04-25T14:38:10.445896+0000","last_became_peered":"2024-04-25T14:38:10.445896+0000","last_unstale":"2024-04-25T14:38:11.421654+0000","last_undegraded":"2024-04-25T14:38:11.421654+0000","last_fullsized":"2024-04-25T14:38:11.421654+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:25.018372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.421998+0000","last_change":"2024-04-25T14:38:09.555825+0000","last_active":"2024-04-25T14:38:11.421998+0000","last_peered":"2024-04-25T14:38:11.421998+0000","last_clean":"2024-04-25T14:38:11.421998+0000","last_became_active":"2024-04-25T14:38:09.555436+0000","last_became_peered":"2024-04-25T14:38:09.555436+0000","last_unstale":"2024-04-25T14:38:11.421998+0000","last_undegraded":"2024-04-25T14:38:11.421998+0000","last_fullsized":"2024-04-25T14:38:11.421998+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:45.097583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.508172+0000","last_change":"2024-04-25T14:38:10.432254+0000","last_active":"2024-04-25T14:38:11.508172+0000","last_peered":"2024-04-25T14:38:11.508172+0000","last_clean":"2024-04-25T14:38:11.508172+0000","last_became_active":"2024-04-25T14:38:10.431835+0000","last_became_peered":"2024-04-25T14:38:10.431835+0000","last_unstale":"2024-04-25T14:38:11.508172+0000","last_undegraded":"2024-04-25T14:38:11.508172+0000","last_fullsized":"2024-04-25T14:38:11.508172+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:23.089447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:15.024366+0000","last_change":"2024-04-25T14:38:09.550254+0000","last_active":"2024-04-25T14:38:15.024366+0000","last_peered":"2024-04-25T14:38:15.024366+0000","last_clean":"2024-04-25T14:38:15.024366+0000","last_became_active":"2024-04-25T14:38:09.549958+0000","last_became_peered":"2024-04-25T14:38:09.549958+0000","last_unstale":"2024-04-25T14:38:15.024366+0000","last_undegraded":"2024-04-25T14:38:15.024366+0000","last_fullsized":"2024-04-25T14:38:15.024366+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:56.161427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.458235+0000","last_change":"2024-04-25T14:38:10.425648+0000","last_active":"2024-04-25T14:38:11.458235+0000","last_peered":"2024-04-25T14:38:11.458235+0000","last_clean":"2024-04-25T14:38:11.458235+0000","last_became_active":"2024-04-25T14:38:10.425009+0000","last_became_peered":"2024-04-25T14:38:10.425009+0000","last_unstale":"2024-04-25T14:38:11.458235+0000","last_undegraded":"2024-04-25T14:38:11.458235+0000","last_fullsized":"2024-04-25T14:38:11.458235+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:46.696610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.540071+0000","last_change":"2024-04-25T14:38:09.544481+0000","last_active":"2024-04-25T14:38:11.540071+0000","last_peered":"2024-04-25T14:38:11.540071+0000","last_clean":"2024-04-25T14:38:11.540071+0000","last_became_active":"2024-04-25T14:38:09.544147+0000","last_became_peered":"2024-04-25T14:38:09.544147+0000","last_unstale":"2024-04-25T14:38:11.540071+0000","last_undegraded":"2024-04-25T14:38:11.540071+0000","last_fullsized":"2024-04-25T14:38:11.540071+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:35:22.971028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.454575+0000","last_change":"2024-04-25T14:38:10.426851+0000","last_active":"2024-04-25T14:38:11.454575+0000","last_peered":"2024-04-25T14:38:11.454575+0000","last_clean":"2024-04-25T14:38:11.454575+0000","last_became_active":"2024-04-25T14:38:10.426475+0000","last_became_peered":"2024-04-25T14:38:10.426475+0000","last_unstale":"2024-04-25T14:38:11.454575+0000","last_undegraded":"2024-04-25T14:38:11.454575+0000","last_fullsized":"2024-04-25T14:38:11.454575+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:28.213318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:16.643049+0000","last_change":"2024-04-25T14:38:09.546357+0000","last_active":"2024-04-25T14:38:16.643049+0000","last_peered":"2024-04-25T14:38:16.643049+0000","last_clean":"2024-04-25T14:38:16.643049+0000","last_became_active":"2024-04-25T14:38:09.545940+0000","last_became_peered":"2024-04-25T14:38:09.545940+0000","last_unstale":"2024-04-25T14:38:16.643049+0000","last_undegraded":"2024-04-25T14:38:16.643049+0000","last_fullsized":"2024-04-25T14:38:16.643049+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:28:37.807461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.540044+0000","last_change":"2024-04-25T14:38:10.425424+0000","last_active":"2024-04-25T14:38:11.540044+0000","last_peered":"2024-04-25T14:38:11.540044+0000","last_clean":"2024-04-25T14:38:11.540044+0000","last_became_active":"2024-04-25T14:38:10.425101+0000","last_became_peered":"2024-04-25T14:38:10.425101+0000","last_unstale":"2024-04-25T14:38:11.540044+0000","last_undegraded":"2024-04-25T14:38:11.540044+0000","last_fullsized":"2024-04-25T14:38:11.540044+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:52:29.871113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:15.027097+0000","last_change":"2024-04-25T14:38:09.552287+0000","last_active":"2024-04-25T14:38:15.027097+0000","last_peered":"2024-04-25T14:38:15.027097+0000","last_clean":"2024-04-25T14:38:15.027097+0000","last_became_active":"2024-04-25T14:38:09.551957+0000","last_became_peered":"2024-04-25T14:38:09.551957+0000","last_unstale":"2024-04-25T14:38:15.027097+0000","last_undegraded":"2024-04-25T14:38:15.027097+0000","last_fullsized":"2024-04-25T14:38:15.027097+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:22.586868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.454610+0000","last_change":"2024-04-25T14:38:10.443959+0000","last_active":"2024-04-25T14:38:11.454610+0000","last_peered":"2024-04-25T14:38:11.454610+0000","last_clean":"2024-04-25T14:38:11.454610+0000","last_became_active":"2024-04-25T14:38:10.443535+0000","last_became_peered":"2024-04-25T14:38:10.443535+0000","last_unstale":"2024-04-25T14:38:11.454610+0000","last_undegraded":"2024-04-25T14:38:11.454610+0000","last_fullsized":"2024-04-25T14:38:11.454610+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:59.972322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:15.028249+0000","last_change":"2024-04-25T14:38:09.552376+0000","last_active":"2024-04-25T14:38:15.028249+0000","last_peered":"2024-04-25T14:38:15.028249+0000","last_clean":"2024-04-25T14:38:15.028249+0000","last_became_active":"2024-04-25T14:38:09.548614+0000","last_became_peered":"2024-04-25T14:38:09.548614+0000","last_unstale":"2024-04-25T14:38:15.028249+0000","last_undegraded":"2024-04-25T14:38:15.028249+0000","last_fullsized":"2024-04-25T14:38:15.028249+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:27:27.133106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.420020+0000","last_change":"2024-04-25T14:38:10.434611+0000","last_active":"2024-04-25T14:38:11.420020+0000","last_peered":"2024-04-25T14:38:11.420020+0000","last_clean":"2024-04-25T14:38:11.420020+0000","last_became_active":"2024-04-25T14:38:10.434280+0000","last_became_peered":"2024-04-25T14:38:10.434280+0000","last_unstale":"2024-04-25T14:38:11.420020+0000","last_undegraded":"2024-04-25T14:38:11.420020+0000","last_fullsized":"2024-04-25T14:38:11.420020+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:12:32.275663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.991562+0000","last_change":"2024-04-25T14:38:09.539743+0000","last_active":"2024-04-25T14:38:11.991562+0000","last_peered":"2024-04-25T14:38:11.991562+0000","last_clean":"2024-04-25T14:38:11.991562+0000","last_became_active":"2024-04-25T14:38:09.539272+0000","last_became_peered":"2024-04-25T14:38:09.539272+0000","last_unstale":"2024-04-25T14:38:11.991562+0000","last_undegraded":"2024-04-25T14:38:11.991562+0000","last_fullsized":"2024-04-25T14:38:11.991562+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:29:23.159901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.421073+0000","last_change":"2024-04-25T14:38:10.444899+0000","last_active":"2024-04-25T14:38:11.421073+0000","last_peered":"2024-04-25T14:38:11.421073+0000","last_clean":"2024-04-25T14:38:11.421073+0000","last_became_active":"2024-04-25T14:38:10.444383+0000","last_became_peered":"2024-04-25T14:38:10.444383+0000","last_unstale":"2024-04-25T14:38:11.421073+0000","last_undegraded":"2024-04-25T14:38:11.421073+0000","last_fullsized":"2024-04-25T14:38:11.421073+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:21.351436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:14.007200+0000","last_change":"2024-04-25T14:38:09.550181+0000","last_active":"2024-04-25T14:38:14.007200+0000","last_peered":"2024-04-25T14:38:14.007200+0000","last_clean":"2024-04-25T14:38:14.007200+0000","last_became_active":"2024-04-25T14:38:09.549824+0000","last_became_peered":"2024-04-25T14:38:09.549824+0000","last_unstale":"2024-04-25T14:38:14.007200+0000","last_undegraded":"2024-04-25T14:38:14.007200+0000","last_fullsized":"2024-04-25T14:38:14.007200+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:59.220917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.458502+0000","last_change":"2024-04-25T14:38:10.421339+0000","last_active":"2024-04-25T14:38:11.458502+0000","last_peered":"2024-04-25T14:38:11.458502+0000","last_clean":"2024-04-25T14:38:11.458502+0000","last_became_active":"2024-04-25T14:38:10.421005+0000","last_became_peered":"2024-04-25T14:38:10.421005+0000","last_unstale":"2024-04-25T14:38:11.458502+0000","last_undegraded":"2024-04-25T14:38:11.458502+0000","last_fullsized":"2024-04-25T14:38:11.458502+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:51.949054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:14.004053+0000","last_change":"2024-04-25T14:38:09.548245+0000","last_active":"2024-04-25T14:38:14.004053+0000","last_peered":"2024-04-25T14:38:14.004053+0000","last_clean":"2024-04-25T14:38:14.004053+0000","last_became_active":"2024-04-25T14:38:09.547905+0000","last_became_peered":"2024-04-25T14:38:09.547905+0000","last_unstale":"2024-04-25T14:38:14.004053+0000","last_undegraded":"2024-04-25T14:38:14.004053+0000","last_fullsized":"2024-04-25T14:38:14.004053+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:56.798594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.508168+0000","last_change":"2024-04-25T14:38:10.431427+0000","last_active":"2024-04-25T14:38:11.508168+0000","last_peered":"2024-04-25T14:38:11.508168+0000","last_clean":"2024-04-25T14:38:11.508168+0000","last_became_active":"2024-04-25T14:38:10.428759+0000","last_became_peered":"2024-04-25T14:38:10.428759+0000","last_unstale":"2024-04-25T14:38:11.508168+0000","last_undegraded":"2024-04-25T14:38:11.508168+0000","last_fullsized":"2024-04-25T14:38:11.508168+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:15:27.341445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:14.006219+0000","last_change":"2024-04-25T14:38:09.556839+0000","last_active":"2024-04-25T14:38:14.006219+0000","last_peered":"2024-04-25T14:38:14.006219+0000","last_clean":"2024-04-25T14:38:14.006219+0000","last_became_active":"2024-04-25T14:38:09.556443+0000","last_became_peered":"2024-04-25T14:38:09.556443+0000","last_unstale":"2024-04-25T14:38:14.006219+0000","last_undegraded":"2024-04-25T14:38:14.006219+0000","last_fullsized":"2024-04-25T14:38:14.006219+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:46.621883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.458372+0000","last_change":"2024-04-25T14:38:10.414507+0000","last_active":"2024-04-25T14:38:11.458372+0000","last_peered":"2024-04-25T14:38:11.458372+0000","last_clean":"2024-04-25T14:38:11.458372+0000","last_became_active":"2024-04-25T14:38:10.414154+0000","last_became_peered":"2024-04-25T14:38:10.414154+0000","last_unstale":"2024-04-25T14:38:11.458372+0000","last_undegraded":"2024-04-25T14:38:11.458372+0000","last_fullsized":"2024-04-25T14:38:11.458372+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:12.208774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'8","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:14.004348+0000","last_change":"2024-04-25T14:38:09.549239+0000","last_active":"2024-04-25T14:38:14.004348+0000","last_peered":"2024-04-25T14:38:14.004348+0000","last_clean":"2024-04-25T14:38:14.004348+0000","last_became_active":"2024-04-25T14:38:09.548888+0000","last_became_peered":"2024-04-25T14:38:09.548888+0000","last_unstale":"2024-04-25T14:38:14.004348+0000","last_undegraded":"2024-04-25T14:38:14.004348+0000","last_fullsized":"2024-04-25T14:38:14.004348+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:36:46.536908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.540092+0000","last_change":"2024-04-25T14:38:10.417510+0000","last_active":"2024-04-25T14:38:11.540092+0000","last_peered":"2024-04-25T14:38:11.540092+0000","last_clean":"2024-04-25T14:38:11.540092+0000","last_became_active":"2024-04-25T14:38:10.417079+0000","last_became_peered":"2024-04-25T14:38:10.417079+0000","last_unstale":"2024-04-25T14:38:11.540092+0000","last_undegraded":"2024-04-25T14:38:11.540092+0000","last_fullsized":"2024-04-25T14:38:11.540092+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:35.213199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:15.030513+0000","last_change":"2024-04-25T14:38:09.563650+0000","last_active":"2024-04-25T14:38:15.030513+0000","last_peered":"2024-04-25T14:38:15.030513+0000","last_clean":"2024-04-25T14:38:15.030513+0000","last_became_active":"2024-04-25T14:38:09.563347+0000","last_became_peered":"2024-04-25T14:38:09.563347+0000","last_unstale":"2024-04-25T14:38:15.030513+0000","last_undegraded":"2024-04-25T14:38:15.030513+0000","last_fullsized":"2024-04-25T14:38:15.030513+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:16.793191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.759439+0000","last_change":"2024-04-25T14:38:10.447504+0000","last_active":"2024-04-25T14:38:11.759439+0000","last_peered":"2024-04-25T14:38:11.759439+0000","last_clean":"2024-04-25T14:38:11.759439+0000","last_became_active":"2024-04-25T14:38:10.447157+0000","last_became_peered":"2024-04-25T14:38:10.447157+0000","last_unstale":"2024-04-25T14:38:11.759439+0000","last_undegraded":"2024-04-25T14:38:11.759439+0000","last_fullsized":"2024-04-25T14:38:11.759439+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:38.940320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:14.007260+0000","last_change":"2024-04-25T14:38:09.545214+0000","last_active":"2024-04-25T14:38:14.007260+0000","last_peered":"2024-04-25T14:38:14.007260+0000","last_clean":"2024-04-25T14:38:14.007260+0000","last_became_active":"2024-04-25T14:38:09.544903+0000","last_became_peered":"2024-04-25T14:38:09.544903+0000","last_unstale":"2024-04-25T14:38:14.007260+0000","last_undegraded":"2024-04-25T14:38:14.007260+0000","last_fullsized":"2024-04-25T14:38:14.007260+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:58:15.817036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.214622+0000","last_change":"2024-04-25T14:38:10.432017+0000","last_active":"2024-04-25T14:38:12.214622+0000","last_peered":"2024-04-25T14:38:12.214622+0000","last_clean":"2024-04-25T14:38:12.214622+0000","last_became_active":"2024-04-25T14:38:10.431596+0000","last_became_peered":"2024-04-25T14:38:10.431596+0000","last_unstale":"2024-04-25T14:38:12.214622+0000","last_undegraded":"2024-04-25T14:38:12.214622+0000","last_fullsized":"2024-04-25T14:38:12.214622+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:30.653642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.420027+0000","last_change":"2024-04-25T14:38:09.545477+0000","last_active":"2024-04-25T14:38:11.420027+0000","last_peered":"2024-04-25T14:38:11.420027+0000","last_clean":"2024-04-25T14:38:11.420027+0000","last_became_active":"2024-04-25T14:38:09.545093+0000","last_became_peered":"2024-04-25T14:38:09.545093+0000","last_unstale":"2024-04-25T14:38:11.420027+0000","last_undegraded":"2024-04-25T14:38:11.420027+0000","last_fullsized":"2024-04-25T14:38:11.420027+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:56.589231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.213871+0000","last_change":"2024-04-25T14:38:10.418914+0000","last_active":"2024-04-25T14:38:12.213871+0000","last_peered":"2024-04-25T14:38:12.213871+0000","last_clean":"2024-04-25T14:38:12.213871+0000","last_became_active":"2024-04-25T14:38:10.418600+0000","last_became_peered":"2024-04-25T14:38:10.418600+0000","last_unstale":"2024-04-25T14:38:12.213871+0000","last_undegraded":"2024-04-25T14:38:12.213871+0000","last_fullsized":"2024-04-25T14:38:12.213871+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:31.324035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:15.029690+0000","last_change":"2024-04-25T14:38:09.556574+0000","last_active":"2024-04-25T14:38:15.029690+0000","last_peered":"2024-04-25T14:38:15.029690+0000","last_clean":"2024-04-25T14:38:15.029690+0000","last_became_active":"2024-04-25T14:38:09.556071+0000","last_became_peered":"2024-04-25T14:38:09.556071+0000","last_unstale":"2024-04-25T14:38:15.029690+0000","last_undegraded":"2024-04-25T14:38:15.029690+0000","last_fullsized":"2024-04-25T14:38:15.029690+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:05.424593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.454366+0000","last_change":"2024-04-25T14:38:10.424591+0000","last_active":"2024-04-25T14:38:11.454366+0000","last_peered":"2024-04-25T14:38:11.454366+0000","last_clean":"2024-04-25T14:38:11.454366+0000","last_became_active":"2024-04-25T14:38:10.424258+0000","last_became_peered":"2024-04-25T14:38:10.424258+0000","last_unstale":"2024-04-25T14:38:11.454366+0000","last_undegraded":"2024-04-25T14:38:11.454366+0000","last_fullsized":"2024-04-25T14:38:11.454366+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:22.753595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:15.640707+0000","last_change":"2024-04-25T14:38:09.546073+0000","last_active":"2024-04-25T14:38:15.640707+0000","last_peered":"2024-04-25T14:38:15.640707+0000","last_clean":"2024-04-25T14:38:15.640707+0000","last_became_active":"2024-04-25T14:38:09.545783+0000","last_became_peered":"2024-04-25T14:38:09.545783+0000","last_unstale":"2024-04-25T14:38:15.640707+0000","last_undegraded":"2024-04-25T14:38:15.640707+0000","last_fullsized":"2024-04-25T14:38:15.640707+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:42.537431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.214028+0000","last_change":"2024-04-25T14:38:10.417230+0000","last_active":"2024-04-25T14:38:12.214028+0000","last_peered":"2024-04-25T14:38:12.214028+0000","last_clean":"2024-04-25T14:38:12.214028+0000","last_became_active":"2024-04-25T14:38:10.416891+0000","last_became_peered":"2024-04-25T14:38:10.416891+0000","last_unstale":"2024-04-25T14:38:12.214028+0000","last_undegraded":"2024-04-25T14:38:12.214028+0000","last_fullsized":"2024-04-25T14:38:12.214028+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:10:13.434518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.421556+0000","last_change":"2024-04-25T14:38:09.553869+0000","last_active":"2024-04-25T14:38:11.421556+0000","last_peered":"2024-04-25T14:38:11.421556+0000","last_clean":"2024-04-25T14:38:11.421556+0000","last_became_active":"2024-04-25T14:38:09.552625+0000","last_became_peered":"2024-04-25T14:38:09.552625+0000","last_unstale":"2024-04-25T14:38:11.421556+0000","last_undegraded":"2024-04-25T14:38:11.421556+0000","last_fullsized":"2024-04-25T14:38:11.421556+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:16.751312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.214817+0000","last_change":"2024-04-25T14:38:10.446846+0000","last_active":"2024-04-25T14:38:12.214817+0000","last_peered":"2024-04-25T14:38:12.214817+0000","last_clean":"2024-04-25T14:38:12.214817+0000","last_became_active":"2024-04-25T14:38:10.446498+0000","last_became_peered":"2024-04-25T14:38:10.446498+0000","last_unstale":"2024-04-25T14:38:12.214817+0000","last_undegraded":"2024-04-25T14:38:12.214817+0000","last_fullsized":"2024-04-25T14:38:12.214817+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:14:37.179198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.214680+0000","last_change":"2024-04-25T14:38:09.544283+0000","last_active":"2024-04-25T14:38:12.214680+0000","last_peered":"2024-04-25T14:38:12.214680+0000","last_clean":"2024-04-25T14:38:12.214680+0000","last_became_active":"2024-04-25T14:38:09.543135+0000","last_became_peered":"2024-04-25T14:38:09.543135+0000","last_unstale":"2024-04-25T14:38:12.214680+0000","last_undegraded":"2024-04-25T14:38:12.214680+0000","last_fullsized":"2024-04-25T14:38:12.214680+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:05.977062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.214751+0000","last_change":"2024-04-25T14:38:10.432863+0000","last_active":"2024-04-25T14:38:12.214751+0000","last_peered":"2024-04-25T14:38:12.214751+0000","last_clean":"2024-04-25T14:38:12.214751+0000","last_became_active":"2024-04-25T14:38:10.432493+0000","last_became_peered":"2024-04-25T14:38:10.432493+0000","last_unstale":"2024-04-25T14:38:12.214751+0000","last_undegraded":"2024-04-25T14:38:12.214751+0000","last_fullsized":"2024-04-25T14:38:12.214751+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:25:53.774764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.454423+0000","last_change":"2024-04-25T14:38:10.422265+0000","last_active":"2024-04-25T14:38:11.454423+0000","last_peered":"2024-04-25T14:38:11.454423+0000","last_clean":"2024-04-25T14:38:11.454423+0000","last_became_active":"2024-04-25T14:38:10.421952+0000","last_became_peered":"2024-04-25T14:38:10.421952+0000","last_unstale":"2024-04-25T14:38:11.454423+0000","last_undegraded":"2024-04-25T14:38:11.454423+0000","last_fullsized":"2024-04-25T14:38:11.454423+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:39.252185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:15.030691+0000","last_change":"2024-04-25T14:38:09.554305+0000","last_active":"2024-04-25T14:38:15.030691+0000","last_peered":"2024-04-25T14:38:15.030691+0000","last_clean":"2024-04-25T14:38:15.030691+0000","last_became_active":"2024-04-25T14:38:09.553984+0000","last_became_peered":"2024-04-25T14:38:09.553984+0000","last_unstale":"2024-04-25T14:38:15.030691+0000","last_undegraded":"2024-04-25T14:38:15.030691+0000","last_fullsized":"2024-04-25T14:38:15.030691+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:41.580867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.420161+0000","last_change":"2024-04-25T14:38:10.448290+0000","last_active":"2024-04-25T14:38:11.420161+0000","last_peered":"2024-04-25T14:38:11.420161+0000","last_clean":"2024-04-25T14:38:11.420161+0000","last_became_active":"2024-04-25T14:38:10.446147+0000","last_became_peered":"2024-04-25T14:38:10.446147+0000","last_unstale":"2024-04-25T14:38:11.420161+0000","last_undegraded":"2024-04-25T14:38:11.420161+0000","last_fullsized":"2024-04-25T14:38:11.420161+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:12.775551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:14.006895+0000","last_change":"2024-04-25T14:38:09.545209+0000","last_active":"2024-04-25T14:38:14.006895+0000","last_peered":"2024-04-25T14:38:14.006895+0000","last_clean":"2024-04-25T14:38:14.006895+0000","last_became_active":"2024-04-25T14:38:09.544861+0000","last_became_peered":"2024-04-25T14:38:09.544861+0000","last_unstale":"2024-04-25T14:38:14.006895+0000","last_undegraded":"2024-04-25T14:38:14.006895+0000","last_fullsized":"2024-04-25T14:38:14.006895+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:25.829162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":349,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.213852+0000","last_change":"2024-04-25T14:36:48.250081+0000","last_active":"2024-04-25T14:38:12.213852+0000","last_peered":"2024-04-25T14:38:12.213852+0000","last_clean":"2024-04-25T14:38:12.213852+0000","last_became_active":"2024-04-25T14:36:48.249520+0000","last_became_peered":"2024-04-25T14:36:48.249520+0000","last_unstale":"2024-04-25T14:38:12.213852+0000","last_undegraded":"2024-04-25T14:38:12.213852+0000","last_fullsized":"2024-04-25T14:38:12.213852+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:33:03.061016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:33:03.061016+0000","last_clean_scrub_stamp":"2024-04-25T14:33:03.061016+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:04:06.653326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.759241+0000","last_change":"2024-04-25T14:38:10.422966+0000","last_active":"2024-04-25T14:38:11.759241+0000","last_peered":"2024-04-25T14:38:11.759241+0000","last_clean":"2024-04-25T14:38:11.759241+0000","last_became_active":"2024-04-25T14:38:10.422568+0000","last_became_peered":"2024-04-25T14:38:10.422568+0000","last_unstale":"2024-04-25T14:38:11.759241+0000","last_undegraded":"2024-04-25T14:38:11.759241+0000","last_fullsized":"2024-04-25T14:38:11.759241+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:17.405074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:15.025689+0000","last_change":"2024-04-25T14:38:09.542051+0000","last_active":"2024-04-25T14:38:15.025689+0000","last_peered":"2024-04-25T14:38:15.025689+0000","last_clean":"2024-04-25T14:38:15.025689+0000","last_became_active":"2024-04-25T14:38:09.541564+0000","last_became_peered":"2024-04-25T14:38:09.541564+0000","last_unstale":"2024-04-25T14:38:15.025689+0000","last_undegraded":"2024-04-25T14:38:15.025689+0000","last_fullsized":"2024-04-25T14:38:15.025689+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:44:30.122416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:14.007421+0000","last_change":"2024-04-25T14:38:09.560357+0000","last_active":"2024-04-25T14:38:14.007421+0000","last_peered":"2024-04-25T14:38:14.007421+0000","last_clean":"2024-04-25T14:38:14.007421+0000","last_became_active":"2024-04-25T14:38:09.560046+0000","last_became_peered":"2024-04-25T14:38:09.560046+0000","last_unstale":"2024-04-25T14:38:14.007421+0000","last_undegraded":"2024-04-25T14:38:14.007421+0000","last_fullsized":"2024-04-25T14:38:14.007421+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:22.692116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.759793+0000","last_change":"2024-04-25T14:38:10.429554+0000","last_active":"2024-04-25T14:38:11.759793+0000","last_peered":"2024-04-25T14:38:11.759793+0000","last_clean":"2024-04-25T14:38:11.759793+0000","last_became_active":"2024-04-25T14:38:10.428944+0000","last_became_peered":"2024-04-25T14:38:10.428944+0000","last_unstale":"2024-04-25T14:38:11.759793+0000","last_undegraded":"2024-04-25T14:38:11.759793+0000","last_fullsized":"2024-04-25T14:38:11.759793+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:38.013028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.421388+0000","last_change":"2024-04-25T14:38:09.564849+0000","last_active":"2024-04-25T14:38:11.421388+0000","last_peered":"2024-04-25T14:38:11.421388+0000","last_clean":"2024-04-25T14:38:11.421388+0000","last_became_active":"2024-04-25T14:38:09.564354+0000","last_became_peered":"2024-04-25T14:38:09.564354+0000","last_unstale":"2024-04-25T14:38:11.421388+0000","last_undegraded":"2024-04-25T14:38:11.421388+0000","last_fullsized":"2024-04-25T14:38:11.421388+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:44.308366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.421519+0000","last_change":"2024-04-25T14:38:10.438178+0000","last_active":"2024-04-25T14:38:11.421519+0000","last_peered":"2024-04-25T14:38:11.421519+0000","last_clean":"2024-04-25T14:38:11.421519+0000","last_became_active":"2024-04-25T14:38:10.429936+0000","last_became_peered":"2024-04-25T14:38:10.429936+0000","last_unstale":"2024-04-25T14:38:11.421519+0000","last_undegraded":"2024-04-25T14:38:11.421519+0000","last_fullsized":"2024-04-25T14:38:11.421519+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:33.598747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:14.006393+0000","last_change":"2024-04-25T14:38:09.547680+0000","last_active":"2024-04-25T14:38:14.006393+0000","last_peered":"2024-04-25T14:38:14.006393+0000","last_clean":"2024-04-25T14:38:14.006393+0000","last_became_active":"2024-04-25T14:38:09.547337+0000","last_became_peered":"2024-04-25T14:38:09.547337+0000","last_unstale":"2024-04-25T14:38:14.006393+0000","last_undegraded":"2024-04-25T14:38:14.006393+0000","last_fullsized":"2024-04-25T14:38:14.006393+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:32.754439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.507943+0000","last_change":"2024-04-25T14:38:10.447115+0000","last_active":"2024-04-25T14:38:11.507943+0000","last_peered":"2024-04-25T14:38:11.507943+0000","last_clean":"2024-04-25T14:38:11.507943+0000","last_became_active":"2024-04-25T14:38:10.446681+0000","last_became_peered":"2024-04-25T14:38:10.446681+0000","last_unstale":"2024-04-25T14:38:11.507943+0000","last_undegraded":"2024-04-25T14:38:11.507943+0000","last_fullsized":"2024-04-25T14:38:11.507943+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:02:58.942053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:14.004053+0000","last_change":"2024-04-25T14:38:09.551523+0000","last_active":"2024-04-25T14:38:14.004053+0000","last_peered":"2024-04-25T14:38:14.004053+0000","last_clean":"2024-04-25T14:38:14.004053+0000","last_became_active":"2024-04-25T14:38:09.551213+0000","last_became_peered":"2024-04-25T14:38:09.551213+0000","last_unstale":"2024-04-25T14:38:14.004053+0000","last_undegraded":"2024-04-25T14:38:14.004053+0000","last_fullsized":"2024-04-25T14:38:14.004053+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:48.948571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.539827+0000","last_change":"2024-04-25T14:38:10.420032+0000","last_active":"2024-04-25T14:38:11.539827+0000","last_peered":"2024-04-25T14:38:11.539827+0000","last_clean":"2024-04-25T14:38:11.539827+0000","last_became_active":"2024-04-25T14:38:10.418349+0000","last_became_peered":"2024-04-25T14:38:10.418349+0000","last_unstale":"2024-04-25T14:38:11.539827+0000","last_undegraded":"2024-04-25T14:38:11.539827+0000","last_fullsized":"2024-04-25T14:38:11.539827+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:38:55.669414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:14.013121+0000","last_change":"2024-04-25T14:38:09.567097+0000","last_active":"2024-04-25T14:38:14.013121+0000","last_peered":"2024-04-25T14:38:14.013121+0000","last_clean":"2024-04-25T14:38:14.013121+0000","last_became_active":"2024-04-25T14:38:09.566852+0000","last_became_peered":"2024-04-25T14:38:09.566852+0000","last_unstale":"2024-04-25T14:38:14.013121+0000","last_undegraded":"2024-04-25T14:38:14.013121+0000","last_fullsized":"2024-04-25T14:38:14.013121+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:41:55.773005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.213818+0000","last_change":"2024-04-25T14:38:10.432691+0000","last_active":"2024-04-25T14:38:12.213818+0000","last_peered":"2024-04-25T14:38:12.213818+0000","last_clean":"2024-04-25T14:38:12.213818+0000","last_became_active":"2024-04-25T14:38:10.432321+0000","last_became_peered":"2024-04-25T14:38:10.432321+0000","last_unstale":"2024-04-25T14:38:12.213818+0000","last_undegraded":"2024-04-25T14:38:12.213818+0000","last_fullsized":"2024-04-25T14:38:12.213818+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:47:58.026331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.214168+0000","last_change":"2024-04-25T14:38:09.563692+0000","last_active":"2024-04-25T14:38:12.214168+0000","last_peered":"2024-04-25T14:38:12.214168+0000","last_clean":"2024-04-25T14:38:12.214168+0000","last_became_active":"2024-04-25T14:38:09.563345+0000","last_became_peered":"2024-04-25T14:38:09.563345+0000","last_unstale":"2024-04-25T14:38:12.214168+0000","last_undegraded":"2024-04-25T14:38:12.214168+0000","last_fullsized":"2024-04-25T14:38:12.214168+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:43.221429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.420283+0000","last_change":"2024-04-25T14:38:10.442519+0000","last_active":"2024-04-25T14:38:11.420283+0000","last_peered":"2024-04-25T14:38:11.420283+0000","last_clean":"2024-04-25T14:38:11.420283+0000","last_became_active":"2024-04-25T14:38:10.439045+0000","last_became_peered":"2024-04-25T14:38:10.439045+0000","last_unstale":"2024-04-25T14:38:11.420283+0000","last_undegraded":"2024-04-25T14:38:11.420283+0000","last_fullsized":"2024-04-25T14:38:11.420283+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:42.425106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:14.005037+0000","last_change":"2024-04-25T14:38:09.550363+0000","last_active":"2024-04-25T14:38:14.005037+0000","last_peered":"2024-04-25T14:38:14.005037+0000","last_clean":"2024-04-25T14:38:14.005037+0000","last_became_active":"2024-04-25T14:38:09.550044+0000","last_became_peered":"2024-04-25T14:38:09.550044+0000","last_unstale":"2024-04-25T14:38:14.005037+0000","last_undegraded":"2024-04-25T14:38:14.005037+0000","last_fullsized":"2024-04-25T14:38:14.005037+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:25.573353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.214422+0000","last_change":"2024-04-25T14:38:10.428736+0000","last_active":"2024-04-25T14:38:12.214422+0000","last_peered":"2024-04-25T14:38:12.214422+0000","last_clean":"2024-04-25T14:38:12.214422+0000","last_became_active":"2024-04-25T14:38:10.428410+0000","last_became_peered":"2024-04-25T14:38:10.428410+0000","last_unstale":"2024-04-25T14:38:12.214422+0000","last_undegraded":"2024-04-25T14:38:12.214422+0000","last_fullsized":"2024-04-25T14:38:12.214422+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:02:51.474220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:15.029278+0000","last_change":"2024-04-25T14:38:09.552800+0000","last_active":"2024-04-25T14:38:15.029278+0000","last_peered":"2024-04-25T14:38:15.029278+0000","last_clean":"2024-04-25T14:38:15.029278+0000","last_became_active":"2024-04-25T14:38:09.552460+0000","last_became_peered":"2024-04-25T14:38:09.552460+0000","last_unstale":"2024-04-25T14:38:15.029278+0000","last_undegraded":"2024-04-25T14:38:15.029278+0000","last_fullsized":"2024-04-25T14:38:15.029278+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:00.800770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.214494+0000","last_change":"2024-04-25T14:38:10.424359+0000","last_active":"2024-04-25T14:38:12.214494+0000","last_peered":"2024-04-25T14:38:12.214494+0000","last_clean":"2024-04-25T14:38:12.214494+0000","last_became_active":"2024-04-25T14:38:10.424020+0000","last_became_peered":"2024-04-25T14:38:10.424020+0000","last_unstale":"2024-04-25T14:38:12.214494+0000","last_undegraded":"2024-04-25T14:38:12.214494+0000","last_fullsized":"2024-04-25T14:38:12.214494+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:14.700772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.991648+0000","last_change":"2024-04-25T14:38:09.545637+0000","last_active":"2024-04-25T14:38:11.991648+0000","last_peered":"2024-04-25T14:38:11.991648+0000","last_clean":"2024-04-25T14:38:11.991648+0000","last_became_active":"2024-04-25T14:38:09.545319+0000","last_became_peered":"2024-04-25T14:38:09.545319+0000","last_unstale":"2024-04-25T14:38:11.991648+0000","last_undegraded":"2024-04-25T14:38:11.991648+0000","last_fullsized":"2024-04-25T14:38:11.991648+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:02.284715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.214197+0000","last_change":"2024-04-25T14:38:10.446928+0000","last_active":"2024-04-25T14:38:12.214197+0000","last_peered":"2024-04-25T14:38:12.214197+0000","last_clean":"2024-04-25T14:38:12.214197+0000","last_became_active":"2024-04-25T14:38:10.446518+0000","last_became_peered":"2024-04-25T14:38:10.446518+0000","last_unstale":"2024-04-25T14:38:12.214197+0000","last_undegraded":"2024-04-25T14:38:12.214197+0000","last_fullsized":"2024-04-25T14:38:12.214197+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:40.769059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.760049+0000","last_change":"2024-04-25T14:38:09.557381+0000","last_active":"2024-04-25T14:38:11.760049+0000","last_peered":"2024-04-25T14:38:11.760049+0000","last_clean":"2024-04-25T14:38:11.760049+0000","last_became_active":"2024-04-25T14:38:09.556504+0000","last_became_peered":"2024-04-25T14:38:09.556504+0000","last_unstale":"2024-04-25T14:38:11.760049+0000","last_undegraded":"2024-04-25T14:38:11.760049+0000","last_fullsized":"2024-04-25T14:38:11.760049+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:01:02.761249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.214467+0000","last_change":"2024-04-25T14:38:10.418104+0000","last_active":"2024-04-25T14:38:12.214467+0000","last_peered":"2024-04-25T14:38:12.214467+0000","last_clean":"2024-04-25T14:38:12.214467+0000","last_became_active":"2024-04-25T14:38:10.417774+0000","last_became_peered":"2024-04-25T14:38:10.417774+0000","last_unstale":"2024-04-25T14:38:12.214467+0000","last_undegraded":"2024-04-25T14:38:12.214467+0000","last_fullsized":"2024-04-25T14:38:12.214467+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:32:44.229899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.213939+0000","last_change":"2024-04-25T14:38:10.423546+0000","last_active":"2024-04-25T14:38:12.213939+0000","last_peered":"2024-04-25T14:38:12.213939+0000","last_clean":"2024-04-25T14:38:12.213939+0000","last_became_active":"2024-04-25T14:38:10.423216+0000","last_became_peered":"2024-04-25T14:38:10.423216+0000","last_unstale":"2024-04-25T14:38:12.213939+0000","last_undegraded":"2024-04-25T14:38:12.213939+0000","last_fullsized":"2024-04-25T14:38:12.213939+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:58.640669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.214113+0000","last_change":"2024-04-25T14:38:10.422504+0000","last_active":"2024-04-25T14:38:12.214113+0000","last_peered":"2024-04-25T14:38:12.214113+0000","last_clean":"2024-04-25T14:38:12.214113+0000","last_became_active":"2024-04-25T14:38:10.422095+0000","last_became_peered":"2024-04-25T14:38:10.422095+0000","last_unstale":"2024-04-25T14:38:12.214113+0000","last_undegraded":"2024-04-25T14:38:12.214113+0000","last_fullsized":"2024-04-25T14:38:12.214113+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:25.795962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.214150+0000","last_change":"2024-04-25T14:38:10.447446+0000","last_active":"2024-04-25T14:38:12.214150+0000","last_peered":"2024-04-25T14:38:12.214150+0000","last_clean":"2024-04-25T14:38:12.214150+0000","last_became_active":"2024-04-25T14:38:10.447056+0000","last_became_peered":"2024-04-25T14:38:10.447056+0000","last_unstale":"2024-04-25T14:38:12.214150+0000","last_undegraded":"2024-04-25T14:38:12.214150+0000","last_fullsized":"2024-04-25T14:38:12.214150+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:40.462340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.507677+0000","last_change":"2024-04-25T14:38:10.433444+0000","last_active":"2024-04-25T14:38:11.507677+0000","last_peered":"2024-04-25T14:38:11.507677+0000","last_clean":"2024-04-25T14:38:11.507677+0000","last_became_active":"2024-04-25T14:38:10.433153+0000","last_became_peered":"2024-04-25T14:38:10.433153+0000","last_unstale":"2024-04-25T14:38:11.507677+0000","last_undegraded":"2024-04-25T14:38:11.507677+0000","last_fullsized":"2024-04-25T14:38:11.507677+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:10.722650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.759790+0000","last_change":"2024-04-25T14:38:10.438322+0000","last_active":"2024-04-25T14:38:11.759790+0000","last_peered":"2024-04-25T14:38:11.759790+0000","last_clean":"2024-04-25T14:38:11.759790+0000","last_became_active":"2024-04-25T14:38:10.436103+0000","last_became_peered":"2024-04-25T14:38:10.436103+0000","last_unstale":"2024-04-25T14:38:11.759790+0000","last_undegraded":"2024-04-25T14:38:11.759790+0000","last_fullsized":"2024-04-25T14:38:11.759790+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:16.790202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.214092+0000","last_change":"2024-04-25T14:38:10.419710+0000","last_active":"2024-04-25T14:38:12.214092+0000","last_peered":"2024-04-25T14:38:12.214092+0000","last_clean":"2024-04-25T14:38:12.214092+0000","last_became_active":"2024-04-25T14:38:10.419406+0000","last_became_peered":"2024-04-25T14:38:10.419406+0000","last_unstale":"2024-04-25T14:38:12.214092+0000","last_undegraded":"2024-04-25T14:38:12.214092+0000","last_fullsized":"2024-04-25T14:38:12.214092+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:37.964698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.214679+0000","last_change":"2024-04-25T14:38:10.429510+0000","last_active":"2024-04-25T14:38:12.214679+0000","last_peered":"2024-04-25T14:38:12.214679+0000","last_clean":"2024-04-25T14:38:12.214679+0000","last_became_active":"2024-04-25T14:38:10.429219+0000","last_became_peered":"2024-04-25T14:38:10.429219+0000","last_unstale":"2024-04-25T14:38:12.214679+0000","last_undegraded":"2024-04-25T14:38:12.214679+0000","last_fullsized":"2024-04-25T14:38:12.214679+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:52:46.673126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.421361+0000","last_change":"2024-04-25T14:38:10.441487+0000","last_active":"2024-04-25T14:38:11.421361+0000","last_peered":"2024-04-25T14:38:11.421361+0000","last_clean":"2024-04-25T14:38:11.421361+0000","last_became_active":"2024-04-25T14:38:10.441111+0000","last_became_peered":"2024-04-25T14:38:10.441111+0000","last_unstale":"2024-04-25T14:38:11.421361+0000","last_undegraded":"2024-04-25T14:38:11.421361+0000","last_fullsized":"2024-04-25T14:38:11.421361+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:59.842687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.454563+0000","last_change":"2024-04-25T14:38:10.436511+0000","last_active":"2024-04-25T14:38:11.454563+0000","last_peered":"2024-04-25T14:38:11.454563+0000","last_clean":"2024-04-25T14:38:11.454563+0000","last_became_active":"2024-04-25T14:38:10.436171+0000","last_became_peered":"2024-04-25T14:38:10.436171+0000","last_unstale":"2024-04-25T14:38:11.454563+0000","last_undegraded":"2024-04-25T14:38:11.454563+0000","last_fullsized":"2024-04-25T14:38:11.454563+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:27:19.605907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.420047+0000","last_change":"2024-04-25T14:38:10.433624+0000","last_active":"2024-04-25T14:38:11.420047+0000","last_peered":"2024-04-25T14:38:11.420047+0000","last_clean":"2024-04-25T14:38:11.420047+0000","last_became_active":"2024-04-25T14:38:10.433258+0000","last_became_peered":"2024-04-25T14:38:10.433258+0000","last_unstale":"2024-04-25T14:38:11.420047+0000","last_undegraded":"2024-04-25T14:38:11.420047+0000","last_fullsized":"2024-04-25T14:38:11.420047+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:53.563110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.458280+0000","last_change":"2024-04-25T14:38:10.446924+0000","last_active":"2024-04-25T14:38:11.458280+0000","last_peered":"2024-04-25T14:38:11.458280+0000","last_clean":"2024-04-25T14:38:11.458280+0000","last_became_active":"2024-04-25T14:38:10.446553+0000","last_became_peered":"2024-04-25T14:38:10.446553+0000","last_unstale":"2024-04-25T14:38:11.458280+0000","last_undegraded":"2024-04-25T14:38:11.458280+0000","last_fullsized":"2024-04-25T14:38:11.458280+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:00.203796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.759908+0000","last_change":"2024-04-25T14:38:10.429952+0000","last_active":"2024-04-25T14:38:11.759908+0000","last_peered":"2024-04-25T14:38:11.759908+0000","last_clean":"2024-04-25T14:38:11.759908+0000","last_became_active":"2024-04-25T14:38:10.429626+0000","last_became_peered":"2024-04-25T14:38:10.429626+0000","last_unstale":"2024-04-25T14:38:11.759908+0000","last_undegraded":"2024-04-25T14:38:11.759908+0000","last_fullsized":"2024-04-25T14:38:11.759908+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:07.287113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.214245+0000","last_change":"2024-04-25T14:38:10.418292+0000","last_active":"2024-04-25T14:38:12.214245+0000","last_peered":"2024-04-25T14:38:12.214245+0000","last_clean":"2024-04-25T14:38:12.214245+0000","last_became_active":"2024-04-25T14:38:10.418005+0000","last_became_peered":"2024-04-25T14:38:10.418005+0000","last_unstale":"2024-04-25T14:38:12.214245+0000","last_undegraded":"2024-04-25T14:38:12.214245+0000","last_fullsized":"2024-04-25T14:38:12.214245+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:25.486993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4991,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":15,"num_read_kb":13,"num_write":114,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":249856,"data_stored":35075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285584,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":452438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547220,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1585152,"data_stored":1045414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710745,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":452438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972385,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":452594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266724,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28532,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716620,"statfs":{"total":95995035648,"available":95965818880,"internally_reserved":0,"allocated":1552384,"data_stored":1043428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430249,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":451946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":453468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":974848,"data_stored":453334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411394,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":987136,"data_stored":461809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149755,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":452140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888119,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":950272,"data_stored":451878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691889,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32708,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712444,"statfs":{"total":95995035648,"available":95961542656,"internally_reserved":0,"allocated":1568768,"data_stored":1043700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":22636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:38:19.653 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:38:19.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.909+0000 7f365f000640 1 -- 172.21.15.53:0/3104393696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f36581039a0 msgr2=0x7f365810a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:19.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.909+0000 7f365f000640 1 --2- 172.21.15.53:0/3104393696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f36581039a0 0x7f365810a230 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3650009ca0 tx=0x7f365002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:38:19.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.909+0000 7f365f000640 1 -- 172.21.15.53:0/3104393696 shutdown_connections 2024-04-25T14:38:19.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.909+0000 7f365f000640 1 --2- 172.21.15.53:0/3104393696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f36581039a0 0x7f365810a230 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:19.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.909+0000 7f365f000640 1 --2- 172.21.15.53:0/3104393696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3658102fe0 0x7f3658103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:19.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.909+0000 7f365f000640 1 --2- 172.21.15.53:0/3104393696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3658101df0 0x7f3658102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:19.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.909+0000 7f365f000640 1 -- 172.21.15.53:0/3104393696 >> 172.21.15.53:0/3104393696 conn(0x7f36580fd360 msgr2=0x7f36580ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:19.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.910+0000 7f365f000640 1 -- 172.21.15.53:0/3104393696 shutdown_connections 2024-04-25T14:38:19.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.910+0000 7f365f000640 1 -- 172.21.15.53:0/3104393696 wait complete. 2024-04-25T14:38:19.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.910+0000 7f365f000640 1 Processor -- start 2024-04-25T14:38:19.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.911+0000 7f365f000640 1 -- start start 2024-04-25T14:38:19.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.911+0000 7f365f000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3658101df0 0x7f3658075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:19.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.911+0000 7f365f000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3658102fe0 0x7f3658074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:19.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.911+0000 7f365f000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36581039a0 0x7f3658074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:19.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.911+0000 7f365f000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f365810d920 con 0x7f36581039a0 2024-04-25T14:38:19.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.911+0000 7f365f000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f365810d7a0 con 0x7f3658101df0 2024-04-25T14:38:19.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.911+0000 7f365f000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f365810daa0 con 0x7f3658102fe0 2024-04-25T14:38:19.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.911+0000 7f365dc00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3658101df0 0x7f3658075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:19.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.911+0000 7f365dc00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3658101df0 0x7f3658075d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.53:38446/0 (socket says 172.21.15.53:38446) 2024-04-25T14:38:19.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.911+0000 7f365dc00640 1 -- 172.21.15.53:0/1758888260 learned_addr learned my addr 172.21.15.53:0/1758888260 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:38:19.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.911+0000 7f365e600640 1 --2- 172.21.15.53:0/1758888260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36581039a0 0x7f3658074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:19.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.911+0000 7f365d200640 1 --2- 172.21.15.53:0/1758888260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3658102fe0 0x7f3658074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:19.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.912+0000 7f365dc00640 1 -- 172.21.15.53:0/1758888260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3658102fe0 msgr2=0x7f3658074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:19.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.912+0000 7f365dc00640 1 --2- 172.21.15.53:0/1758888260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3658102fe0 0x7f3658074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:19.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.912+0000 7f365dc00640 1 -- 172.21.15.53:0/1758888260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36581039a0 msgr2=0x7f3658074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:19.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.912+0000 7f365dc00640 1 --2- 172.21.15.53:0/1758888260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36581039a0 0x7f3658074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:19.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.912+0000 7f365dc00640 1 -- 172.21.15.53:0/1758888260 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3658075110 con 0x7f3658101df0 2024-04-25T14:38:19.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.912+0000 7f365dc00640 1 --2- 172.21.15.53:0/1758888260 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3658101df0 0x7f3658075d00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f364800e9c0 tx=0x7f364800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:19.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.912+0000 7f3656000640 1 -- 172.21.15.53:0/1758888260 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364800cd80 con 0x7f3658101df0 2024-04-25T14:38:19.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.912+0000 7f3656000640 1 -- 172.21.15.53:0/1758888260 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36480035f0 con 0x7f3658101df0 2024-04-25T14:38:19.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.912+0000 7f365f000640 1 -- 172.21.15.53:0/1758888260 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36581a20f0 con 0x7f3658101df0 2024-04-25T14:38:19.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.912+0000 7f3656000640 1 -- 172.21.15.53:0/1758888260 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3648010690 con 0x7f3658101df0 2024-04-25T14:38:19.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.912+0000 7f365f000640 1 -- 172.21.15.53:0/1758888260 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36581a25f0 con 0x7f3658101df0 2024-04-25T14:38:19.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.913+0000 7f365f000640 1 -- 172.21.15.53:0/1758888260 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36581091d0 con 0x7f3658101df0 2024-04-25T14:38:19.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.916+0000 7f3656000640 1 -- 172.21.15.53:0/1758888260 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3648004000 con 0x7f3658101df0 2024-04-25T14:38:19.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.917+0000 7f3656000640 1 --2- 172.21.15.53:0/1758888260 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f363c0a3c40 0x7f363c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:19.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.917+0000 7f3656000640 1 -- 172.21.15.53:0/1758888260 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f36480d0b10 con 0x7f3658101df0 2024-04-25T14:38:19.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.917+0000 7f365d200640 1 --2- 172.21.15.53:0/1758888260 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f363c0a3c40 0x7f363c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:19.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.918+0000 7f365d200640 1 --2- 172.21.15.53:0/1758888260 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f363c0a3c40 0x7f363c0a6100 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3658075a70 tx=0x7f3644005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:19.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:19.919+0000 7f3656000640 1 -- 172.21.15.53:0/1758888260 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f364809d440 con 0x7f3658101df0 2024-04-25T14:38:20.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:20.121+0000 7f365f000640 1 -- 172.21.15.53:0/1758888260 --> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3658062390 con 0x7f363c0a3c40 2024-04-25T14:38:20.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:20.145+0000 7f3656000640 1 -- 172.21.15.53:0/1758888260 <== mgr.14152 v2:172.21.15.53:6800/4154933108 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258736 (secure 0 0 0) 0x7f3658062390 con 0x7f363c0a3c40 2024-04-25T14:38:20.150 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:38:20.155 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-04-25T14:38:20.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:20.150+0000 7f365f000640 1 -- 172.21.15.53:0/1758888260 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f363c0a3c40 msgr2=0x7f363c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:20.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:20.150+0000 7f365f000640 1 --2- 172.21.15.53:0/1758888260 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f363c0a3c40 0x7f363c0a6100 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3658075a70 tx=0x7f3644005f90 comp rx=0 tx=0).stop 2024-04-25T14:38:20.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:20.150+0000 7f365f000640 1 -- 172.21.15.53:0/1758888260 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3658101df0 msgr2=0x7f3658075d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:20.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:20.150+0000 7f365f000640 1 --2- 172.21.15.53:0/1758888260 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3658101df0 0x7f3658075d00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f364800e9c0 tx=0x7f364800ee80 comp rx=0 tx=0).stop 2024-04-25T14:38:20.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:20.151+0000 7f365f000640 1 -- 172.21.15.53:0/1758888260 shutdown_connections 2024-04-25T14:38:20.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:20.151+0000 7f365f000640 1 --2- 172.21.15.53:0/1758888260 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f363c0a3c40 0x7f363c0a6100 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:20.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:20.151+0000 7f365f000640 1 --2- 172.21.15.53:0/1758888260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36581039a0 0x7f3658074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:20.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:20.151+0000 7f365f000640 1 --2- 172.21.15.53:0/1758888260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3658102fe0 0x7f3658074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:20.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:20.151+0000 7f365f000640 1 --2- 172.21.15.53:0/1758888260 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3658101df0 0x7f3658075d00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:20.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:20.151+0000 7f365f000640 1 -- 172.21.15.53:0/1758888260 >> 172.21.15.53:0/1758888260 conn(0x7f36580fd360 msgr2=0x7f36581063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:20.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:20.151+0000 7f365f000640 1 -- 172.21.15.53:0/1758888260 shutdown_connections 2024-04-25T14:38:20.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:20.151+0000 7f365f000640 1 -- 172.21.15.53:0/1758888260 wait complete. 2024-04-25T14:38:20.173 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":320,"stamp":"2024-04-25T14:38:18.305302+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595359,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":151,"num_read_kb":132,"num_write":295,"num_write_kb":2975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":236,"ondisk_log_size":236,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374692,"kb_used_data":13044,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567132,"statfs":{"total":1151940427776,"available":1151556743168,"internally_reserved":0,"allocated":13357056,"data_stored":7214587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19137,"internal_metadata":370193727},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4991,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":15,"num_read_kb":13,"num_write":114,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003640"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.214257+0000","last_change":"2024-04-25T14:38:10.435535+0000","last_active":"2024-04-25T14:38:12.214257+0000","last_peered":"2024-04-25T14:38:12.214257+0000","last_clean":"2024-04-25T14:38:12.214257+0000","last_became_active":"2024-04-25T14:38:10.435205+0000","last_became_peered":"2024-04-25T14:38:10.435205+0000","last_unstale":"2024-04-25T14:38:12.214257+0000","last_undegraded":"2024-04-25T14:38:12.214257+0000","last_fullsized":"2024-04-25T14:38:12.214257+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:12.301771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.507996+0000","last_change":"2024-04-25T14:38:10.428216+0000","last_active":"2024-04-25T14:38:11.507996+0000","last_peered":"2024-04-25T14:38:11.507996+0000","last_clean":"2024-04-25T14:38:11.507996+0000","last_became_active":"2024-04-25T14:38:10.427419+0000","last_became_peered":"2024-04-25T14:38:10.427419+0000","last_unstale":"2024-04-25T14:38:11.507996+0000","last_undegraded":"2024-04-25T14:38:11.507996+0000","last_fullsized":"2024-04-25T14:38:11.507996+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:42.580424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.507851+0000","last_change":"2024-04-25T14:38:10.427827+0000","last_active":"2024-04-25T14:38:11.507851+0000","last_peered":"2024-04-25T14:38:11.507851+0000","last_clean":"2024-04-25T14:38:11.507851+0000","last_became_active":"2024-04-25T14:38:10.427435+0000","last_became_peered":"2024-04-25T14:38:10.427435+0000","last_unstale":"2024-04-25T14:38:11.507851+0000","last_undegraded":"2024-04-25T14:38:11.507851+0000","last_fullsized":"2024-04-25T14:38:11.507851+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:15.539399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.420475+0000","last_change":"2024-04-25T14:38:10.423751+0000","last_active":"2024-04-25T14:38:11.420475+0000","last_peered":"2024-04-25T14:38:11.420475+0000","last_clean":"2024-04-25T14:38:11.420475+0000","last_became_active":"2024-04-25T14:38:10.423409+0000","last_became_peered":"2024-04-25T14:38:10.423409+0000","last_unstale":"2024-04-25T14:38:11.420475+0000","last_undegraded":"2024-04-25T14:38:11.420475+0000","last_fullsized":"2024-04-25T14:38:11.420475+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:45.403788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.539992+0000","last_change":"2024-04-25T14:38:10.430000+0000","last_active":"2024-04-25T14:38:11.539992+0000","last_peered":"2024-04-25T14:38:11.539992+0000","last_clean":"2024-04-25T14:38:11.539992+0000","last_became_active":"2024-04-25T14:38:10.429684+0000","last_became_peered":"2024-04-25T14:38:10.429684+0000","last_unstale":"2024-04-25T14:38:11.539992+0000","last_undegraded":"2024-04-25T14:38:11.539992+0000","last_fullsized":"2024-04-25T14:38:11.539992+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:18.421970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.214524+0000","last_change":"2024-04-25T14:38:10.425057+0000","last_active":"2024-04-25T14:38:12.214524+0000","last_peered":"2024-04-25T14:38:12.214524+0000","last_clean":"2024-04-25T14:38:12.214524+0000","last_became_active":"2024-04-25T14:38:10.424463+0000","last_became_peered":"2024-04-25T14:38:10.424463+0000","last_unstale":"2024-04-25T14:38:12.214524+0000","last_undegraded":"2024-04-25T14:38:12.214524+0000","last_fullsized":"2024-04-25T14:38:12.214524+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:56.752189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.213996+0000","last_change":"2024-04-25T14:38:10.420022+0000","last_active":"2024-04-25T14:38:12.213996+0000","last_peered":"2024-04-25T14:38:12.213996+0000","last_clean":"2024-04-25T14:38:12.213996+0000","last_became_active":"2024-04-25T14:38:10.419673+0000","last_became_peered":"2024-04-25T14:38:10.419673+0000","last_unstale":"2024-04-25T14:38:12.213996+0000","last_undegraded":"2024-04-25T14:38:12.213996+0000","last_fullsized":"2024-04-25T14:38:12.213996+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:09.177814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.419909+0000","last_change":"2024-04-25T14:38:10.436154+0000","last_active":"2024-04-25T14:38:11.419909+0000","last_peered":"2024-04-25T14:38:11.419909+0000","last_clean":"2024-04-25T14:38:11.419909+0000","last_became_active":"2024-04-25T14:38:10.435788+0000","last_became_peered":"2024-04-25T14:38:10.435788+0000","last_unstale":"2024-04-25T14:38:11.419909+0000","last_undegraded":"2024-04-25T14:38:11.419909+0000","last_fullsized":"2024-04-25T14:38:11.419909+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:15.317674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.458628+0000","last_change":"2024-04-25T14:38:10.424086+0000","last_active":"2024-04-25T14:38:11.458628+0000","last_peered":"2024-04-25T14:38:11.458628+0000","last_clean":"2024-04-25T14:38:11.458628+0000","last_became_active":"2024-04-25T14:38:10.423769+0000","last_became_peered":"2024-04-25T14:38:10.423769+0000","last_unstale":"2024-04-25T14:38:11.458628+0000","last_undegraded":"2024-04-25T14:38:11.458628+0000","last_fullsized":"2024-04-25T14:38:11.458628+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:19:22.736053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.215123+0000","last_change":"2024-04-25T14:38:10.427049+0000","last_active":"2024-04-25T14:38:12.215123+0000","last_peered":"2024-04-25T14:38:12.215123+0000","last_clean":"2024-04-25T14:38:12.215123+0000","last_became_active":"2024-04-25T14:38:10.426729+0000","last_became_peered":"2024-04-25T14:38:10.426729+0000","last_unstale":"2024-04-25T14:38:12.215123+0000","last_undegraded":"2024-04-25T14:38:12.215123+0000","last_fullsized":"2024-04-25T14:38:12.215123+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:00.849650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.214437+0000","last_change":"2024-04-25T14:38:10.424976+0000","last_active":"2024-04-25T14:38:12.214437+0000","last_peered":"2024-04-25T14:38:12.214437+0000","last_clean":"2024-04-25T14:38:12.214437+0000","last_became_active":"2024-04-25T14:38:10.424585+0000","last_became_peered":"2024-04-25T14:38:10.424585+0000","last_unstale":"2024-04-25T14:38:12.214437+0000","last_undegraded":"2024-04-25T14:38:12.214437+0000","last_fullsized":"2024-04-25T14:38:12.214437+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:54.639471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.420470+0000","last_change":"2024-04-25T14:38:10.418022+0000","last_active":"2024-04-25T14:38:11.420470+0000","last_peered":"2024-04-25T14:38:11.420470+0000","last_clean":"2024-04-25T14:38:11.420470+0000","last_became_active":"2024-04-25T14:38:10.417735+0000","last_became_peered":"2024-04-25T14:38:10.417735+0000","last_unstale":"2024-04-25T14:38:11.420470+0000","last_undegraded":"2024-04-25T14:38:11.420470+0000","last_fullsized":"2024-04-25T14:38:11.420470+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:48.785521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.759636+0000","last_change":"2024-04-25T14:38:10.423464+0000","last_active":"2024-04-25T14:38:11.759636+0000","last_peered":"2024-04-25T14:38:11.759636+0000","last_clean":"2024-04-25T14:38:11.759636+0000","last_became_active":"2024-04-25T14:38:10.423105+0000","last_became_peered":"2024-04-25T14:38:10.423105+0000","last_unstale":"2024-04-25T14:38:11.759636+0000","last_undegraded":"2024-04-25T14:38:11.759636+0000","last_fullsized":"2024-04-25T14:38:11.759636+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:26.568198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.507518+0000","last_change":"2024-04-25T14:38:10.426870+0000","last_active":"2024-04-25T14:38:11.507518+0000","last_peered":"2024-04-25T14:38:11.507518+0000","last_clean":"2024-04-25T14:38:11.507518+0000","last_became_active":"2024-04-25T14:38:10.426506+0000","last_became_peered":"2024-04-25T14:38:10.426506+0000","last_unstale":"2024-04-25T14:38:11.507518+0000","last_undegraded":"2024-04-25T14:38:11.507518+0000","last_fullsized":"2024-04-25T14:38:11.507518+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:16:49.539221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.507428+0000","last_change":"2024-04-25T14:38:10.434618+0000","last_active":"2024-04-25T14:38:11.507428+0000","last_peered":"2024-04-25T14:38:11.507428+0000","last_clean":"2024-04-25T14:38:11.507428+0000","last_became_active":"2024-04-25T14:38:10.430661+0000","last_became_peered":"2024-04-25T14:38:10.430661+0000","last_unstale":"2024-04-25T14:38:11.507428+0000","last_undegraded":"2024-04-25T14:38:11.507428+0000","last_fullsized":"2024-04-25T14:38:11.507428+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:44.548788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.458022+0000","last_change":"2024-04-25T14:38:10.429898+0000","last_active":"2024-04-25T14:38:11.458022+0000","last_peered":"2024-04-25T14:38:11.458022+0000","last_clean":"2024-04-25T14:38:11.458022+0000","last_became_active":"2024-04-25T14:38:10.429557+0000","last_became_peered":"2024-04-25T14:38:10.429557+0000","last_unstale":"2024-04-25T14:38:11.458022+0000","last_undegraded":"2024-04-25T14:38:11.458022+0000","last_fullsized":"2024-04-25T14:38:11.458022+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:54.258526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.214414+0000","last_change":"2024-04-25T14:38:10.435134+0000","last_active":"2024-04-25T14:38:12.214414+0000","last_peered":"2024-04-25T14:38:12.214414+0000","last_clean":"2024-04-25T14:38:12.214414+0000","last_became_active":"2024-04-25T14:38:10.434775+0000","last_became_peered":"2024-04-25T14:38:10.434775+0000","last_unstale":"2024-04-25T14:38:12.214414+0000","last_undegraded":"2024-04-25T14:38:12.214414+0000","last_fullsized":"2024-04-25T14:38:12.214414+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:49.827881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.421743+0000","last_change":"2024-04-25T14:38:10.424929+0000","last_active":"2024-04-25T14:38:11.421743+0000","last_peered":"2024-04-25T14:38:11.421743+0000","last_clean":"2024-04-25T14:38:11.421743+0000","last_became_active":"2024-04-25T14:38:10.424399+0000","last_became_peered":"2024-04-25T14:38:10.424399+0000","last_unstale":"2024-04-25T14:38:11.421743+0000","last_undegraded":"2024-04-25T14:38:11.421743+0000","last_fullsized":"2024-04-25T14:38:11.421743+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:08.834176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.214480+0000","last_change":"2024-04-25T14:38:10.444004+0000","last_active":"2024-04-25T14:38:12.214480+0000","last_peered":"2024-04-25T14:38:12.214480+0000","last_clean":"2024-04-25T14:38:12.214480+0000","last_became_active":"2024-04-25T14:38:10.443669+0000","last_became_peered":"2024-04-25T14:38:10.443669+0000","last_unstale":"2024-04-25T14:38:12.214480+0000","last_undegraded":"2024-04-25T14:38:12.214480+0000","last_fullsized":"2024-04-25T14:38:12.214480+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:02.302057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.421262+0000","last_change":"2024-04-25T14:38:09.545010+0000","last_active":"2024-04-25T14:38:11.421262+0000","last_peered":"2024-04-25T14:38:11.421262+0000","last_clean":"2024-04-25T14:38:11.421262+0000","last_became_active":"2024-04-25T14:38:09.544622+0000","last_became_peered":"2024-04-25T14:38:09.544622+0000","last_unstale":"2024-04-25T14:38:11.421262+0000","last_undegraded":"2024-04-25T14:38:11.421262+0000","last_fullsized":"2024-04-25T14:38:11.421262+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:51.200242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.421654+0000","last_change":"2024-04-25T14:38:10.446273+0000","last_active":"2024-04-25T14:38:11.421654+0000","last_peered":"2024-04-25T14:38:11.421654+0000","last_clean":"2024-04-25T14:38:11.421654+0000","last_became_active":"2024-04-25T14:38:10.445896+0000","last_became_peered":"2024-04-25T14:38:10.445896+0000","last_unstale":"2024-04-25T14:38:11.421654+0000","last_undegraded":"2024-04-25T14:38:11.421654+0000","last_fullsized":"2024-04-25T14:38:11.421654+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:25.018372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.421998+0000","last_change":"2024-04-25T14:38:09.555825+0000","last_active":"2024-04-25T14:38:11.421998+0000","last_peered":"2024-04-25T14:38:11.421998+0000","last_clean":"2024-04-25T14:38:11.421998+0000","last_became_active":"2024-04-25T14:38:09.555436+0000","last_became_peered":"2024-04-25T14:38:09.555436+0000","last_unstale":"2024-04-25T14:38:11.421998+0000","last_undegraded":"2024-04-25T14:38:11.421998+0000","last_fullsized":"2024-04-25T14:38:11.421998+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:45.097583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.508172+0000","last_change":"2024-04-25T14:38:10.432254+0000","last_active":"2024-04-25T14:38:11.508172+0000","last_peered":"2024-04-25T14:38:11.508172+0000","last_clean":"2024-04-25T14:38:11.508172+0000","last_became_active":"2024-04-25T14:38:10.431835+0000","last_became_peered":"2024-04-25T14:38:10.431835+0000","last_unstale":"2024-04-25T14:38:11.508172+0000","last_undegraded":"2024-04-25T14:38:11.508172+0000","last_fullsized":"2024-04-25T14:38:11.508172+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:23.089447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:15.024366+0000","last_change":"2024-04-25T14:38:09.550254+0000","last_active":"2024-04-25T14:38:15.024366+0000","last_peered":"2024-04-25T14:38:15.024366+0000","last_clean":"2024-04-25T14:38:15.024366+0000","last_became_active":"2024-04-25T14:38:09.549958+0000","last_became_peered":"2024-04-25T14:38:09.549958+0000","last_unstale":"2024-04-25T14:38:15.024366+0000","last_undegraded":"2024-04-25T14:38:15.024366+0000","last_fullsized":"2024-04-25T14:38:15.024366+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:56.161427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.458235+0000","last_change":"2024-04-25T14:38:10.425648+0000","last_active":"2024-04-25T14:38:11.458235+0000","last_peered":"2024-04-25T14:38:11.458235+0000","last_clean":"2024-04-25T14:38:11.458235+0000","last_became_active":"2024-04-25T14:38:10.425009+0000","last_became_peered":"2024-04-25T14:38:10.425009+0000","last_unstale":"2024-04-25T14:38:11.458235+0000","last_undegraded":"2024-04-25T14:38:11.458235+0000","last_fullsized":"2024-04-25T14:38:11.458235+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:46.696610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.540071+0000","last_change":"2024-04-25T14:38:09.544481+0000","last_active":"2024-04-25T14:38:11.540071+0000","last_peered":"2024-04-25T14:38:11.540071+0000","last_clean":"2024-04-25T14:38:11.540071+0000","last_became_active":"2024-04-25T14:38:09.544147+0000","last_became_peered":"2024-04-25T14:38:09.544147+0000","last_unstale":"2024-04-25T14:38:11.540071+0000","last_undegraded":"2024-04-25T14:38:11.540071+0000","last_fullsized":"2024-04-25T14:38:11.540071+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:35:22.971028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.454575+0000","last_change":"2024-04-25T14:38:10.426851+0000","last_active":"2024-04-25T14:38:11.454575+0000","last_peered":"2024-04-25T14:38:11.454575+0000","last_clean":"2024-04-25T14:38:11.454575+0000","last_became_active":"2024-04-25T14:38:10.426475+0000","last_became_peered":"2024-04-25T14:38:10.426475+0000","last_unstale":"2024-04-25T14:38:11.454575+0000","last_undegraded":"2024-04-25T14:38:11.454575+0000","last_fullsized":"2024-04-25T14:38:11.454575+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:28.213318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:16.643049+0000","last_change":"2024-04-25T14:38:09.546357+0000","last_active":"2024-04-25T14:38:16.643049+0000","last_peered":"2024-04-25T14:38:16.643049+0000","last_clean":"2024-04-25T14:38:16.643049+0000","last_became_active":"2024-04-25T14:38:09.545940+0000","last_became_peered":"2024-04-25T14:38:09.545940+0000","last_unstale":"2024-04-25T14:38:16.643049+0000","last_undegraded":"2024-04-25T14:38:16.643049+0000","last_fullsized":"2024-04-25T14:38:16.643049+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:28:37.807461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.540044+0000","last_change":"2024-04-25T14:38:10.425424+0000","last_active":"2024-04-25T14:38:11.540044+0000","last_peered":"2024-04-25T14:38:11.540044+0000","last_clean":"2024-04-25T14:38:11.540044+0000","last_became_active":"2024-04-25T14:38:10.425101+0000","last_became_peered":"2024-04-25T14:38:10.425101+0000","last_unstale":"2024-04-25T14:38:11.540044+0000","last_undegraded":"2024-04-25T14:38:11.540044+0000","last_fullsized":"2024-04-25T14:38:11.540044+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:52:29.871113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:15.027097+0000","last_change":"2024-04-25T14:38:09.552287+0000","last_active":"2024-04-25T14:38:15.027097+0000","last_peered":"2024-04-25T14:38:15.027097+0000","last_clean":"2024-04-25T14:38:15.027097+0000","last_became_active":"2024-04-25T14:38:09.551957+0000","last_became_peered":"2024-04-25T14:38:09.551957+0000","last_unstale":"2024-04-25T14:38:15.027097+0000","last_undegraded":"2024-04-25T14:38:15.027097+0000","last_fullsized":"2024-04-25T14:38:15.027097+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:22.586868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.454610+0000","last_change":"2024-04-25T14:38:10.443959+0000","last_active":"2024-04-25T14:38:11.454610+0000","last_peered":"2024-04-25T14:38:11.454610+0000","last_clean":"2024-04-25T14:38:11.454610+0000","last_became_active":"2024-04-25T14:38:10.443535+0000","last_became_peered":"2024-04-25T14:38:10.443535+0000","last_unstale":"2024-04-25T14:38:11.454610+0000","last_undegraded":"2024-04-25T14:38:11.454610+0000","last_fullsized":"2024-04-25T14:38:11.454610+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:59.972322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:15.028249+0000","last_change":"2024-04-25T14:38:09.552376+0000","last_active":"2024-04-25T14:38:15.028249+0000","last_peered":"2024-04-25T14:38:15.028249+0000","last_clean":"2024-04-25T14:38:15.028249+0000","last_became_active":"2024-04-25T14:38:09.548614+0000","last_became_peered":"2024-04-25T14:38:09.548614+0000","last_unstale":"2024-04-25T14:38:15.028249+0000","last_undegraded":"2024-04-25T14:38:15.028249+0000","last_fullsized":"2024-04-25T14:38:15.028249+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:27:27.133106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.420020+0000","last_change":"2024-04-25T14:38:10.434611+0000","last_active":"2024-04-25T14:38:11.420020+0000","last_peered":"2024-04-25T14:38:11.420020+0000","last_clean":"2024-04-25T14:38:11.420020+0000","last_became_active":"2024-04-25T14:38:10.434280+0000","last_became_peered":"2024-04-25T14:38:10.434280+0000","last_unstale":"2024-04-25T14:38:11.420020+0000","last_undegraded":"2024-04-25T14:38:11.420020+0000","last_fullsized":"2024-04-25T14:38:11.420020+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:12:32.275663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.991562+0000","last_change":"2024-04-25T14:38:09.539743+0000","last_active":"2024-04-25T14:38:11.991562+0000","last_peered":"2024-04-25T14:38:11.991562+0000","last_clean":"2024-04-25T14:38:11.991562+0000","last_became_active":"2024-04-25T14:38:09.539272+0000","last_became_peered":"2024-04-25T14:38:09.539272+0000","last_unstale":"2024-04-25T14:38:11.991562+0000","last_undegraded":"2024-04-25T14:38:11.991562+0000","last_fullsized":"2024-04-25T14:38:11.991562+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:29:23.159901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.421073+0000","last_change":"2024-04-25T14:38:10.444899+0000","last_active":"2024-04-25T14:38:11.421073+0000","last_peered":"2024-04-25T14:38:11.421073+0000","last_clean":"2024-04-25T14:38:11.421073+0000","last_became_active":"2024-04-25T14:38:10.444383+0000","last_became_peered":"2024-04-25T14:38:10.444383+0000","last_unstale":"2024-04-25T14:38:11.421073+0000","last_undegraded":"2024-04-25T14:38:11.421073+0000","last_fullsized":"2024-04-25T14:38:11.421073+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:21.351436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:14.007200+0000","last_change":"2024-04-25T14:38:09.550181+0000","last_active":"2024-04-25T14:38:14.007200+0000","last_peered":"2024-04-25T14:38:14.007200+0000","last_clean":"2024-04-25T14:38:14.007200+0000","last_became_active":"2024-04-25T14:38:09.549824+0000","last_became_peered":"2024-04-25T14:38:09.549824+0000","last_unstale":"2024-04-25T14:38:14.007200+0000","last_undegraded":"2024-04-25T14:38:14.007200+0000","last_fullsized":"2024-04-25T14:38:14.007200+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:59.220917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.458502+0000","last_change":"2024-04-25T14:38:10.421339+0000","last_active":"2024-04-25T14:38:11.458502+0000","last_peered":"2024-04-25T14:38:11.458502+0000","last_clean":"2024-04-25T14:38:11.458502+0000","last_became_active":"2024-04-25T14:38:10.421005+0000","last_became_peered":"2024-04-25T14:38:10.421005+0000","last_unstale":"2024-04-25T14:38:11.458502+0000","last_undegraded":"2024-04-25T14:38:11.458502+0000","last_fullsized":"2024-04-25T14:38:11.458502+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:51.949054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:14.004053+0000","last_change":"2024-04-25T14:38:09.548245+0000","last_active":"2024-04-25T14:38:14.004053+0000","last_peered":"2024-04-25T14:38:14.004053+0000","last_clean":"2024-04-25T14:38:14.004053+0000","last_became_active":"2024-04-25T14:38:09.547905+0000","last_became_peered":"2024-04-25T14:38:09.547905+0000","last_unstale":"2024-04-25T14:38:14.004053+0000","last_undegraded":"2024-04-25T14:38:14.004053+0000","last_fullsized":"2024-04-25T14:38:14.004053+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:56.798594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.508168+0000","last_change":"2024-04-25T14:38:10.431427+0000","last_active":"2024-04-25T14:38:11.508168+0000","last_peered":"2024-04-25T14:38:11.508168+0000","last_clean":"2024-04-25T14:38:11.508168+0000","last_became_active":"2024-04-25T14:38:10.428759+0000","last_became_peered":"2024-04-25T14:38:10.428759+0000","last_unstale":"2024-04-25T14:38:11.508168+0000","last_undegraded":"2024-04-25T14:38:11.508168+0000","last_fullsized":"2024-04-25T14:38:11.508168+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:15:27.341445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:14.006219+0000","last_change":"2024-04-25T14:38:09.556839+0000","last_active":"2024-04-25T14:38:14.006219+0000","last_peered":"2024-04-25T14:38:14.006219+0000","last_clean":"2024-04-25T14:38:14.006219+0000","last_became_active":"2024-04-25T14:38:09.556443+0000","last_became_peered":"2024-04-25T14:38:09.556443+0000","last_unstale":"2024-04-25T14:38:14.006219+0000","last_undegraded":"2024-04-25T14:38:14.006219+0000","last_fullsized":"2024-04-25T14:38:14.006219+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:46.621883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.458372+0000","last_change":"2024-04-25T14:38:10.414507+0000","last_active":"2024-04-25T14:38:11.458372+0000","last_peered":"2024-04-25T14:38:11.458372+0000","last_clean":"2024-04-25T14:38:11.458372+0000","last_became_active":"2024-04-25T14:38:10.414154+0000","last_became_peered":"2024-04-25T14:38:10.414154+0000","last_unstale":"2024-04-25T14:38:11.458372+0000","last_undegraded":"2024-04-25T14:38:11.458372+0000","last_fullsized":"2024-04-25T14:38:11.458372+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:12.208774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'8","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:14.004348+0000","last_change":"2024-04-25T14:38:09.549239+0000","last_active":"2024-04-25T14:38:14.004348+0000","last_peered":"2024-04-25T14:38:14.004348+0000","last_clean":"2024-04-25T14:38:14.004348+0000","last_became_active":"2024-04-25T14:38:09.548888+0000","last_became_peered":"2024-04-25T14:38:09.548888+0000","last_unstale":"2024-04-25T14:38:14.004348+0000","last_undegraded":"2024-04-25T14:38:14.004348+0000","last_fullsized":"2024-04-25T14:38:14.004348+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:36:46.536908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.540092+0000","last_change":"2024-04-25T14:38:10.417510+0000","last_active":"2024-04-25T14:38:11.540092+0000","last_peered":"2024-04-25T14:38:11.540092+0000","last_clean":"2024-04-25T14:38:11.540092+0000","last_became_active":"2024-04-25T14:38:10.417079+0000","last_became_peered":"2024-04-25T14:38:10.417079+0000","last_unstale":"2024-04-25T14:38:11.540092+0000","last_undegraded":"2024-04-25T14:38:11.540092+0000","last_fullsized":"2024-04-25T14:38:11.540092+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:35.213199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:15.030513+0000","last_change":"2024-04-25T14:38:09.563650+0000","last_active":"2024-04-25T14:38:15.030513+0000","last_peered":"2024-04-25T14:38:15.030513+0000","last_clean":"2024-04-25T14:38:15.030513+0000","last_became_active":"2024-04-25T14:38:09.563347+0000","last_became_peered":"2024-04-25T14:38:09.563347+0000","last_unstale":"2024-04-25T14:38:15.030513+0000","last_undegraded":"2024-04-25T14:38:15.030513+0000","last_fullsized":"2024-04-25T14:38:15.030513+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:16.793191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.759439+0000","last_change":"2024-04-25T14:38:10.447504+0000","last_active":"2024-04-25T14:38:11.759439+0000","last_peered":"2024-04-25T14:38:11.759439+0000","last_clean":"2024-04-25T14:38:11.759439+0000","last_became_active":"2024-04-25T14:38:10.447157+0000","last_became_peered":"2024-04-25T14:38:10.447157+0000","last_unstale":"2024-04-25T14:38:11.759439+0000","last_undegraded":"2024-04-25T14:38:11.759439+0000","last_fullsized":"2024-04-25T14:38:11.759439+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:38.940320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:14.007260+0000","last_change":"2024-04-25T14:38:09.545214+0000","last_active":"2024-04-25T14:38:14.007260+0000","last_peered":"2024-04-25T14:38:14.007260+0000","last_clean":"2024-04-25T14:38:14.007260+0000","last_became_active":"2024-04-25T14:38:09.544903+0000","last_became_peered":"2024-04-25T14:38:09.544903+0000","last_unstale":"2024-04-25T14:38:14.007260+0000","last_undegraded":"2024-04-25T14:38:14.007260+0000","last_fullsized":"2024-04-25T14:38:14.007260+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:58:15.817036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.214622+0000","last_change":"2024-04-25T14:38:10.432017+0000","last_active":"2024-04-25T14:38:12.214622+0000","last_peered":"2024-04-25T14:38:12.214622+0000","last_clean":"2024-04-25T14:38:12.214622+0000","last_became_active":"2024-04-25T14:38:10.431596+0000","last_became_peered":"2024-04-25T14:38:10.431596+0000","last_unstale":"2024-04-25T14:38:12.214622+0000","last_undegraded":"2024-04-25T14:38:12.214622+0000","last_fullsized":"2024-04-25T14:38:12.214622+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:30.653642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.420027+0000","last_change":"2024-04-25T14:38:09.545477+0000","last_active":"2024-04-25T14:38:11.420027+0000","last_peered":"2024-04-25T14:38:11.420027+0000","last_clean":"2024-04-25T14:38:11.420027+0000","last_became_active":"2024-04-25T14:38:09.545093+0000","last_became_peered":"2024-04-25T14:38:09.545093+0000","last_unstale":"2024-04-25T14:38:11.420027+0000","last_undegraded":"2024-04-25T14:38:11.420027+0000","last_fullsized":"2024-04-25T14:38:11.420027+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:56.589231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.213871+0000","last_change":"2024-04-25T14:38:10.418914+0000","last_active":"2024-04-25T14:38:12.213871+0000","last_peered":"2024-04-25T14:38:12.213871+0000","last_clean":"2024-04-25T14:38:12.213871+0000","last_became_active":"2024-04-25T14:38:10.418600+0000","last_became_peered":"2024-04-25T14:38:10.418600+0000","last_unstale":"2024-04-25T14:38:12.213871+0000","last_undegraded":"2024-04-25T14:38:12.213871+0000","last_fullsized":"2024-04-25T14:38:12.213871+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:31.324035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:15.029690+0000","last_change":"2024-04-25T14:38:09.556574+0000","last_active":"2024-04-25T14:38:15.029690+0000","last_peered":"2024-04-25T14:38:15.029690+0000","last_clean":"2024-04-25T14:38:15.029690+0000","last_became_active":"2024-04-25T14:38:09.556071+0000","last_became_peered":"2024-04-25T14:38:09.556071+0000","last_unstale":"2024-04-25T14:38:15.029690+0000","last_undegraded":"2024-04-25T14:38:15.029690+0000","last_fullsized":"2024-04-25T14:38:15.029690+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:05.424593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.454366+0000","last_change":"2024-04-25T14:38:10.424591+0000","last_active":"2024-04-25T14:38:11.454366+0000","last_peered":"2024-04-25T14:38:11.454366+0000","last_clean":"2024-04-25T14:38:11.454366+0000","last_became_active":"2024-04-25T14:38:10.424258+0000","last_became_peered":"2024-04-25T14:38:10.424258+0000","last_unstale":"2024-04-25T14:38:11.454366+0000","last_undegraded":"2024-04-25T14:38:11.454366+0000","last_fullsized":"2024-04-25T14:38:11.454366+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:22.753595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:15.640707+0000","last_change":"2024-04-25T14:38:09.546073+0000","last_active":"2024-04-25T14:38:15.640707+0000","last_peered":"2024-04-25T14:38:15.640707+0000","last_clean":"2024-04-25T14:38:15.640707+0000","last_became_active":"2024-04-25T14:38:09.545783+0000","last_became_peered":"2024-04-25T14:38:09.545783+0000","last_unstale":"2024-04-25T14:38:15.640707+0000","last_undegraded":"2024-04-25T14:38:15.640707+0000","last_fullsized":"2024-04-25T14:38:15.640707+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:42.537431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.214028+0000","last_change":"2024-04-25T14:38:10.417230+0000","last_active":"2024-04-25T14:38:12.214028+0000","last_peered":"2024-04-25T14:38:12.214028+0000","last_clean":"2024-04-25T14:38:12.214028+0000","last_became_active":"2024-04-25T14:38:10.416891+0000","last_became_peered":"2024-04-25T14:38:10.416891+0000","last_unstale":"2024-04-25T14:38:12.214028+0000","last_undegraded":"2024-04-25T14:38:12.214028+0000","last_fullsized":"2024-04-25T14:38:12.214028+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:10:13.434518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.421556+0000","last_change":"2024-04-25T14:38:09.553869+0000","last_active":"2024-04-25T14:38:11.421556+0000","last_peered":"2024-04-25T14:38:11.421556+0000","last_clean":"2024-04-25T14:38:11.421556+0000","last_became_active":"2024-04-25T14:38:09.552625+0000","last_became_peered":"2024-04-25T14:38:09.552625+0000","last_unstale":"2024-04-25T14:38:11.421556+0000","last_undegraded":"2024-04-25T14:38:11.421556+0000","last_fullsized":"2024-04-25T14:38:11.421556+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:16.751312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.214817+0000","last_change":"2024-04-25T14:38:10.446846+0000","last_active":"2024-04-25T14:38:12.214817+0000","last_peered":"2024-04-25T14:38:12.214817+0000","last_clean":"2024-04-25T14:38:12.214817+0000","last_became_active":"2024-04-25T14:38:10.446498+0000","last_became_peered":"2024-04-25T14:38:10.446498+0000","last_unstale":"2024-04-25T14:38:12.214817+0000","last_undegraded":"2024-04-25T14:38:12.214817+0000","last_fullsized":"2024-04-25T14:38:12.214817+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:14:37.179198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.214680+0000","last_change":"2024-04-25T14:38:09.544283+0000","last_active":"2024-04-25T14:38:12.214680+0000","last_peered":"2024-04-25T14:38:12.214680+0000","last_clean":"2024-04-25T14:38:12.214680+0000","last_became_active":"2024-04-25T14:38:09.543135+0000","last_became_peered":"2024-04-25T14:38:09.543135+0000","last_unstale":"2024-04-25T14:38:12.214680+0000","last_undegraded":"2024-04-25T14:38:12.214680+0000","last_fullsized":"2024-04-25T14:38:12.214680+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:05.977062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.214751+0000","last_change":"2024-04-25T14:38:10.432863+0000","last_active":"2024-04-25T14:38:12.214751+0000","last_peered":"2024-04-25T14:38:12.214751+0000","last_clean":"2024-04-25T14:38:12.214751+0000","last_became_active":"2024-04-25T14:38:10.432493+0000","last_became_peered":"2024-04-25T14:38:10.432493+0000","last_unstale":"2024-04-25T14:38:12.214751+0000","last_undegraded":"2024-04-25T14:38:12.214751+0000","last_fullsized":"2024-04-25T14:38:12.214751+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:25:53.774764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.454423+0000","last_change":"2024-04-25T14:38:10.422265+0000","last_active":"2024-04-25T14:38:11.454423+0000","last_peered":"2024-04-25T14:38:11.454423+0000","last_clean":"2024-04-25T14:38:11.454423+0000","last_became_active":"2024-04-25T14:38:10.421952+0000","last_became_peered":"2024-04-25T14:38:10.421952+0000","last_unstale":"2024-04-25T14:38:11.454423+0000","last_undegraded":"2024-04-25T14:38:11.454423+0000","last_fullsized":"2024-04-25T14:38:11.454423+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:39.252185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:15.030691+0000","last_change":"2024-04-25T14:38:09.554305+0000","last_active":"2024-04-25T14:38:15.030691+0000","last_peered":"2024-04-25T14:38:15.030691+0000","last_clean":"2024-04-25T14:38:15.030691+0000","last_became_active":"2024-04-25T14:38:09.553984+0000","last_became_peered":"2024-04-25T14:38:09.553984+0000","last_unstale":"2024-04-25T14:38:15.030691+0000","last_undegraded":"2024-04-25T14:38:15.030691+0000","last_fullsized":"2024-04-25T14:38:15.030691+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:41.580867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.420161+0000","last_change":"2024-04-25T14:38:10.448290+0000","last_active":"2024-04-25T14:38:11.420161+0000","last_peered":"2024-04-25T14:38:11.420161+0000","last_clean":"2024-04-25T14:38:11.420161+0000","last_became_active":"2024-04-25T14:38:10.446147+0000","last_became_peered":"2024-04-25T14:38:10.446147+0000","last_unstale":"2024-04-25T14:38:11.420161+0000","last_undegraded":"2024-04-25T14:38:11.420161+0000","last_fullsized":"2024-04-25T14:38:11.420161+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:12.775551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:14.006895+0000","last_change":"2024-04-25T14:38:09.545209+0000","last_active":"2024-04-25T14:38:14.006895+0000","last_peered":"2024-04-25T14:38:14.006895+0000","last_clean":"2024-04-25T14:38:14.006895+0000","last_became_active":"2024-04-25T14:38:09.544861+0000","last_became_peered":"2024-04-25T14:38:09.544861+0000","last_unstale":"2024-04-25T14:38:14.006895+0000","last_undegraded":"2024-04-25T14:38:14.006895+0000","last_fullsized":"2024-04-25T14:38:14.006895+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:25.829162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":349,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.213852+0000","last_change":"2024-04-25T14:36:48.250081+0000","last_active":"2024-04-25T14:38:12.213852+0000","last_peered":"2024-04-25T14:38:12.213852+0000","last_clean":"2024-04-25T14:38:12.213852+0000","last_became_active":"2024-04-25T14:36:48.249520+0000","last_became_peered":"2024-04-25T14:36:48.249520+0000","last_unstale":"2024-04-25T14:38:12.213852+0000","last_undegraded":"2024-04-25T14:38:12.213852+0000","last_fullsized":"2024-04-25T14:38:12.213852+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:33:03.061016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:33:03.061016+0000","last_clean_scrub_stamp":"2024-04-25T14:33:03.061016+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:04:06.653326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.759241+0000","last_change":"2024-04-25T14:38:10.422966+0000","last_active":"2024-04-25T14:38:11.759241+0000","last_peered":"2024-04-25T14:38:11.759241+0000","last_clean":"2024-04-25T14:38:11.759241+0000","last_became_active":"2024-04-25T14:38:10.422568+0000","last_became_peered":"2024-04-25T14:38:10.422568+0000","last_unstale":"2024-04-25T14:38:11.759241+0000","last_undegraded":"2024-04-25T14:38:11.759241+0000","last_fullsized":"2024-04-25T14:38:11.759241+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:17.405074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:15.025689+0000","last_change":"2024-04-25T14:38:09.542051+0000","last_active":"2024-04-25T14:38:15.025689+0000","last_peered":"2024-04-25T14:38:15.025689+0000","last_clean":"2024-04-25T14:38:15.025689+0000","last_became_active":"2024-04-25T14:38:09.541564+0000","last_became_peered":"2024-04-25T14:38:09.541564+0000","last_unstale":"2024-04-25T14:38:15.025689+0000","last_undegraded":"2024-04-25T14:38:15.025689+0000","last_fullsized":"2024-04-25T14:38:15.025689+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:44:30.122416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:14.007421+0000","last_change":"2024-04-25T14:38:09.560357+0000","last_active":"2024-04-25T14:38:14.007421+0000","last_peered":"2024-04-25T14:38:14.007421+0000","last_clean":"2024-04-25T14:38:14.007421+0000","last_became_active":"2024-04-25T14:38:09.560046+0000","last_became_peered":"2024-04-25T14:38:09.560046+0000","last_unstale":"2024-04-25T14:38:14.007421+0000","last_undegraded":"2024-04-25T14:38:14.007421+0000","last_fullsized":"2024-04-25T14:38:14.007421+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:22.692116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.759793+0000","last_change":"2024-04-25T14:38:10.429554+0000","last_active":"2024-04-25T14:38:11.759793+0000","last_peered":"2024-04-25T14:38:11.759793+0000","last_clean":"2024-04-25T14:38:11.759793+0000","last_became_active":"2024-04-25T14:38:10.428944+0000","last_became_peered":"2024-04-25T14:38:10.428944+0000","last_unstale":"2024-04-25T14:38:11.759793+0000","last_undegraded":"2024-04-25T14:38:11.759793+0000","last_fullsized":"2024-04-25T14:38:11.759793+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:38.013028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.421388+0000","last_change":"2024-04-25T14:38:09.564849+0000","last_active":"2024-04-25T14:38:11.421388+0000","last_peered":"2024-04-25T14:38:11.421388+0000","last_clean":"2024-04-25T14:38:11.421388+0000","last_became_active":"2024-04-25T14:38:09.564354+0000","last_became_peered":"2024-04-25T14:38:09.564354+0000","last_unstale":"2024-04-25T14:38:11.421388+0000","last_undegraded":"2024-04-25T14:38:11.421388+0000","last_fullsized":"2024-04-25T14:38:11.421388+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:44.308366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.421519+0000","last_change":"2024-04-25T14:38:10.438178+0000","last_active":"2024-04-25T14:38:11.421519+0000","last_peered":"2024-04-25T14:38:11.421519+0000","last_clean":"2024-04-25T14:38:11.421519+0000","last_became_active":"2024-04-25T14:38:10.429936+0000","last_became_peered":"2024-04-25T14:38:10.429936+0000","last_unstale":"2024-04-25T14:38:11.421519+0000","last_undegraded":"2024-04-25T14:38:11.421519+0000","last_fullsized":"2024-04-25T14:38:11.421519+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:33.598747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:14.006393+0000","last_change":"2024-04-25T14:38:09.547680+0000","last_active":"2024-04-25T14:38:14.006393+0000","last_peered":"2024-04-25T14:38:14.006393+0000","last_clean":"2024-04-25T14:38:14.006393+0000","last_became_active":"2024-04-25T14:38:09.547337+0000","last_became_peered":"2024-04-25T14:38:09.547337+0000","last_unstale":"2024-04-25T14:38:14.006393+0000","last_undegraded":"2024-04-25T14:38:14.006393+0000","last_fullsized":"2024-04-25T14:38:14.006393+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:32.754439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.507943+0000","last_change":"2024-04-25T14:38:10.447115+0000","last_active":"2024-04-25T14:38:11.507943+0000","last_peered":"2024-04-25T14:38:11.507943+0000","last_clean":"2024-04-25T14:38:11.507943+0000","last_became_active":"2024-04-25T14:38:10.446681+0000","last_became_peered":"2024-04-25T14:38:10.446681+0000","last_unstale":"2024-04-25T14:38:11.507943+0000","last_undegraded":"2024-04-25T14:38:11.507943+0000","last_fullsized":"2024-04-25T14:38:11.507943+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:02:58.942053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:14.004053+0000","last_change":"2024-04-25T14:38:09.551523+0000","last_active":"2024-04-25T14:38:14.004053+0000","last_peered":"2024-04-25T14:38:14.004053+0000","last_clean":"2024-04-25T14:38:14.004053+0000","last_became_active":"2024-04-25T14:38:09.551213+0000","last_became_peered":"2024-04-25T14:38:09.551213+0000","last_unstale":"2024-04-25T14:38:14.004053+0000","last_undegraded":"2024-04-25T14:38:14.004053+0000","last_fullsized":"2024-04-25T14:38:14.004053+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:48.948571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.539827+0000","last_change":"2024-04-25T14:38:10.420032+0000","last_active":"2024-04-25T14:38:11.539827+0000","last_peered":"2024-04-25T14:38:11.539827+0000","last_clean":"2024-04-25T14:38:11.539827+0000","last_became_active":"2024-04-25T14:38:10.418349+0000","last_became_peered":"2024-04-25T14:38:10.418349+0000","last_unstale":"2024-04-25T14:38:11.539827+0000","last_undegraded":"2024-04-25T14:38:11.539827+0000","last_fullsized":"2024-04-25T14:38:11.539827+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:38:55.669414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:14.013121+0000","last_change":"2024-04-25T14:38:09.567097+0000","last_active":"2024-04-25T14:38:14.013121+0000","last_peered":"2024-04-25T14:38:14.013121+0000","last_clean":"2024-04-25T14:38:14.013121+0000","last_became_active":"2024-04-25T14:38:09.566852+0000","last_became_peered":"2024-04-25T14:38:09.566852+0000","last_unstale":"2024-04-25T14:38:14.013121+0000","last_undegraded":"2024-04-25T14:38:14.013121+0000","last_fullsized":"2024-04-25T14:38:14.013121+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:41:55.773005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.213818+0000","last_change":"2024-04-25T14:38:10.432691+0000","last_active":"2024-04-25T14:38:12.213818+0000","last_peered":"2024-04-25T14:38:12.213818+0000","last_clean":"2024-04-25T14:38:12.213818+0000","last_became_active":"2024-04-25T14:38:10.432321+0000","last_became_peered":"2024-04-25T14:38:10.432321+0000","last_unstale":"2024-04-25T14:38:12.213818+0000","last_undegraded":"2024-04-25T14:38:12.213818+0000","last_fullsized":"2024-04-25T14:38:12.213818+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:47:58.026331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.214168+0000","last_change":"2024-04-25T14:38:09.563692+0000","last_active":"2024-04-25T14:38:12.214168+0000","last_peered":"2024-04-25T14:38:12.214168+0000","last_clean":"2024-04-25T14:38:12.214168+0000","last_became_active":"2024-04-25T14:38:09.563345+0000","last_became_peered":"2024-04-25T14:38:09.563345+0000","last_unstale":"2024-04-25T14:38:12.214168+0000","last_undegraded":"2024-04-25T14:38:12.214168+0000","last_fullsized":"2024-04-25T14:38:12.214168+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:43.221429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.420283+0000","last_change":"2024-04-25T14:38:10.442519+0000","last_active":"2024-04-25T14:38:11.420283+0000","last_peered":"2024-04-25T14:38:11.420283+0000","last_clean":"2024-04-25T14:38:11.420283+0000","last_became_active":"2024-04-25T14:38:10.439045+0000","last_became_peered":"2024-04-25T14:38:10.439045+0000","last_unstale":"2024-04-25T14:38:11.420283+0000","last_undegraded":"2024-04-25T14:38:11.420283+0000","last_fullsized":"2024-04-25T14:38:11.420283+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:42.425106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:14.005037+0000","last_change":"2024-04-25T14:38:09.550363+0000","last_active":"2024-04-25T14:38:14.005037+0000","last_peered":"2024-04-25T14:38:14.005037+0000","last_clean":"2024-04-25T14:38:14.005037+0000","last_became_active":"2024-04-25T14:38:09.550044+0000","last_became_peered":"2024-04-25T14:38:09.550044+0000","last_unstale":"2024-04-25T14:38:14.005037+0000","last_undegraded":"2024-04-25T14:38:14.005037+0000","last_fullsized":"2024-04-25T14:38:14.005037+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:25.573353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.214422+0000","last_change":"2024-04-25T14:38:10.428736+0000","last_active":"2024-04-25T14:38:12.214422+0000","last_peered":"2024-04-25T14:38:12.214422+0000","last_clean":"2024-04-25T14:38:12.214422+0000","last_became_active":"2024-04-25T14:38:10.428410+0000","last_became_peered":"2024-04-25T14:38:10.428410+0000","last_unstale":"2024-04-25T14:38:12.214422+0000","last_undegraded":"2024-04-25T14:38:12.214422+0000","last_fullsized":"2024-04-25T14:38:12.214422+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:02:51.474220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:15.029278+0000","last_change":"2024-04-25T14:38:09.552800+0000","last_active":"2024-04-25T14:38:15.029278+0000","last_peered":"2024-04-25T14:38:15.029278+0000","last_clean":"2024-04-25T14:38:15.029278+0000","last_became_active":"2024-04-25T14:38:09.552460+0000","last_became_peered":"2024-04-25T14:38:09.552460+0000","last_unstale":"2024-04-25T14:38:15.029278+0000","last_undegraded":"2024-04-25T14:38:15.029278+0000","last_fullsized":"2024-04-25T14:38:15.029278+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:00.800770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.214494+0000","last_change":"2024-04-25T14:38:10.424359+0000","last_active":"2024-04-25T14:38:12.214494+0000","last_peered":"2024-04-25T14:38:12.214494+0000","last_clean":"2024-04-25T14:38:12.214494+0000","last_became_active":"2024-04-25T14:38:10.424020+0000","last_became_peered":"2024-04-25T14:38:10.424020+0000","last_unstale":"2024-04-25T14:38:12.214494+0000","last_undegraded":"2024-04-25T14:38:12.214494+0000","last_fullsized":"2024-04-25T14:38:12.214494+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:14.700772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.991648+0000","last_change":"2024-04-25T14:38:09.545637+0000","last_active":"2024-04-25T14:38:11.991648+0000","last_peered":"2024-04-25T14:38:11.991648+0000","last_clean":"2024-04-25T14:38:11.991648+0000","last_became_active":"2024-04-25T14:38:09.545319+0000","last_became_peered":"2024-04-25T14:38:09.545319+0000","last_unstale":"2024-04-25T14:38:11.991648+0000","last_undegraded":"2024-04-25T14:38:11.991648+0000","last_fullsized":"2024-04-25T14:38:11.991648+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:02.284715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.214197+0000","last_change":"2024-04-25T14:38:10.446928+0000","last_active":"2024-04-25T14:38:12.214197+0000","last_peered":"2024-04-25T14:38:12.214197+0000","last_clean":"2024-04-25T14:38:12.214197+0000","last_became_active":"2024-04-25T14:38:10.446518+0000","last_became_peered":"2024-04-25T14:38:10.446518+0000","last_unstale":"2024-04-25T14:38:12.214197+0000","last_undegraded":"2024-04-25T14:38:12.214197+0000","last_fullsized":"2024-04-25T14:38:12.214197+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:40.769059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.760049+0000","last_change":"2024-04-25T14:38:09.557381+0000","last_active":"2024-04-25T14:38:11.760049+0000","last_peered":"2024-04-25T14:38:11.760049+0000","last_clean":"2024-04-25T14:38:11.760049+0000","last_became_active":"2024-04-25T14:38:09.556504+0000","last_became_peered":"2024-04-25T14:38:09.556504+0000","last_unstale":"2024-04-25T14:38:11.760049+0000","last_undegraded":"2024-04-25T14:38:11.760049+0000","last_fullsized":"2024-04-25T14:38:11.760049+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:01:02.761249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.214467+0000","last_change":"2024-04-25T14:38:10.418104+0000","last_active":"2024-04-25T14:38:12.214467+0000","last_peered":"2024-04-25T14:38:12.214467+0000","last_clean":"2024-04-25T14:38:12.214467+0000","last_became_active":"2024-04-25T14:38:10.417774+0000","last_became_peered":"2024-04-25T14:38:10.417774+0000","last_unstale":"2024-04-25T14:38:12.214467+0000","last_undegraded":"2024-04-25T14:38:12.214467+0000","last_fullsized":"2024-04-25T14:38:12.214467+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:32:44.229899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.213939+0000","last_change":"2024-04-25T14:38:10.423546+0000","last_active":"2024-04-25T14:38:12.213939+0000","last_peered":"2024-04-25T14:38:12.213939+0000","last_clean":"2024-04-25T14:38:12.213939+0000","last_became_active":"2024-04-25T14:38:10.423216+0000","last_became_peered":"2024-04-25T14:38:10.423216+0000","last_unstale":"2024-04-25T14:38:12.213939+0000","last_undegraded":"2024-04-25T14:38:12.213939+0000","last_fullsized":"2024-04-25T14:38:12.213939+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:58.640669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.214113+0000","last_change":"2024-04-25T14:38:10.422504+0000","last_active":"2024-04-25T14:38:12.214113+0000","last_peered":"2024-04-25T14:38:12.214113+0000","last_clean":"2024-04-25T14:38:12.214113+0000","last_became_active":"2024-04-25T14:38:10.422095+0000","last_became_peered":"2024-04-25T14:38:10.422095+0000","last_unstale":"2024-04-25T14:38:12.214113+0000","last_undegraded":"2024-04-25T14:38:12.214113+0000","last_fullsized":"2024-04-25T14:38:12.214113+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:25.795962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.214150+0000","last_change":"2024-04-25T14:38:10.447446+0000","last_active":"2024-04-25T14:38:12.214150+0000","last_peered":"2024-04-25T14:38:12.214150+0000","last_clean":"2024-04-25T14:38:12.214150+0000","last_became_active":"2024-04-25T14:38:10.447056+0000","last_became_peered":"2024-04-25T14:38:10.447056+0000","last_unstale":"2024-04-25T14:38:12.214150+0000","last_undegraded":"2024-04-25T14:38:12.214150+0000","last_fullsized":"2024-04-25T14:38:12.214150+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:40.462340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.507677+0000","last_change":"2024-04-25T14:38:10.433444+0000","last_active":"2024-04-25T14:38:11.507677+0000","last_peered":"2024-04-25T14:38:11.507677+0000","last_clean":"2024-04-25T14:38:11.507677+0000","last_became_active":"2024-04-25T14:38:10.433153+0000","last_became_peered":"2024-04-25T14:38:10.433153+0000","last_unstale":"2024-04-25T14:38:11.507677+0000","last_undegraded":"2024-04-25T14:38:11.507677+0000","last_fullsized":"2024-04-25T14:38:11.507677+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:10.722650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.759790+0000","last_change":"2024-04-25T14:38:10.438322+0000","last_active":"2024-04-25T14:38:11.759790+0000","last_peered":"2024-04-25T14:38:11.759790+0000","last_clean":"2024-04-25T14:38:11.759790+0000","last_became_active":"2024-04-25T14:38:10.436103+0000","last_became_peered":"2024-04-25T14:38:10.436103+0000","last_unstale":"2024-04-25T14:38:11.759790+0000","last_undegraded":"2024-04-25T14:38:11.759790+0000","last_fullsized":"2024-04-25T14:38:11.759790+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:16.790202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.214092+0000","last_change":"2024-04-25T14:38:10.419710+0000","last_active":"2024-04-25T14:38:12.214092+0000","last_peered":"2024-04-25T14:38:12.214092+0000","last_clean":"2024-04-25T14:38:12.214092+0000","last_became_active":"2024-04-25T14:38:10.419406+0000","last_became_peered":"2024-04-25T14:38:10.419406+0000","last_unstale":"2024-04-25T14:38:12.214092+0000","last_undegraded":"2024-04-25T14:38:12.214092+0000","last_fullsized":"2024-04-25T14:38:12.214092+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:37.964698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.214679+0000","last_change":"2024-04-25T14:38:10.429510+0000","last_active":"2024-04-25T14:38:12.214679+0000","last_peered":"2024-04-25T14:38:12.214679+0000","last_clean":"2024-04-25T14:38:12.214679+0000","last_became_active":"2024-04-25T14:38:10.429219+0000","last_became_peered":"2024-04-25T14:38:10.429219+0000","last_unstale":"2024-04-25T14:38:12.214679+0000","last_undegraded":"2024-04-25T14:38:12.214679+0000","last_fullsized":"2024-04-25T14:38:12.214679+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:52:46.673126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.421361+0000","last_change":"2024-04-25T14:38:10.441487+0000","last_active":"2024-04-25T14:38:11.421361+0000","last_peered":"2024-04-25T14:38:11.421361+0000","last_clean":"2024-04-25T14:38:11.421361+0000","last_became_active":"2024-04-25T14:38:10.441111+0000","last_became_peered":"2024-04-25T14:38:10.441111+0000","last_unstale":"2024-04-25T14:38:11.421361+0000","last_undegraded":"2024-04-25T14:38:11.421361+0000","last_fullsized":"2024-04-25T14:38:11.421361+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:59.842687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.454563+0000","last_change":"2024-04-25T14:38:10.436511+0000","last_active":"2024-04-25T14:38:11.454563+0000","last_peered":"2024-04-25T14:38:11.454563+0000","last_clean":"2024-04-25T14:38:11.454563+0000","last_became_active":"2024-04-25T14:38:10.436171+0000","last_became_peered":"2024-04-25T14:38:10.436171+0000","last_unstale":"2024-04-25T14:38:11.454563+0000","last_undegraded":"2024-04-25T14:38:11.454563+0000","last_fullsized":"2024-04-25T14:38:11.454563+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:27:19.605907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.420047+0000","last_change":"2024-04-25T14:38:10.433624+0000","last_active":"2024-04-25T14:38:11.420047+0000","last_peered":"2024-04-25T14:38:11.420047+0000","last_clean":"2024-04-25T14:38:11.420047+0000","last_became_active":"2024-04-25T14:38:10.433258+0000","last_became_peered":"2024-04-25T14:38:10.433258+0000","last_unstale":"2024-04-25T14:38:11.420047+0000","last_undegraded":"2024-04-25T14:38:11.420047+0000","last_fullsized":"2024-04-25T14:38:11.420047+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:53.563110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.458280+0000","last_change":"2024-04-25T14:38:10.446924+0000","last_active":"2024-04-25T14:38:11.458280+0000","last_peered":"2024-04-25T14:38:11.458280+0000","last_clean":"2024-04-25T14:38:11.458280+0000","last_became_active":"2024-04-25T14:38:10.446553+0000","last_became_peered":"2024-04-25T14:38:10.446553+0000","last_unstale":"2024-04-25T14:38:11.458280+0000","last_undegraded":"2024-04-25T14:38:11.458280+0000","last_fullsized":"2024-04-25T14:38:11.458280+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:00.203796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:11.759908+0000","last_change":"2024-04-25T14:38:10.429952+0000","last_active":"2024-04-25T14:38:11.759908+0000","last_peered":"2024-04-25T14:38:11.759908+0000","last_clean":"2024-04-25T14:38:11.759908+0000","last_became_active":"2024-04-25T14:38:10.429626+0000","last_became_peered":"2024-04-25T14:38:10.429626+0000","last_unstale":"2024-04-25T14:38:11.759908+0000","last_undegraded":"2024-04-25T14:38:11.759908+0000","last_fullsized":"2024-04-25T14:38:11.759908+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:07.287113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T14:38:12.214245+0000","last_change":"2024-04-25T14:38:10.418292+0000","last_active":"2024-04-25T14:38:12.214245+0000","last_peered":"2024-04-25T14:38:12.214245+0000","last_clean":"2024-04-25T14:38:12.214245+0000","last_became_active":"2024-04-25T14:38:10.418005+0000","last_became_peered":"2024-04-25T14:38:10.418005+0000","last_unstale":"2024-04-25T14:38:12.214245+0000","last_undegraded":"2024-04-25T14:38:12.214245+0000","last_fullsized":"2024-04-25T14:38:12.214245+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:25.486993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4991,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":15,"num_read_kb":13,"num_write":114,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":249856,"data_stored":35075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285584,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":452438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547220,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1585152,"data_stored":1045414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710745,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":452438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972385,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":452594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266724,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28532,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716620,"statfs":{"total":95995035648,"available":95965818880,"internally_reserved":0,"allocated":1552384,"data_stored":1043428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430249,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":451946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":453468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":974848,"data_stored":453334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411394,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":987136,"data_stored":461809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149755,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":452140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888119,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":950272,"data_stored":451878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691889,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32708,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712444,"statfs":{"total":95995035648,"available":95961542656,"internally_reserved":0,"allocated":1568768,"data_stored":1043700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":22636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:38:20.178 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T14:38:20.179 INFO:tasks.cephadm:Setting up client nodes... 2024-04-25T14:38:20.179 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-25T14:38:20.294 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:20 smithi053 ceph-mon[27773]: from='client.24718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:38:20.294 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:20 smithi053 ceph-mon[27773]: pgmap v320: 97 pgs: 97 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 4.6 KiB/s wr, 16 op/s 2024-04-25T14:38:20.295 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:20 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/855048276' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:38:20.362 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:20 smithi150 ceph-mon[32504]: from='client.24718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:38:20.362 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:20 smithi150 ceph-mon[32504]: pgmap v320: 97 pgs: 97 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 4.6 KiB/s wr, 16 op/s 2024-04-25T14:38:20.362 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:20 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/855048276' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:38:20.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:20 smithi183 ceph-mon[32584]: from='client.24718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:38:20.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:20 smithi183 ceph-mon[32584]: pgmap v320: 97 pgs: 97 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 4.6 KiB/s wr, 16 op/s 2024-04-25T14:38:20.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:20 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/855048276' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:38:20.648 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 25 14:38:20 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-b[62911]: 2024-04-25T14:38:20.501+0000 7fb568e9d700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-25T14:38:20.690 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:38:21.340 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:21 smithi053 ceph-mon[27773]: from='client.14598 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:38:21.340 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:21 smithi053 ceph-mon[27773]: from='client.24722 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:38:21.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:21 smithi183 ceph-mon[32584]: from='client.14598 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:38:21.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:21 smithi183 ceph-mon[32584]: from='client.24722 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:38:21.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:21 smithi150 ceph-mon[32504]: from='client.14598 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:38:21.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:21 smithi150 ceph-mon[32504]: from='client.24722 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:38:22.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:22 smithi183 ceph-mon[32584]: pgmap v321: 97 pgs: 97 active+clean; 624 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 9.0 KiB/s wr, 19 op/s 2024-04-25T14:38:22.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:22 smithi150 ceph-mon[32504]: pgmap v321: 97 pgs: 97 active+clean; 624 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 9.0 KiB/s wr, 19 op/s 2024-04-25T14:38:22.487 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:22 smithi053 ceph-mon[27773]: pgmap v321: 97 pgs: 97 active+clean; 624 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 9.0 KiB/s wr, 19 op/s 2024-04-25T14:38:23.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.027+0000 7f449da00700 1 -- 172.21.15.53:0/2118104852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44981087c0 msgr2=0x7f449810ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:23.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.027+0000 7f449da00700 1 --2- 172.21.15.53:0/2118104852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44981087c0 0x7f449810ac30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f449000b0a0 tx=0x7f449000b3b0 comp rx=0 tx=0).stop 2024-04-25T14:38:23.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.028+0000 7f449da00700 1 -- 172.21.15.53:0/2118104852 shutdown_connections 2024-04-25T14:38:23.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.028+0000 7f449da00700 1 --2- 172.21.15.53:0/2118104852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44981087c0 0x7f449810ac30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:23.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.028+0000 7f449da00700 1 --2- 172.21.15.53:0/2118104852 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4498103560 0x7f4498105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:23.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.028+0000 7f449da00700 1 --2- 172.21.15.53:0/2118104852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4498105e90 0x7f4498108280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:23.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.028+0000 7f449da00700 1 -- 172.21.15.53:0/2118104852 >> 172.21.15.53:0/2118104852 conn(0x7f44980fcf90 msgr2=0x7f44980ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:23.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.028+0000 7f449da00700 1 -- 172.21.15.53:0/2118104852 shutdown_connections 2024-04-25T14:38:23.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.028+0000 7f449da00700 1 -- 172.21.15.53:0/2118104852 wait complete. 2024-04-25T14:38:23.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.029+0000 7f449da00700 1 Processor -- start 2024-04-25T14:38:23.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.029+0000 7f449da00700 1 -- start start 2024-04-25T14:38:23.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.030+0000 7f449da00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4498103560 0x7f449819acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:23.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.030+0000 7f449da00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4498105e90 0x7f449819b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:23.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.030+0000 7f449da00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44981087c0 0x7f44981a2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:23.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.030+0000 7f449da00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f449810e550 con 0x7f4498103560 2024-04-25T14:38:23.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.030+0000 7f449da00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f449810e3d0 con 0x7f44981087c0 2024-04-25T14:38:23.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.030+0000 7f449da00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f449810e6d0 con 0x7f4498105e90 2024-04-25T14:38:23.031 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.030+0000 7f4496a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4498103560 0x7f449819acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:23.031 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.030+0000 7f4496000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4498105e90 0x7f449819b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:23.031 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.030+0000 7f4496a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4498103560 0x7f449819acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53224/0 (socket says 172.21.15.53:53224) 2024-04-25T14:38:23.031 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.030+0000 7f4496a00700 1 -- 172.21.15.53:0/1871547468 learned_addr learned my addr 172.21.15.53:0/1871547468 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:38:23.031 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.030+0000 7f4497400700 1 --2- 172.21.15.53:0/1871547468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44981087c0 0x7f44981a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:23.031 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.031+0000 7f4497400700 1 -- 172.21.15.53:0/1871547468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4498105e90 msgr2=0x7f449819b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:23.031 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.031+0000 7f4497400700 1 --2- 172.21.15.53:0/1871547468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4498105e90 0x7f449819b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:23.031 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.031+0000 7f4497400700 1 -- 172.21.15.53:0/1871547468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4498103560 msgr2=0x7f449819acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:23.031 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.031+0000 7f4497400700 1 --2- 172.21.15.53:0/1871547468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4498103560 0x7f449819acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:23.031 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.031+0000 7f4497400700 1 -- 172.21.15.53:0/1871547468 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44981a2930 con 0x7f44981087c0 2024-04-25T14:38:23.031 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.031+0000 7f4497400700 1 --2- 172.21.15.53:0/1871547468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44981087c0 0x7f44981a2260 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4490006130 tx=0x7f4490004ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:23.032 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.031+0000 7f448f400700 1 -- 172.21.15.53:0/1871547468 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4490021070 con 0x7f44981087c0 2024-04-25T14:38:23.032 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.032+0000 7f449da00700 1 -- 172.21.15.53:0/1871547468 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44981a2bc0 con 0x7f44981087c0 2024-04-25T14:38:23.032 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.032+0000 7f448f400700 1 -- 172.21.15.53:0/1871547468 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4490007b80 con 0x7f44981087c0 2024-04-25T14:38:23.032 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.032+0000 7f448f400700 1 -- 172.21.15.53:0/1871547468 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4490005040 con 0x7f44981087c0 2024-04-25T14:38:23.032 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.032+0000 7f449da00700 1 -- 172.21.15.53:0/1871547468 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44981a3150 con 0x7f44981087c0 2024-04-25T14:38:23.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.033+0000 7f449da00700 1 -- 172.21.15.53:0/1871547468 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4498050f80 con 0x7f44981087c0 2024-04-25T14:38:23.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.035+0000 7f448f400700 1 -- 172.21.15.53:0/1871547468 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f449001d450 con 0x7f44981087c0 2024-04-25T14:38:23.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.036+0000 7f448f400700 1 --2- 172.21.15.53:0/1871547468 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f447c0a3e00 0x7f447c0a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:23.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.036+0000 7f448f400700 1 -- 172.21.15.53:0/1871547468 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f449000f020 con 0x7f44981087c0 2024-04-25T14:38:23.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.037+0000 7f4496a00700 1 --2- 172.21.15.53:0/1871547468 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f447c0a3e00 0x7f447c0a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:23.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.038+0000 7f4496a00700 1 --2- 172.21.15.53:0/1871547468 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f447c0a3e00 0x7f447c0a62c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f449819bb90 tx=0x7f4488009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:23.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.039+0000 7f448f400700 1 -- 172.21.15.53:0/1871547468 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f449002a030 con 0x7f44981087c0 2024-04-25T14:38:23.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.371+0000 7f449da00700 1 -- 172.21.15.53:0/1871547468 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f4498063650 con 0x7f44981087c0 2024-04-25T14:38:23.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.380+0000 7f448f400700 1 -- 172.21.15.53:0/1871547468 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f449009df50 con 0x7f44981087c0 2024-04-25T14:38:23.380 INFO:teuthology.orchestra.run.smithi053.stdout:[client.0] 2024-04-25T14:38:23.380 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQDfaipmM8ZLFhAAwt5ixG0yUc+YXl+l7JFhfg== 2024-04-25T14:38:23.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.383+0000 7f449da00700 1 -- 172.21.15.53:0/1871547468 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f447c0a3e00 msgr2=0x7f447c0a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:23.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.383+0000 7f449da00700 1 --2- 172.21.15.53:0/1871547468 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f447c0a3e00 0x7f447c0a62c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f449819bb90 tx=0x7f4488009260 comp rx=0 tx=0).stop 2024-04-25T14:38:23.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.383+0000 7f449da00700 1 -- 172.21.15.53:0/1871547468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44981087c0 msgr2=0x7f44981a2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:23.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.383+0000 7f449da00700 1 --2- 172.21.15.53:0/1871547468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44981087c0 0x7f44981a2260 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4490006130 tx=0x7f4490004ce0 comp rx=0 tx=0).stop 2024-04-25T14:38:23.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.383+0000 7f449da00700 1 -- 172.21.15.53:0/1871547468 shutdown_connections 2024-04-25T14:38:23.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.383+0000 7f449da00700 1 --2- 172.21.15.53:0/1871547468 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f447c0a3e00 0x7f447c0a62c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:23.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.383+0000 7f449da00700 1 --2- 172.21.15.53:0/1871547468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44981087c0 0x7f44981a2260 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:23.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.383+0000 7f449da00700 1 --2- 172.21.15.53:0/1871547468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4498103560 0x7f449819acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:23.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.383+0000 7f449da00700 1 --2- 172.21.15.53:0/1871547468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4498105e90 0x7f449819b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:23.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.383+0000 7f449da00700 1 -- 172.21.15.53:0/1871547468 >> 172.21.15.53:0/1871547468 conn(0x7f44980fcf90 msgr2=0x7f4498102bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:23.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.383+0000 7f449da00700 1 -- 172.21.15.53:0/1871547468 shutdown_connections 2024-04-25T14:38:23.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:23.384+0000 7f449da00700 1 -- 172.21.15.53:0/1871547468 wait complete. 2024-04-25T14:38:24.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:24 smithi053 ceph-mon[27773]: pgmap v322: 97 pgs: 97 active+clean; 624 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 8.3 KiB/s wr, 18 op/s 2024-04-25T14:38:24.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:24 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1871547468' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T14:38:24.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:24 smithi053 ceph-mon[27773]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T14:38:24.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:24 smithi053 ceph-mon[27773]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T14:38:24.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:24 smithi183 ceph-mon[32584]: pgmap v322: 97 pgs: 97 active+clean; 624 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 8.3 KiB/s wr, 18 op/s 2024-04-25T14:38:24.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:24 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/1871547468' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T14:38:24.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:24 smithi183 ceph-mon[32584]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T14:38:24.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:24 smithi183 ceph-mon[32584]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T14:38:24.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:24 smithi150 ceph-mon[32504]: pgmap v322: 97 pgs: 97 active+clean; 624 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 8.3 KiB/s wr, 18 op/s 2024-04-25T14:38:24.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:24 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/1871547468' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T14:38:24.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:24 smithi150 ceph-mon[32504]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T14:38:24.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:24 smithi150 ceph-mon[32504]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T14:38:24.461 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-25T14:38:24.461 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-25T14:38:24.461 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T14:38:24.605 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T14:38:24.605 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-25T14:38:24.605 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-25T14:38:25.109 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:38:26.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:26 smithi053 ceph-mon[27773]: pgmap v323: 97 pgs: 97 active+clean; 624 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.7 KiB/s wr, 14 op/s 2024-04-25T14:38:26.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:26 smithi183 ceph-mon[32584]: pgmap v323: 97 pgs: 97 active+clean; 624 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.7 KiB/s wr, 14 op/s 2024-04-25T14:38:26.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:26 smithi150 ceph-mon[32504]: pgmap v323: 97 pgs: 97 active+clean; 624 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.7 KiB/s wr, 14 op/s 2024-04-25T14:38:27.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.320+0000 7f7996000700 1 -- 172.21.15.53:0/2948548375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7990103450 msgr2=0x7f799010f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:27.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.320+0000 7f7996000700 1 --2- 172.21.15.53:0/2948548375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7990103450 0x7f799010f8b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f798800b0a0 tx=0x7f798800b3b0 comp rx=0 tx=0).stop 2024-04-25T14:38:27.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.320+0000 7f7996000700 1 -- 172.21.15.53:0/2948548375 shutdown_connections 2024-04-25T14:38:27.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.320+0000 7f7996000700 1 --2- 172.21.15.53:0/2948548375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7990103450 0x7f799010f8b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:27.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.320+0000 7f7996000700 1 --2- 172.21.15.53:0/2948548375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7990108a50 0x7f7990108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:27.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.320+0000 7f7996000700 1 --2- 172.21.15.53:0/2948548375 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7990102a90 0x7f7990102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:27.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.320+0000 7f7996000700 1 -- 172.21.15.53:0/2948548375 >> 172.21.15.53:0/2948548375 conn(0x7f7990079ba0 msgr2=0x7f7990079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:27.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.320+0000 7f7996000700 1 -- 172.21.15.53:0/2948548375 shutdown_connections 2024-04-25T14:38:27.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.320+0000 7f7996000700 1 -- 172.21.15.53:0/2948548375 wait complete. 2024-04-25T14:38:27.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.322+0000 7f7996000700 1 Processor -- start 2024-04-25T14:38:27.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.322+0000 7f7996000700 1 -- start start 2024-04-25T14:38:27.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.322+0000 7f7996000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7990102a90 0x7f799019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:27.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.322+0000 7f7996000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7990103450 0x7f799019b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:27.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.322+0000 7f7996000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7990108a50 0x7f799019fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:27.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.322+0000 7f7996000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7990112c50 con 0x7f7990108a50 2024-04-25T14:38:27.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.322+0000 7f7996000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7990112ad0 con 0x7f7990102a90 2024-04-25T14:38:27.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.322+0000 7f7996000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7990112dd0 con 0x7f7990103450 2024-04-25T14:38:27.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.323+0000 7f798fe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7990108a50 0x7f799019fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:27.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.323+0000 7f798f400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7990102a90 0x7f799019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:27.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.323+0000 7f798f400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7990102a90 0x7f799019b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.53:36388/0 (socket says 172.21.15.53:36388) 2024-04-25T14:38:27.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.323+0000 7f798f400700 1 -- 172.21.15.53:0/3278325029 learned_addr learned my addr 172.21.15.53:0/3278325029 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:38:27.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.323+0000 7f7987e00700 1 --2- 172.21.15.53:0/3278325029 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7990103450 0x7f799019b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:27.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.323+0000 7f798f400700 1 -- 172.21.15.53:0/3278325029 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7990103450 msgr2=0x7f799019b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:27.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.323+0000 7f798f400700 1 --2- 172.21.15.53:0/3278325029 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7990103450 0x7f799019b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:27.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.323+0000 7f798f400700 1 -- 172.21.15.53:0/3278325029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7990108a50 msgr2=0x7f799019fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:27.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.323+0000 7f798f400700 1 --2- 172.21.15.53:0/3278325029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7990108a50 0x7f799019fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:27.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.323+0000 7f798f400700 1 -- 172.21.15.53:0/3278325029 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79901a0370 con 0x7f7990102a90 2024-04-25T14:38:27.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.323+0000 7f798fe00700 1 --2- 172.21.15.53:0/3278325029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7990108a50 0x7f799019fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:38:27.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.323+0000 7f7987e00700 1 --2- 172.21.15.53:0/3278325029 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7990103450 0x7f799019b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:38:27.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.324+0000 7f798f400700 1 --2- 172.21.15.53:0/3278325029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7990102a90 0x7f799019b360 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f798000e9a0 tx=0x7f798000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:27.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.324+0000 7f798cc00700 1 -- 172.21.15.53:0/3278325029 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798000ca10 con 0x7f7990102a90 2024-04-25T14:38:27.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.324+0000 7f7996000700 1 -- 172.21.15.53:0/3278325029 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79901a0660 con 0x7f7990102a90 2024-04-25T14:38:27.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.324+0000 7f798cc00700 1 -- 172.21.15.53:0/3278325029 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f798000cbb0 con 0x7f7990102a90 2024-04-25T14:38:27.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.324+0000 7f7996000700 1 -- 172.21.15.53:0/3278325029 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79901a7ee0 con 0x7f7990102a90 2024-04-25T14:38:27.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.324+0000 7f798cc00700 1 -- 172.21.15.53:0/3278325029 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7980018420 con 0x7f7990102a90 2024-04-25T14:38:27.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.325+0000 7f7996000700 1 -- 172.21.15.53:0/3278325029 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f796c005330 con 0x7f7990102a90 2024-04-25T14:38:27.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.327+0000 7f798cc00700 1 -- 172.21.15.53:0/3278325029 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7980010430 con 0x7f7990102a90 2024-04-25T14:38:27.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.328+0000 7f798cc00700 1 --2- 172.21.15.53:0/3278325029 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f79740a3cd0 0x7f79740a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:27.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.329+0000 7f798cc00700 1 -- 172.21.15.53:0/3278325029 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f79800d0550 con 0x7f7990102a90 2024-04-25T14:38:27.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.329+0000 7f7987e00700 1 --2- 172.21.15.53:0/3278325029 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f79740a3cd0 0x7f79740a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:27.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.329+0000 7f7987e00700 1 --2- 172.21.15.53:0/3278325029 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f79740a3cd0 0x7f79740a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f799010fcf0 tx=0x7f7978009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:27.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.333+0000 7f798cc00700 1 -- 172.21.15.53:0/3278325029 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f798009ce80 con 0x7f7990102a90 2024-04-25T14:38:27.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.626+0000 7f7996000700 1 -- 172.21.15.53:0/3278325029 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f796c0058f0 con 0x7f7990102a90 2024-04-25T14:38:27.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.632+0000 7f798cc00700 1 -- 172.21.15.53:0/3278325029 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264996 (secure 0 0 0) 0x7f798009c780 con 0x7f7990102a90 2024-04-25T14:38:27.636 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:38:27.638 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.637+0000 7f7996000700 1 -- 172.21.15.53:0/3278325029 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f79740a3cd0 msgr2=0x7f79740a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:27.638 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.638+0000 7f7996000700 1 --2- 172.21.15.53:0/3278325029 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f79740a3cd0 0x7f79740a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f799010fcf0 tx=0x7f7978009260 comp rx=0 tx=0).stop 2024-04-25T14:38:27.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.638+0000 7f7996000700 1 -- 172.21.15.53:0/3278325029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7990102a90 msgr2=0x7f799019b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:27.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.638+0000 7f7996000700 1 --2- 172.21.15.53:0/3278325029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7990102a90 0x7f799019b360 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f798000e9a0 tx=0x7f798000ecb0 comp rx=0 tx=0).stop 2024-04-25T14:38:27.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.638+0000 7f7996000700 1 -- 172.21.15.53:0/3278325029 shutdown_connections 2024-04-25T14:38:27.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.638+0000 7f7996000700 1 --2- 172.21.15.53:0/3278325029 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f79740a3cd0 0x7f79740a6190 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:27.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.638+0000 7f7996000700 1 --2- 172.21.15.53:0/3278325029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7990108a50 0x7f799019fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:27.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.638+0000 7f7996000700 1 --2- 172.21.15.53:0/3278325029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7990102a90 0x7f799019b360 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:27.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.638+0000 7f7996000700 1 --2- 172.21.15.53:0/3278325029 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7990103450 0x7f799019b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:27.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.638+0000 7f7996000700 1 -- 172.21.15.53:0/3278325029 >> 172.21.15.53:0/3278325029 conn(0x7f7990079ba0 msgr2=0x7f7990101540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:27.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.638+0000 7f7996000700 1 -- 172.21.15.53:0/3278325029 shutdown_connections 2024-04-25T14:38:27.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:27.638+0000 7f7996000700 1 -- 172.21.15.53:0/3278325029 wait complete. 2024-04-25T14:38:28.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:28 smithi053 ceph-mon[27773]: pgmap v324: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.5 KiB/s wr, 16 op/s 2024-04-25T14:38:28.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:28 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/3278325029' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T14:38:28.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:28 smithi183 ceph-mon[32584]: pgmap v324: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.5 KiB/s wr, 16 op/s 2024-04-25T14:38:28.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:28 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/3278325029' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T14:38:28.456 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":4154933108},{"type":"v1","addr":"172.21.15.53:6801","nonce":4154933108}]},"active_addr":"172.21.15.53:6801/4154933108","active_change":"2024-04-25T14:30:14.162641+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.53:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.53:0","nonce":920465993}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.53:0","nonce":3916258673}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.53:0","nonce":4267178590}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.53:0","nonce":309091034}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.53:0","nonce":1015301915}]}]} 2024-04-25T14:38:28.462 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-25T14:38:28.462 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-25T14:38:28.462 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T14:38:28.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:28 smithi150 ceph-mon[32504]: pgmap v324: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.5 KiB/s wr, 16 op/s 2024-04-25T14:38:28.463 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:28 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/3278325029' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T14:38:28.920 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:38:30.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:30 smithi053 ceph-mon[27773]: pgmap v325: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.9 KiB/s wr, 13 op/s 2024-04-25T14:38:30.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:30 smithi183 ceph-mon[32584]: pgmap v325: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.9 KiB/s wr, 13 op/s 2024-04-25T14:38:30.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:30 smithi150 ceph-mon[32504]: pgmap v325: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.9 KiB/s wr, 13 op/s 2024-04-25T14:38:31.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.219+0000 7f49fdc00700 1 -- 172.21.15.53:0/2936933856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49f8108f80 msgr2=0x7f49f8109400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:31.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.219+0000 7f49fdc00700 1 --2- 172.21.15.53:0/2936933856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49f8108f80 0x7f49f8109400 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f49e4009990 tx=0x7f49e4009ca0 comp rx=0 tx=0).stop 2024-04-25T14:38:31.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.219+0000 7f49fdc00700 1 -- 172.21.15.53:0/2936933856 shutdown_connections 2024-04-25T14:38:31.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.220+0000 7f49fdc00700 1 --2- 172.21.15.53:0/2936933856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49f8109940 0x7f49f8115de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:31.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.220+0000 7f49fdc00700 1 --2- 172.21.15.53:0/2936933856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49f810f030 0x7f49f810f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:31.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.220+0000 7f49fdc00700 1 --2- 172.21.15.53:0/2936933856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49f8108f80 0x7f49f8109400 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:31.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.220+0000 7f49fdc00700 1 -- 172.21.15.53:0/2936933856 >> 172.21.15.53:0/2936933856 conn(0x7f49f80787d0 msgr2=0x7f49f8107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:31.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.220+0000 7f49fdc00700 1 -- 172.21.15.53:0/2936933856 shutdown_connections 2024-04-25T14:38:31.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.220+0000 7f49fdc00700 1 -- 172.21.15.53:0/2936933856 wait complete. 2024-04-25T14:38:31.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.221+0000 7f49fdc00700 1 Processor -- start 2024-04-25T14:38:31.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.221+0000 7f49fdc00700 1 -- start start 2024-04-25T14:38:31.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.222+0000 7f49fdc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49f8108f80 0x7f49f819f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:31.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.222+0000 7f49fdc00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49f8109940 0x7f49f819fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:31.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.222+0000 7f49fdc00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49f810f030 0x7f49f81a3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:31.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.222+0000 7f49fdc00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f49f8119120 con 0x7f49f8108f80 2024-04-25T14:38:31.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.222+0000 7f49fdc00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f49f8118fa0 con 0x7f49f8109940 2024-04-25T14:38:31.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.222+0000 7f49fdc00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f49f81192a0 con 0x7f49f810f030 2024-04-25T14:38:31.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.222+0000 7f49f7e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49f8108f80 0x7f49f819f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:31.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.222+0000 7f49f7e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49f8108f80 0x7f49f819f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53254/0 (socket says 172.21.15.53:53254) 2024-04-25T14:38:31.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.222+0000 7f49f7e00700 1 -- 172.21.15.53:0/2564742915 learned_addr learned my addr 172.21.15.53:0/2564742915 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:38:31.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.222+0000 7f49f7400700 1 --2- 172.21.15.53:0/2564742915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49f8109940 0x7f49f819fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:31.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.222+0000 7f49fd200700 1 --2- 172.21.15.53:0/2564742915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49f810f030 0x7f49f81a3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:31.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.222+0000 7f49f7e00700 1 -- 172.21.15.53:0/2564742915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49f810f030 msgr2=0x7f49f81a3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:31.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.222+0000 7f49f7e00700 1 --2- 172.21.15.53:0/2564742915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49f810f030 0x7f49f81a3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:31.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.222+0000 7f49f7e00700 1 -- 172.21.15.53:0/2564742915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49f8109940 msgr2=0x7f49f819fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:31.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.222+0000 7f49f7e00700 1 --2- 172.21.15.53:0/2564742915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49f8109940 0x7f49f819fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:31.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.222+0000 7f49f7e00700 1 -- 172.21.15.53:0/2564742915 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f81a4580 con 0x7f49f8108f80 2024-04-25T14:38:31.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.223+0000 7f49f7400700 1 --2- 172.21.15.53:0/2564742915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49f8109940 0x7f49f819fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:38:31.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.223+0000 7f49f7e00700 1 --2- 172.21.15.53:0/2564742915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49f8108f80 0x7f49f819f570 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f49e800eb30 tx=0x7f49e800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:31.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.223+0000 7f49f4c00700 1 -- 172.21.15.53:0/2564742915 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e800cbd0 con 0x7f49f8108f80 2024-04-25T14:38:31.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.223+0000 7f49fdc00700 1 -- 172.21.15.53:0/2564742915 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49f81a4870 con 0x7f49f8108f80 2024-04-25T14:38:31.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.223+0000 7f49f4c00700 1 -- 172.21.15.53:0/2564742915 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49e80044d0 con 0x7f49f8108f80 2024-04-25T14:38:31.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.223+0000 7f49f4c00700 1 -- 172.21.15.53:0/2564742915 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e8003e30 con 0x7f49f8108f80 2024-04-25T14:38:31.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.223+0000 7f49fdc00700 1 -- 172.21.15.53:0/2564742915 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49f81ac0f0 con 0x7f49f8108f80 2024-04-25T14:38:31.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.224+0000 7f49fdc00700 1 -- 172.21.15.53:0/2564742915 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49f8069730 con 0x7f49f8108f80 2024-04-25T14:38:31.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.227+0000 7f49f4c00700 1 -- 172.21.15.53:0/2564742915 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f49e8004000 con 0x7f49f8108f80 2024-04-25T14:38:31.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.228+0000 7f49f4c00700 1 --2- 172.21.15.53:0/2564742915 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f49dc0a3cd0 0x7f49dc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:31.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.228+0000 7f49f4c00700 1 -- 172.21.15.53:0/2564742915 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f49e80d0830 con 0x7f49f8108f80 2024-04-25T14:38:31.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.229+0000 7f49f7400700 1 --2- 172.21.15.53:0/2564742915 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f49dc0a3cd0 0x7f49dc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:31.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.229+0000 7f49f7400700 1 --2- 172.21.15.53:0/2564742915 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f49dc0a3cd0 0x7f49dc0a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f49e40098e0 tx=0x7f49e4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:31.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.233+0000 7f49f4c00700 1 -- 172.21.15.53:0/2564742915 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f49e80042f0 con 0x7f49f8108f80 2024-04-25T14:38:31.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.463+0000 7f49fdc00700 1 -- 172.21.15.53:0/2564742915 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f49f81a08d0 con 0x7f49f8108f80 2024-04-25T14:38:31.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.468+0000 7f49f4c00700 1 -- 172.21.15.53:0/2564742915 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22171 (secure 0 0 0) 0x7f49e809d160 con 0x7f49f8108f80 2024-04-25T14:38:31.468 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:38:31.468 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":88,"fsid":"fd30e6ce-030f-11ef-bc93-c7b262605968","created":"2024-04-25T14:28:54.330326+0000","modified":"2024-04-25T14:38:11.415753+0000","last_up_change":"2024-04-25T14:37:10.921148+0000","last_in_change":"2024-04-25T14:36:49.566459+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:33:02.338115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:38:07.517974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:38:08.994122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aecfbaf0-39da-4749-a49b-c7205dabd4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6803","nonce":464176015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6805","nonce":464176015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6809","nonce":464176015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6807","nonce":464176015}]},"public_addr":"172.21.15.53:6803/464176015","cluster_addr":"172.21.15.53:6805/464176015","heartbeat_back_addr":"172.21.15.53:6809/464176015","heartbeat_front_addr":"172.21.15.53:6807/464176015","state":["exists","up"]},{"osd":1,"uuid":"2c4a21ae-c43e-4839-8a56-c3fc63aeb7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6801","nonce":1304621982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6803","nonce":1304621982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6807","nonce":1304621982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6805","nonce":1304621982}]},"public_addr":"172.21.15.150:6801/1304621982","cluster_addr":"172.21.15.150:6803/1304621982","heartbeat_back_addr":"172.21.15.150:6807/1304621982","heartbeat_front_addr":"172.21.15.150:6805/1304621982","state":["exists","up"]},{"osd":2,"uuid":"e8effd43-adc0-472a-9eed-a9273b06dc0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6801","nonce":574509311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6803","nonce":574509311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6807","nonce":574509311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6805","nonce":574509311}]},"public_addr":"172.21.15.183:6801/574509311","cluster_addr":"172.21.15.183:6803/574509311","heartbeat_back_addr":"172.21.15.183:6807/574509311","heartbeat_front_addr":"172.21.15.183:6805/574509311","state":["exists","up"]},{"osd":3,"uuid":"e7cfced4-9b40-40de-a303-75ea8d5df4a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6811","nonce":570672219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6813","nonce":570672219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6817","nonce":570672219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6815","nonce":570672219}]},"public_addr":"172.21.15.53:6811/570672219","cluster_addr":"172.21.15.53:6813/570672219","heartbeat_back_addr":"172.21.15.53:6817/570672219","heartbeat_front_addr":"172.21.15.53:6815/570672219","state":["exists","up"]},{"osd":4,"uuid":"44a9e070-2282-43f1-9f05-b1f8ff81e540","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6809","nonce":1526850103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6811","nonce":1526850103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6815","nonce":1526850103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6813","nonce":1526850103}]},"public_addr":"172.21.15.150:6809/1526850103","cluster_addr":"172.21.15.150:6811/1526850103","heartbeat_back_addr":"172.21.15.150:6815/1526850103","heartbeat_front_addr":"172.21.15.150:6813/1526850103","state":["exists","up"]},{"osd":5,"uuid":"2c5619d2-c0f9-47d6-bf7e-2f4afd1b09f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6809","nonce":773726576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6811","nonce":773726576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6815","nonce":773726576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6813","nonce":773726576}]},"public_addr":"172.21.15.183:6809/773726576","cluster_addr":"172.21.15.183:6811/773726576","heartbeat_back_addr":"172.21.15.183:6815/773726576","heartbeat_front_addr":"172.21.15.183:6813/773726576","state":["exists","up"]},{"osd":6,"uuid":"1cf74d94-c118-4738-b6db-040af458a8eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6819","nonce":355884319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6821","nonce":355884319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6825","nonce":355884319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6823","nonce":355884319}]},"public_addr":"172.21.15.53:6819/355884319","cluster_addr":"172.21.15.53:6821/355884319","heartbeat_back_addr":"172.21.15.53:6825/355884319","heartbeat_front_addr":"172.21.15.53:6823/355884319","state":["exists","up"]},{"osd":7,"uuid":"0ce97c04-cd41-4080-bbbc-be9cf5fadc02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6817","nonce":229022562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6819","nonce":229022562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6823","nonce":229022562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6821","nonce":229022562}]},"public_addr":"172.21.15.150:6817/229022562","cluster_addr":"172.21.15.150:6819/229022562","heartbeat_back_addr":"172.21.15.150:6823/229022562","heartbeat_front_addr":"172.21.15.150:6821/229022562","state":["exists","up"]},{"osd":8,"uuid":"5d57787d-2a7c-4514-bd01-357ced413f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6817","nonce":301003844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6819","nonce":301003844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6823","nonce":301003844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6821","nonce":301003844}]},"public_addr":"172.21.15.183:6817/301003844","cluster_addr":"172.21.15.183:6819/301003844","heartbeat_back_addr":"172.21.15.183:6823/301003844","heartbeat_front_addr":"172.21.15.183:6821/301003844","state":["exists","up"]},{"osd":9,"uuid":"e11dd8eb-c55a-4d70-8c58-457540b7a50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6827","nonce":3449260978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6829","nonce":3449260978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6833","nonce":3449260978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6831","nonce":3449260978}]},"public_addr":"172.21.15.53:6827/3449260978","cluster_addr":"172.21.15.53:6829/3449260978","heartbeat_back_addr":"172.21.15.53:6833/3449260978","heartbeat_front_addr":"172.21.15.53:6831/3449260978","state":["exists","up"]},{"osd":10,"uuid":"5de4803d-ad37-4a90-add5-0048f44fc4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6825","nonce":3435992308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6827","nonce":3435992308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6831","nonce":3435992308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6829","nonce":3435992308}]},"public_addr":"172.21.15.150:6825/3435992308","cluster_addr":"172.21.15.150:6827/3435992308","heartbeat_back_addr":"172.21.15.150:6831/3435992308","heartbeat_front_addr":"172.21.15.150:6829/3435992308","state":["exists","up"]},{"osd":11,"uuid":"0e819a68-ed56-477e-b9c0-81e8cc0bce11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6825","nonce":1957842686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6827","nonce":1957842686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6831","nonce":1957842686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6829","nonce":1957842686}]},"public_addr":"172.21.15.183:6825/1957842686","cluster_addr":"172.21.15.183:6827/1957842686","heartbeat_back_addr":"172.21.15.183:6831/1957842686","heartbeat_front_addr":"172.21.15.183:6829/1957842686","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:09.840629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:34.769537+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:58.112969+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:33:29.656790+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:33:55.181287+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:34:19.772645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:34:53.833921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:35:19.076733+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:35:43.163390+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:36:16.568340+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:36:42.492768+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:37:07.944639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6800/3887954889":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/1014501022":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/2576091441":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/1233527569":"2024-04-26T14:29:38.106242+0000","172.21.15.53:6801/3887954889":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/931889810":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/177196141":"2024-04-26T14:30:14.162383+0000","172.21.15.53:6800/1722497390":"2024-04-26T14:29:38.106242+0000","172.21.15.53:6801/1722497390":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/1927549198":"2024-04-26T14:30:14.162383+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:38:31.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.471+0000 7f49fdc00700 1 -- 172.21.15.53:0/2564742915 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f49dc0a3cd0 msgr2=0x7f49dc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:31.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.471+0000 7f49fdc00700 1 --2- 172.21.15.53:0/2564742915 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f49dc0a3cd0 0x7f49dc0a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f49e40098e0 tx=0x7f49e4005960 comp rx=0 tx=0).stop 2024-04-25T14:38:31.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.471+0000 7f49fdc00700 1 -- 172.21.15.53:0/2564742915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49f8108f80 msgr2=0x7f49f819f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:31.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.471+0000 7f49fdc00700 1 --2- 172.21.15.53:0/2564742915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49f8108f80 0x7f49f819f570 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f49e800eb30 tx=0x7f49e800ee40 comp rx=0 tx=0).stop 2024-04-25T14:38:31.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.472+0000 7f49fdc00700 1 -- 172.21.15.53:0/2564742915 shutdown_connections 2024-04-25T14:38:31.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.472+0000 7f49fdc00700 1 --2- 172.21.15.53:0/2564742915 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f49dc0a3cd0 0x7f49dc0a6190 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:31.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.472+0000 7f49fdc00700 1 --2- 172.21.15.53:0/2564742915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49f810f030 0x7f49f81a3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:31.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.472+0000 7f49fdc00700 1 --2- 172.21.15.53:0/2564742915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49f8108f80 0x7f49f819f570 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:31.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.472+0000 7f49fdc00700 1 --2- 172.21.15.53:0/2564742915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49f8109940 0x7f49f819fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:31.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.472+0000 7f49fdc00700 1 -- 172.21.15.53:0/2564742915 >> 172.21.15.53:0/2564742915 conn(0x7f49f80787d0 msgr2=0x7f49f8079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:31.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.472+0000 7f49fdc00700 1 -- 172.21.15.53:0/2564742915 shutdown_connections 2024-04-25T14:38:31.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:31.472+0000 7f49fdc00700 1 -- 172.21.15.53:0/2564742915 wait complete. 2024-04-25T14:38:32.194 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-25T14:38:32.195 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T14:38:32.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:32 smithi183 ceph-mon[32584]: pgmap v326: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-04-25T14:38:32.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:32 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/2564742915' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:38:32.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:32 smithi150 ceph-mon[32504]: pgmap v326: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-04-25T14:38:32.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:32 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/2564742915' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:38:32.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:32 smithi053 ceph-mon[27773]: pgmap v326: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-04-25T14:38:32.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:32 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/2564742915' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:38:32.665 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:38:34.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:34 smithi183 ceph-mon[32584]: pgmap v327: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-04-25T14:38:34.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:34 smithi150 ceph-mon[32504]: pgmap v327: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-04-25T14:38:34.476 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:34 smithi053 ceph-mon[27773]: pgmap v327: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-04-25T14:38:35.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.034+0000 7f1fa0400700 1 -- 172.21.15.53:0/559048097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f98105890 msgr2=0x7f1f981098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:35.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.034+0000 7f1fa0400700 1 --2- 172.21.15.53:0/559048097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f98105890 0x7f1f981098e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1f840099c0 tx=0x7f1f84009cd0 comp rx=0 tx=0).stop 2024-04-25T14:38:35.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.035+0000 7f1fa0400700 1 -- 172.21.15.53:0/559048097 shutdown_connections 2024-04-25T14:38:35.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.035+0000 7f1fa0400700 1 --2- 172.21.15.53:0/559048097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f98109fc0 0x7f1f98111b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:35.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.035+0000 7f1fa0400700 1 --2- 172.21.15.53:0/559048097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f98104ee0 0x7f1f981052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:35.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.035+0000 7f1fa0400700 1 --2- 172.21.15.53:0/559048097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f98105890 0x7f1f981098e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:35.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.035+0000 7f1fa0400700 1 -- 172.21.15.53:0/559048097 >> 172.21.15.53:0/559048097 conn(0x7f1f98100770 msgr2=0x7f1f98102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:35.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.035+0000 7f1fa0400700 1 -- 172.21.15.53:0/559048097 shutdown_connections 2024-04-25T14:38:35.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.035+0000 7f1fa0400700 1 -- 172.21.15.53:0/559048097 wait complete. 2024-04-25T14:38:35.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.036+0000 7f1fa0400700 1 Processor -- start 2024-04-25T14:38:35.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.036+0000 7f1fa0400700 1 -- start start 2024-04-25T14:38:35.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.036+0000 7f1fa0400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f98104ee0 0x7f1f9819b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:35.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.036+0000 7f1fa0400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f98105890 0x7f1f9819b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:35.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.036+0000 7f1fa0400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f98109fc0 0x7f1f9819fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:35.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.036+0000 7f1fa0400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1f98114ea0 con 0x7f1f98104ee0 2024-04-25T14:38:35.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.036+0000 7f1fa0400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1f98114d20 con 0x7f1f98109fc0 2024-04-25T14:38:35.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.036+0000 7f1fa0400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1f98115020 con 0x7f1f98105890 2024-04-25T14:38:35.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.037+0000 7f1f9da00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f98104ee0 0x7f1f9819b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:35.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.037+0000 7f1f9da00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f98104ee0 0x7f1f9819b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42966/0 (socket says 172.21.15.53:42966) 2024-04-25T14:38:35.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.037+0000 7f1f9da00700 1 -- 172.21.15.53:0/4128694637 learned_addr learned my addr 172.21.15.53:0/4128694637 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:38:35.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.037+0000 7f1f9d000700 1 --2- 172.21.15.53:0/4128694637 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f98105890 0x7f1f9819b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:35.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.037+0000 7f1f9e400700 1 --2- 172.21.15.53:0/4128694637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f98109fc0 0x7f1f9819fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:35.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.037+0000 7f1f9da00700 1 -- 172.21.15.53:0/4128694637 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f98105890 msgr2=0x7f1f9819b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:35.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.037+0000 7f1f9da00700 1 --2- 172.21.15.53:0/4128694637 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f98105890 0x7f1f9819b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:35.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.037+0000 7f1f9da00700 1 -- 172.21.15.53:0/4128694637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f98109fc0 msgr2=0x7f1f9819fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:35.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.037+0000 7f1f9da00700 1 --2- 172.21.15.53:0/4128694637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f98109fc0 0x7f1f9819fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:35.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.037+0000 7f1f9da00700 1 -- 172.21.15.53:0/4128694637 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f981a0350 con 0x7f1f98104ee0 2024-04-25T14:38:35.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.038+0000 7f1f9da00700 1 --2- 172.21.15.53:0/4128694637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f98104ee0 0x7f1f9819b390 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c00d980 tx=0x7f1f8c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:35.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.038+0000 7f1f96000700 1 -- 172.21.15.53:0/4128694637 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c016070 con 0x7f1f98104ee0 2024-04-25T14:38:35.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.038+0000 7f1f96000700 1 -- 172.21.15.53:0/4128694637 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f8c004830 con 0x7f1f98104ee0 2024-04-25T14:38:35.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.038+0000 7f1f96000700 1 -- 172.21.15.53:0/4128694637 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c0054c0 con 0x7f1f98104ee0 2024-04-25T14:38:35.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.038+0000 7f1fa0400700 1 -- 172.21.15.53:0/4128694637 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f981a0640 con 0x7f1f98104ee0 2024-04-25T14:38:35.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.038+0000 7f1fa0400700 1 -- 172.21.15.53:0/4128694637 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f981a7ec0 con 0x7f1f98104ee0 2024-04-25T14:38:35.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.039+0000 7f1fa0400700 1 -- 172.21.15.53:0/4128694637 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f98069730 con 0x7f1f98104ee0 2024-04-25T14:38:35.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.041+0000 7f1f96000700 1 -- 172.21.15.53:0/4128694637 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1f8c004220 con 0x7f1f98104ee0 2024-04-25T14:38:35.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.043+0000 7f1f96000700 1 --2- 172.21.15.53:0/4128694637 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f1f800a3d50 0x7f1f800a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:35.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.043+0000 7f1f96000700 1 -- 172.21.15.53:0/4128694637 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f1f8c0d0260 con 0x7f1f98104ee0 2024-04-25T14:38:35.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.043+0000 7f1f9d000700 1 --2- 172.21.15.53:0/4128694637 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f1f800a3d50 0x7f1f800a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:35.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.044+0000 7f1f9d000700 1 --2- 172.21.15.53:0/4128694637 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f1f800a3d50 0x7f1f800a6210 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1f84006010 tx=0x7f1f84005c20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:35.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.047+0000 7f1f96000700 1 -- 172.21.15.53:0/4128694637 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1f8c01f020 con 0x7f1f98104ee0 2024-04-25T14:38:35.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.281+0000 7f1fa0400700 1 -- 172.21.15.53:0/4128694637 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1f9819c6a0 con 0x7f1f98104ee0 2024-04-25T14:38:35.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.284+0000 7f1f96000700 1 -- 172.21.15.53:0/4128694637 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22171 (secure 0 0 0) 0x7f1f8c09cb90 con 0x7f1f98104ee0 2024-04-25T14:38:35.284 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:38:35.285 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":88,"fsid":"fd30e6ce-030f-11ef-bc93-c7b262605968","created":"2024-04-25T14:28:54.330326+0000","modified":"2024-04-25T14:38:11.415753+0000","last_up_change":"2024-04-25T14:37:10.921148+0000","last_in_change":"2024-04-25T14:36:49.566459+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:33:02.338115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:38:07.517974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:38:08.994122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aecfbaf0-39da-4749-a49b-c7205dabd4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6803","nonce":464176015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6805","nonce":464176015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6809","nonce":464176015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6807","nonce":464176015}]},"public_addr":"172.21.15.53:6803/464176015","cluster_addr":"172.21.15.53:6805/464176015","heartbeat_back_addr":"172.21.15.53:6809/464176015","heartbeat_front_addr":"172.21.15.53:6807/464176015","state":["exists","up"]},{"osd":1,"uuid":"2c4a21ae-c43e-4839-8a56-c3fc63aeb7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6801","nonce":1304621982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6803","nonce":1304621982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6807","nonce":1304621982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6805","nonce":1304621982}]},"public_addr":"172.21.15.150:6801/1304621982","cluster_addr":"172.21.15.150:6803/1304621982","heartbeat_back_addr":"172.21.15.150:6807/1304621982","heartbeat_front_addr":"172.21.15.150:6805/1304621982","state":["exists","up"]},{"osd":2,"uuid":"e8effd43-adc0-472a-9eed-a9273b06dc0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6801","nonce":574509311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6803","nonce":574509311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6807","nonce":574509311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6805","nonce":574509311}]},"public_addr":"172.21.15.183:6801/574509311","cluster_addr":"172.21.15.183:6803/574509311","heartbeat_back_addr":"172.21.15.183:6807/574509311","heartbeat_front_addr":"172.21.15.183:6805/574509311","state":["exists","up"]},{"osd":3,"uuid":"e7cfced4-9b40-40de-a303-75ea8d5df4a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6811","nonce":570672219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6813","nonce":570672219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6817","nonce":570672219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6815","nonce":570672219}]},"public_addr":"172.21.15.53:6811/570672219","cluster_addr":"172.21.15.53:6813/570672219","heartbeat_back_addr":"172.21.15.53:6817/570672219","heartbeat_front_addr":"172.21.15.53:6815/570672219","state":["exists","up"]},{"osd":4,"uuid":"44a9e070-2282-43f1-9f05-b1f8ff81e540","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6809","nonce":1526850103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6811","nonce":1526850103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6815","nonce":1526850103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6813","nonce":1526850103}]},"public_addr":"172.21.15.150:6809/1526850103","cluster_addr":"172.21.15.150:6811/1526850103","heartbeat_back_addr":"172.21.15.150:6815/1526850103","heartbeat_front_addr":"172.21.15.150:6813/1526850103","state":["exists","up"]},{"osd":5,"uuid":"2c5619d2-c0f9-47d6-bf7e-2f4afd1b09f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6809","nonce":773726576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6811","nonce":773726576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6815","nonce":773726576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6813","nonce":773726576}]},"public_addr":"172.21.15.183:6809/773726576","cluster_addr":"172.21.15.183:6811/773726576","heartbeat_back_addr":"172.21.15.183:6815/773726576","heartbeat_front_addr":"172.21.15.183:6813/773726576","state":["exists","up"]},{"osd":6,"uuid":"1cf74d94-c118-4738-b6db-040af458a8eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6819","nonce":355884319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6821","nonce":355884319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6825","nonce":355884319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6823","nonce":355884319}]},"public_addr":"172.21.15.53:6819/355884319","cluster_addr":"172.21.15.53:6821/355884319","heartbeat_back_addr":"172.21.15.53:6825/355884319","heartbeat_front_addr":"172.21.15.53:6823/355884319","state":["exists","up"]},{"osd":7,"uuid":"0ce97c04-cd41-4080-bbbc-be9cf5fadc02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6817","nonce":229022562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6819","nonce":229022562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6823","nonce":229022562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6821","nonce":229022562}]},"public_addr":"172.21.15.150:6817/229022562","cluster_addr":"172.21.15.150:6819/229022562","heartbeat_back_addr":"172.21.15.150:6823/229022562","heartbeat_front_addr":"172.21.15.150:6821/229022562","state":["exists","up"]},{"osd":8,"uuid":"5d57787d-2a7c-4514-bd01-357ced413f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6817","nonce":301003844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6819","nonce":301003844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6823","nonce":301003844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6821","nonce":301003844}]},"public_addr":"172.21.15.183:6817/301003844","cluster_addr":"172.21.15.183:6819/301003844","heartbeat_back_addr":"172.21.15.183:6823/301003844","heartbeat_front_addr":"172.21.15.183:6821/301003844","state":["exists","up"]},{"osd":9,"uuid":"e11dd8eb-c55a-4d70-8c58-457540b7a50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6827","nonce":3449260978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6829","nonce":3449260978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6833","nonce":3449260978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6831","nonce":3449260978}]},"public_addr":"172.21.15.53:6827/3449260978","cluster_addr":"172.21.15.53:6829/3449260978","heartbeat_back_addr":"172.21.15.53:6833/3449260978","heartbeat_front_addr":"172.21.15.53:6831/3449260978","state":["exists","up"]},{"osd":10,"uuid":"5de4803d-ad37-4a90-add5-0048f44fc4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6825","nonce":3435992308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6827","nonce":3435992308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6831","nonce":3435992308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6829","nonce":3435992308}]},"public_addr":"172.21.15.150:6825/3435992308","cluster_addr":"172.21.15.150:6827/3435992308","heartbeat_back_addr":"172.21.15.150:6831/3435992308","heartbeat_front_addr":"172.21.15.150:6829/3435992308","state":["exists","up"]},{"osd":11,"uuid":"0e819a68-ed56-477e-b9c0-81e8cc0bce11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6825","nonce":1957842686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6827","nonce":1957842686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6831","nonce":1957842686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6829","nonce":1957842686}]},"public_addr":"172.21.15.183:6825/1957842686","cluster_addr":"172.21.15.183:6827/1957842686","heartbeat_back_addr":"172.21.15.183:6831/1957842686","heartbeat_front_addr":"172.21.15.183:6829/1957842686","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:09.840629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:34.769537+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:58.112969+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:33:29.656790+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:33:55.181287+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:34:19.772645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:34:53.833921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:35:19.076733+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:35:43.163390+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:36:16.568340+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:36:42.492768+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:37:07.944639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6800/3887954889":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/1014501022":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/2576091441":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/1233527569":"2024-04-26T14:29:38.106242+0000","172.21.15.53:6801/3887954889":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/931889810":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/177196141":"2024-04-26T14:30:14.162383+0000","172.21.15.53:6800/1722497390":"2024-04-26T14:29:38.106242+0000","172.21.15.53:6801/1722497390":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/1927549198":"2024-04-26T14:30:14.162383+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:38:35.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.288+0000 7f1fa0400700 1 -- 172.21.15.53:0/4128694637 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f1f800a3d50 msgr2=0x7f1f800a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:35.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.288+0000 7f1fa0400700 1 --2- 172.21.15.53:0/4128694637 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f1f800a3d50 0x7f1f800a6210 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1f84006010 tx=0x7f1f84005c20 comp rx=0 tx=0).stop 2024-04-25T14:38:35.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.288+0000 7f1fa0400700 1 -- 172.21.15.53:0/4128694637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f98104ee0 msgr2=0x7f1f9819b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:35.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.288+0000 7f1fa0400700 1 --2- 172.21.15.53:0/4128694637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f98104ee0 0x7f1f9819b390 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c00d980 tx=0x7f1f8c00dc90 comp rx=0 tx=0).stop 2024-04-25T14:38:35.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.288+0000 7f1fa0400700 1 -- 172.21.15.53:0/4128694637 shutdown_connections 2024-04-25T14:38:35.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.288+0000 7f1fa0400700 1 --2- 172.21.15.53:0/4128694637 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f1f800a3d50 0x7f1f800a6210 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:35.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.288+0000 7f1fa0400700 1 --2- 172.21.15.53:0/4128694637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f98109fc0 0x7f1f9819fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:35.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.288+0000 7f1fa0400700 1 --2- 172.21.15.53:0/4128694637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f98104ee0 0x7f1f9819b390 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:35.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.288+0000 7f1fa0400700 1 --2- 172.21.15.53:0/4128694637 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f98105890 0x7f1f9819b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:35.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.288+0000 7f1fa0400700 1 -- 172.21.15.53:0/4128694637 >> 172.21.15.53:0/4128694637 conn(0x7f1f98100770 msgr2=0x7f1f98101320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:35.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.289+0000 7f1fa0400700 1 -- 172.21.15.53:0/4128694637 shutdown_connections 2024-04-25T14:38:35.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:35.289+0000 7f1fa0400700 1 -- 172.21.15.53:0/4128694637 wait complete. 2024-04-25T14:38:36.109 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-25T14:38:36.110 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-25T14:38:36.110 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-25T14:38:36.111 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-25T14:38:36.111 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-25T14:38:36.111 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-25T14:38:36.112 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-25T14:38:36.112 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-25T14:38:36.113 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-25T14:38:36.113 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-25T14:38:36.113 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-25T14:38:36.114 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-25T14:38:36.286 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:36 smithi053 ceph-mon[27773]: pgmap v328: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-04-25T14:38:36.287 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:36 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/4128694637' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:38:36.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:36 smithi183 ceph-mon[32584]: pgmap v328: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-04-25T14:38:36.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:36 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/4128694637' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:38:36.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:36 smithi150 ceph-mon[32504]: pgmap v328: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-04-25T14:38:36.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:36 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/4128694637' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:38:37.147 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:38:37.160 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:38:37.175 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:38:37.176 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:38:37.180 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:38:37.189 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:38:37.242 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:38:37.540 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:38:38.151 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:38 smithi053 ceph-mon[27773]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-25T14:38:38.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:38 smithi183 ceph-mon[32584]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-25T14:38:38.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:38 smithi150 ceph-mon[32504]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-25T14:38:40.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:40 smithi183 ceph-mon[32584]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T14:38:40.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:40 smithi150 ceph-mon[32504]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T14:38:40.473 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:40 smithi053 ceph-mon[27773]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T14:38:42.270 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:42 smithi053 ceph-mon[27773]: pgmap v331: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T14:38:42.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:42 smithi183 ceph-mon[32584]: pgmap v331: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T14:38:42.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:42 smithi150 ceph-mon[32504]: pgmap v331: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T14:38:44.213 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:44 smithi053 ceph-mon[27773]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T14:38:44.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:44 smithi183 ceph-mon[32584]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T14:38:44.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:44 smithi150 ceph-mon[32504]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T14:38:45.366 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:38:45.492 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:38:45.496 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:38:46.302 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:46 smithi053 ceph-mon[27773]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T14:38:46.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:46 smithi183 ceph-mon[32584]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T14:38:46.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:46 smithi150 ceph-mon[32504]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T14:38:47.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.753+0000 7ff166000700 1 -- 172.21.15.53:0/516680943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff16006a750 msgr2=0x7ff16006abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:47.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.753+0000 7ff166000700 1 --2- 172.21.15.53:0/516680943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff16006a750 0x7ff16006abb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff14c009990 tx=0x7ff14c009ca0 comp rx=0 tx=0).stop 2024-04-25T14:38:47.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.753+0000 7ff166000700 1 -- 172.21.15.53:0/516680943 shutdown_connections 2024-04-25T14:38:47.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.753+0000 7ff166000700 1 --2- 172.21.15.53:0/516680943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff16006b0f0 0x7ff16010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:47.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.753+0000 7ff166000700 1 --2- 172.21.15.53:0/516680943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff160108a30 0x7ff160108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:47.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.753+0000 7ff166000700 1 --2- 172.21.15.53:0/516680943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff16006a750 0x7ff16006abb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:47.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.753+0000 7ff166000700 1 -- 172.21.15.53:0/516680943 >> 172.21.15.53:0/516680943 conn(0x7ff1600788f0 msgr2=0x7ff160078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:47.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.753+0000 7ff166000700 1 -- 172.21.15.53:0/516680943 shutdown_connections 2024-04-25T14:38:47.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.753+0000 7ff166000700 1 -- 172.21.15.53:0/516680943 wait complete. 2024-04-25T14:38:47.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.754+0000 7ff166000700 1 Processor -- start 2024-04-25T14:38:47.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.754+0000 7ff166000700 1 -- start start 2024-04-25T14:38:47.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.754+0000 7ff166000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff16006a750 0x7ff16019b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:47.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.755+0000 7ff166000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff16006b0f0 0x7ff16019b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:47.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.755+0000 7ff166000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff160108a30 0x7ff16019fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:47.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.755+0000 7ff166000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff160112c20 con 0x7ff160108a30 2024-04-25T14:38:47.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.755+0000 7ff166000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff160112aa0 con 0x7ff16006b0f0 2024-04-25T14:38:47.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.755+0000 7ff166000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff160112da0 con 0x7ff16006a750 2024-04-25T14:38:47.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.755+0000 7ff15f400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff16006a750 0x7ff16019b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:47.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.755+0000 7ff15f400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff16006a750 0x7ff16019b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:60700/0 (socket says 172.21.15.53:60700) 2024-04-25T14:38:47.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.755+0000 7ff15f400700 1 -- 172.21.15.53:0/3499371831 learned_addr learned my addr 172.21.15.53:0/3499371831 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:38:47.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.755+0000 7ff15fe00700 1 --2- 172.21.15.53:0/3499371831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff160108a30 0x7ff16019fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:47.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.755+0000 7ff15ea00700 1 --2- 172.21.15.53:0/3499371831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff16006b0f0 0x7ff16019b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:47.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.755+0000 7ff15f400700 1 -- 172.21.15.53:0/3499371831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff16006b0f0 msgr2=0x7ff16019b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:47.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.755+0000 7ff15f400700 1 --2- 172.21.15.53:0/3499371831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff16006b0f0 0x7ff16019b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:47.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.755+0000 7ff15f400700 1 -- 172.21.15.53:0/3499371831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff160108a30 msgr2=0x7ff16019fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:47.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.755+0000 7ff15f400700 1 --2- 172.21.15.53:0/3499371831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff160108a30 0x7ff16019fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:47.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.755+0000 7ff15f400700 1 -- 172.21.15.53:0/3499371831 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1601a03a0 con 0x7ff16006a750 2024-04-25T14:38:47.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.756+0000 7ff15f400700 1 --2- 172.21.15.53:0/3499371831 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff16006a750 0x7ff16019b390 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff15000e9c0 tx=0x7ff15000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:47.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.756+0000 7ff157e00700 1 -- 172.21.15.53:0/3499371831 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15000cbd0 con 0x7ff16006a750 2024-04-25T14:38:47.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.756+0000 7ff166000700 1 -- 172.21.15.53:0/3499371831 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1601a0690 con 0x7ff16006a750 2024-04-25T14:38:47.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.756+0000 7ff157e00700 1 -- 172.21.15.53:0/3499371831 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff15000ee60 con 0x7ff16006a750 2024-04-25T14:38:47.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.756+0000 7ff157e00700 1 -- 172.21.15.53:0/3499371831 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1500055c0 con 0x7ff16006a750 2024-04-25T14:38:47.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.756+0000 7ff166000700 1 -- 172.21.15.53:0/3499371831 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1601a7f10 con 0x7ff16006a750 2024-04-25T14:38:47.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.757+0000 7ff166000700 1 -- 172.21.15.53:0/3499371831 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff160063150 con 0x7ff16006a750 2024-04-25T14:38:47.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.759+0000 7ff157e00700 1 -- 172.21.15.53:0/3499371831 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff150004750 con 0x7ff16006a750 2024-04-25T14:38:47.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.760+0000 7ff157e00700 1 --2- 172.21.15.53:0/3499371831 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff1440a3cd0 0x7ff1440a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:47.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.761+0000 7ff157e00700 1 -- 172.21.15.53:0/3499371831 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7ff1500d0ae0 con 0x7ff16006a750 2024-04-25T14:38:47.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.761+0000 7ff15ea00700 1 --2- 172.21.15.53:0/3499371831 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff1440a3cd0 0x7ff1440a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:47.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.761+0000 7ff157e00700 1 --2- 172.21.15.53:0/3499371831 >> [v2:172.21.15.53:6818/355884319,v1:172.21.15.53:6819/355884319] conn(0x7ff1440ad4b0 0x7ff1440af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:47.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.761+0000 7ff157e00700 1 -- 172.21.15.53:0/3499371831 --> [v2:172.21.15.53:6818/355884319,v1:172.21.15.53:6819/355884319] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff15001ee50 con 0x7ff1440ad4b0 2024-04-25T14:38:47.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.761+0000 7ff157e00700 1 -- 172.21.15.53:0/3499371831 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7ff1500d0e90 con 0x7ff16006a750 2024-04-25T14:38:47.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.761+0000 7ff15fe00700 1 --2- 172.21.15.53:0/3499371831 >> [v2:172.21.15.53:6818/355884319,v1:172.21.15.53:6819/355884319] conn(0x7ff1440ad4b0 0x7ff1440af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:47.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.761+0000 7ff15ea00700 1 --2- 172.21.15.53:0/3499371831 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff1440a3cd0 0x7ff1440a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff160078ed0 tx=0x7ff14c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:47.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.762+0000 7ff15fe00700 1 --2- 172.21.15.53:0/3499371831 >> [v2:172.21.15.53:6818/355884319,v1:172.21.15.53:6819/355884319] conn(0x7ff1440ad4b0 0x7ff1440af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:47.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.763+0000 7ff157e00700 1 -- 172.21.15.53:0/3499371831 <== osd.6 v2:172.21.15.53:6818/355884319 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff15001ee50 con 0x7ff1440ad4b0 2024-04-25T14:38:47.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.781+0000 7ff166000700 1 -- 172.21.15.53:0/3499371831 --> [v2:172.21.15.53:6818/355884319,v1:172.21.15.53:6819/355884319] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff1600635d0 con 0x7ff1440ad4b0 2024-04-25T14:38:47.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.782+0000 7ff157e00700 1 -- 172.21.15.53:0/3499371831 <== osd.6 v2:172.21.15.53:6818/355884319 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff1600635d0 con 0x7ff1440ad4b0 2024-04-25T14:38:47.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.782+0000 7ff166000700 1 -- 172.21.15.53:0/3499371831 >> [v2:172.21.15.53:6818/355884319,v1:172.21.15.53:6819/355884319] conn(0x7ff1440ad4b0 msgr2=0x7ff1440af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:47.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.782+0000 7ff166000700 1 --2- 172.21.15.53:0/3499371831 >> [v2:172.21.15.53:6818/355884319,v1:172.21.15.53:6819/355884319] conn(0x7ff1440ad4b0 0x7ff1440af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:47.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.782+0000 7ff166000700 1 -- 172.21.15.53:0/3499371831 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff1440a3cd0 msgr2=0x7ff1440a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:47.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.782+0000 7ff166000700 1 --2- 172.21.15.53:0/3499371831 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff1440a3cd0 0x7ff1440a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff160078ed0 tx=0x7ff14c00b260 comp rx=0 tx=0).stop 2024-04-25T14:38:47.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.782+0000 7ff166000700 1 -- 172.21.15.53:0/3499371831 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff16006a750 msgr2=0x7ff16019b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:47.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.782+0000 7ff166000700 1 --2- 172.21.15.53:0/3499371831 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff16006a750 0x7ff16019b390 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff15000e9c0 tx=0x7ff15000ecd0 comp rx=0 tx=0).stop 2024-04-25T14:38:47.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.782+0000 7ff15fe00700 1 -- 172.21.15.53:0/3499371831 reap_dead start 2024-04-25T14:38:47.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.782+0000 7ff166000700 1 -- 172.21.15.53:0/3499371831 shutdown_connections 2024-04-25T14:38:47.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.782+0000 7ff166000700 1 -- 172.21.15.53:0/3499371831 >> 172.21.15.53:0/3499371831 conn(0x7ff1600788f0 msgr2=0x7ff16010df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:47.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.783+0000 7ff166000700 1 -- 172.21.15.53:0/3499371831 shutdown_connections 2024-04-25T14:38:47.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:47.783+0000 7ff166000700 1 -- 172.21.15.53:0/3499371831 wait complete. 2024-04-25T14:38:48.370 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:48 smithi053 ceph-mon[27773]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T14:38:48.433 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.369+0000 7f58ade00700 1 -- 172.21.15.53:0/2532255208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58a8108a50 msgr2=0x7f58a8108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:48.433 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.369+0000 7f58ade00700 1 --2- 172.21.15.53:0/2532255208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58a8108a50 0x7f58a8108e30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f589800b0a0 tx=0x7f589800b3b0 comp rx=0 tx=0).stop 2024-04-25T14:38:48.433 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.370+0000 7f58ade00700 1 -- 172.21.15.53:0/2532255208 shutdown_connections 2024-04-25T14:38:48.433 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.370+0000 7f58ade00700 1 --2- 172.21.15.53:0/2532255208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58a8103450 0x7f58a810f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:48.433 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.370+0000 7f58ade00700 1 --2- 172.21.15.53:0/2532255208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58a8108a50 0x7f58a8108e30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:48.433 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.370+0000 7f58ade00700 1 --2- 172.21.15.53:0/2532255208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58a8102a90 0x7f58a8102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:48.433 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.370+0000 7f58ade00700 1 -- 172.21.15.53:0/2532255208 >> 172.21.15.53:0/2532255208 conn(0x7f58a8079ba0 msgr2=0x7f58a8079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:48.433 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.370+0000 7f58ade00700 1 -- 172.21.15.53:0/2532255208 shutdown_connections 2024-04-25T14:38:48.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.370+0000 7f58ade00700 1 -- 172.21.15.53:0/2532255208 wait complete. 2024-04-25T14:38:48.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.371+0000 7f58ade00700 1 Processor -- start 2024-04-25T14:38:48.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.371+0000 7f58ade00700 1 -- start start 2024-04-25T14:38:48.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.372+0000 7f58ade00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58a8102a90 0x7f58a819b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:48.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.372+0000 7f58ade00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58a8103450 0x7f58a819b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:48.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.372+0000 7f58ade00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58a8108a50 0x7f58a819fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:48.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.372+0000 7f58ade00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f58a8112bf0 con 0x7f58a8108a50 2024-04-25T14:38:48.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.372+0000 7f58ade00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f58a8112a70 con 0x7f58a8102a90 2024-04-25T14:38:48.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.372+0000 7f58ade00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f58a8112d70 con 0x7f58a8103450 2024-04-25T14:38:48.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.372+0000 7f58a6a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58a8102a90 0x7f58a819b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:48.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.372+0000 7f58a7400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58a8108a50 0x7f58a819fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:48.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.372+0000 7f58a6a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58a8102a90 0x7f58a819b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.53:45998/0 (socket says 172.21.15.53:45998) 2024-04-25T14:38:48.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.372+0000 7f58a6a00700 1 -- 172.21.15.53:0/1845477203 learned_addr learned my addr 172.21.15.53:0/1845477203 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:38:48.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.372+0000 7f589e000700 1 --2- 172.21.15.53:0/1845477203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58a8103450 0x7f58a819b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:48.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.372+0000 7f58a6a00700 1 -- 172.21.15.53:0/1845477203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58a8103450 msgr2=0x7f58a819b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:48.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.372+0000 7f58a6a00700 1 --2- 172.21.15.53:0/1845477203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58a8103450 0x7f58a819b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:48.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.372+0000 7f58a6a00700 1 -- 172.21.15.53:0/1845477203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58a8108a50 msgr2=0x7f58a819fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:48.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.372+0000 7f58a6a00700 1 --2- 172.21.15.53:0/1845477203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58a8108a50 0x7f58a819fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:48.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.372+0000 7f58a6a00700 1 -- 172.21.15.53:0/1845477203 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a81a03b0 con 0x7f58a8102a90 2024-04-25T14:38:48.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.373+0000 7f58a6a00700 1 --2- 172.21.15.53:0/1845477203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58a8102a90 0x7f58a819b3a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f589800b070 tx=0x7f5898004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:48.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.373+0000 7f58aca00700 1 -- 172.21.15.53:0/1845477203 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5898021070 con 0x7f58a8102a90 2024-04-25T14:38:48.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.373+0000 7f58aca00700 1 -- 172.21.15.53:0/1845477203 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5898007e30 con 0x7f58a8102a90 2024-04-25T14:38:48.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.373+0000 7f58ade00700 1 -- 172.21.15.53:0/1845477203 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58a81a0640 con 0x7f58a8102a90 2024-04-25T14:38:48.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.373+0000 7f58aca00700 1 -- 172.21.15.53:0/1845477203 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58980046c0 con 0x7f58a8102a90 2024-04-25T14:38:48.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.373+0000 7f58ade00700 1 -- 172.21.15.53:0/1845477203 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58a81a7f20 con 0x7f58a8102a90 2024-04-25T14:38:48.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.374+0000 7f58ade00700 1 -- 172.21.15.53:0/1845477203 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f58a810f8b0 con 0x7f58a8102a90 2024-04-25T14:38:48.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.376+0000 7f58aca00700 1 -- 172.21.15.53:0/1845477203 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5898007980 con 0x7f58a8102a90 2024-04-25T14:38:48.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.378+0000 7f58aca00700 1 --2- 172.21.15.53:0/1845477203 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f588c0a3cd0 0x7f588c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:48.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.378+0000 7f58aca00700 1 -- 172.21.15.53:0/1845477203 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f589800f020 con 0x7f58a8102a90 2024-04-25T14:38:48.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.378+0000 7f589e000700 1 --2- 172.21.15.53:0/1845477203 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f588c0a3cd0 0x7f588c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:48.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.378+0000 7f58aca00700 1 --2- 172.21.15.53:0/1845477203 >> [v2:172.21.15.183:6808/773726576,v1:172.21.15.183:6809/773726576] conn(0x7f588c0ad4b0 0x7f588c0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:48.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.378+0000 7f58aca00700 1 -- 172.21.15.53:0/1845477203 --> [v2:172.21.15.183:6808/773726576,v1:172.21.15.183:6809/773726576] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f589801fe80 con 0x7f588c0ad4b0 2024-04-25T14:38:48.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.378+0000 7f58aca00700 1 -- 172.21.15.53:0/1845477203 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f58980d1db0 con 0x7f58a8102a90 2024-04-25T14:38:48.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.379+0000 7f58a7400700 1 --2- 172.21.15.53:0/1845477203 >> [v2:172.21.15.183:6808/773726576,v1:172.21.15.183:6809/773726576] conn(0x7f588c0ad4b0 0x7f588c0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:48.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.379+0000 7f589e000700 1 --2- 172.21.15.53:0/1845477203 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f588c0a3cd0 0x7f588c0a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f58a807a580 tx=0x7f5890009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:48.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.379+0000 7f58a7400700 1 --2- 172.21.15.53:0/1845477203 >> [v2:172.21.15.183:6808/773726576,v1:172.21.15.183:6809/773726576] conn(0x7f588c0ad4b0 0x7f588c0af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:48.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.380+0000 7f58aca00700 1 -- 172.21.15.53:0/1845477203 <== osd.5 v2:172.21.15.183:6808/773726576 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f589801fe80 con 0x7f588c0ad4b0 2024-04-25T14:38:48.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.399+0000 7f58ade00700 1 -- 172.21.15.53:0/1845477203 --> [v2:172.21.15.183:6808/773726576,v1:172.21.15.183:6809/773726576] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f58a8104180 con 0x7f588c0ad4b0 2024-04-25T14:38:48.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.399+0000 7f58aca00700 1 -- 172.21.15.53:0/1845477203 <== osd.5 v2:172.21.15.183:6808/773726576 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f58a8104180 con 0x7f588c0ad4b0 2024-04-25T14:38:48.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.400+0000 7f58ade00700 1 -- 172.21.15.53:0/1845477203 >> [v2:172.21.15.183:6808/773726576,v1:172.21.15.183:6809/773726576] conn(0x7f588c0ad4b0 msgr2=0x7f588c0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:48.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.400+0000 7f58ade00700 1 --2- 172.21.15.53:0/1845477203 >> [v2:172.21.15.183:6808/773726576,v1:172.21.15.183:6809/773726576] conn(0x7f588c0ad4b0 0x7f588c0af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:48.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.400+0000 7f58ade00700 1 -- 172.21.15.53:0/1845477203 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f588c0a3cd0 msgr2=0x7f588c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:48.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.400+0000 7f58ade00700 1 --2- 172.21.15.53:0/1845477203 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f588c0a3cd0 0x7f588c0a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f58a807a580 tx=0x7f5890009260 comp rx=0 tx=0).stop 2024-04-25T14:38:48.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.400+0000 7f58ade00700 1 -- 172.21.15.53:0/1845477203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58a8102a90 msgr2=0x7f58a819b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:48.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.400+0000 7f58ade00700 1 --2- 172.21.15.53:0/1845477203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58a8102a90 0x7f58a819b3a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f589800b070 tx=0x7f5898004d00 comp rx=0 tx=0).stop 2024-04-25T14:38:48.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.400+0000 7f58a7400700 1 -- 172.21.15.53:0/1845477203 reap_dead start 2024-04-25T14:38:48.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.400+0000 7f58ade00700 1 -- 172.21.15.53:0/1845477203 shutdown_connections 2024-04-25T14:38:48.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.400+0000 7f58ade00700 1 -- 172.21.15.53:0/1845477203 >> 172.21.15.53:0/1845477203 conn(0x7f58a8079ba0 msgr2=0x7f58a810df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:48.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.400+0000 7f58ade00700 1 -- 172.21.15.53:0/1845477203 shutdown_connections 2024-04-25T14:38:48.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:48.400+0000 7f58ade00700 1 -- 172.21.15.53:0/1845477203 wait complete. 2024-04-25T14:38:48.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:48 smithi183 ceph-mon[32584]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T14:38:48.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:48 smithi150 ceph-mon[32504]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T14:38:50.259 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.194+0000 7f0f50c00700 1 -- 172.21.15.53:0/1562018269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f4c108a50 msgr2=0x7f0f4c108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:50.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.194+0000 7f0f50c00700 1 --2- 172.21.15.53:0/1562018269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f4c108a50 0x7f0f4c108e30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c00b0a0 tx=0x7f0f3c00b3b0 comp rx=0 tx=0).stop 2024-04-25T14:38:50.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.194+0000 7f0f50c00700 1 -- 172.21.15.53:0/1562018269 shutdown_connections 2024-04-25T14:38:50.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.194+0000 7f0f50c00700 1 --2- 172.21.15.53:0/1562018269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f4c103450 0x7f0f4c10f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:50.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.194+0000 7f0f50c00700 1 --2- 172.21.15.53:0/1562018269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f4c108a50 0x7f0f4c108e30 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:50.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.194+0000 7f0f50c00700 1 --2- 172.21.15.53:0/1562018269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f4c102a90 0x7f0f4c102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:50.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.194+0000 7f0f50c00700 1 -- 172.21.15.53:0/1562018269 >> 172.21.15.53:0/1562018269 conn(0x7f0f4c079ba0 msgr2=0x7f0f4c079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:50.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.194+0000 7f0f50c00700 1 -- 172.21.15.53:0/1562018269 shutdown_connections 2024-04-25T14:38:50.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.194+0000 7f0f50c00700 1 -- 172.21.15.53:0/1562018269 wait complete. 2024-04-25T14:38:50.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.195+0000 7f0f50c00700 1 Processor -- start 2024-04-25T14:38:50.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.195+0000 7f0f50c00700 1 -- start start 2024-04-25T14:38:50.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.196+0000 7f0f50c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f4c102a90 0x7f0f4c19b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:50.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.196+0000 7f0f50c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f4c103450 0x7f0f4c19b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:50.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.196+0000 7f0f50c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f4c108a50 0x7f0f4c19fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.196+0000 7f0f50c00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4c112c30 con 0x7f0f4c102a90 2024-04-25T14:38:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.196+0000 7f0f50c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4c112ab0 con 0x7f0f4c108a50 2024-04-25T14:38:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.196+0000 7f0f50c00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4c112db0 con 0x7f0f4c103450 2024-04-25T14:38:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.196+0000 7f0f49e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f4c102a90 0x7f0f4c19b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.196+0000 7f0f49e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f4c102a90 0x7f0f4c19b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35484/0 (socket says 172.21.15.53:35484) 2024-04-25T14:38:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.196+0000 7f0f49e00700 1 -- 172.21.15.53:0/3434693720 learned_addr learned my addr 172.21.15.53:0/3434693720 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:38:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.196+0000 7f0f41400700 1 --2- 172.21.15.53:0/3434693720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f4c103450 0x7f0f4c19b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.196+0000 7f0f4a800700 1 --2- 172.21.15.53:0/3434693720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f4c108a50 0x7f0f4c19fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.197+0000 7f0f49e00700 1 -- 172.21.15.53:0/3434693720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f4c103450 msgr2=0x7f0f4c19b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.197+0000 7f0f49e00700 1 --2- 172.21.15.53:0/3434693720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f4c103450 0x7f0f4c19b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.197+0000 7f0f49e00700 1 -- 172.21.15.53:0/3434693720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f4c108a50 msgr2=0x7f0f4c19fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.197+0000 7f0f49e00700 1 --2- 172.21.15.53:0/3434693720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f4c108a50 0x7f0f4c19fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.197+0000 7f0f49e00700 1 -- 172.21.15.53:0/3434693720 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f4c1a0310 con 0x7f0f4c102a90 2024-04-25T14:38:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.197+0000 7f0f41400700 1 --2- 172.21.15.53:0/3434693720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f4c103450 0x7f0f4c19b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:38:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.197+0000 7f0f49e00700 1 --2- 172.21.15.53:0/3434693720 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f4c102a90 0x7f0f4c19b300 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c00d010 tx=0x7f0f3c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.197+0000 7f0f43400700 1 -- 172.21.15.53:0/3434693720 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c021070 con 0x7f0f4c102a90 2024-04-25T14:38:50.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.197+0000 7f0f43400700 1 -- 172.21.15.53:0/3434693720 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f3c007e30 con 0x7f0f4c102a90 2024-04-25T14:38:50.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.197+0000 7f0f43400700 1 -- 172.21.15.53:0/3434693720 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c004700 con 0x7f0f4c102a90 2024-04-25T14:38:50.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.197+0000 7f0f50c00700 1 -- 172.21.15.53:0/3434693720 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f4c1a05a0 con 0x7f0f4c102a90 2024-04-25T14:38:50.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.198+0000 7f0f50c00700 1 -- 172.21.15.53:0/3434693720 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f4c1a7e80 con 0x7f0f4c102a90 2024-04-25T14:38:50.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.198+0000 7f0f50c00700 1 -- 172.21.15.53:0/3434693720 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0f4c10f8c0 con 0x7f0f4c102a90 2024-04-25T14:38:50.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.201+0000 7f0f43400700 1 -- 172.21.15.53:0/3434693720 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0f3c007980 con 0x7f0f4c102a90 2024-04-25T14:38:50.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.202+0000 7f0f43400700 1 --2- 172.21.15.53:0/3434693720 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f0f300a3d50 0x7f0f300a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:50.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.202+0000 7f0f43400700 1 -- 172.21.15.53:0/3434693720 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f0f3c00f020 con 0x7f0f4c102a90 2024-04-25T14:38:50.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.202+0000 7f0f41400700 1 --2- 172.21.15.53:0/3434693720 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f0f300a3d50 0x7f0f300a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:50.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.202+0000 7f0f43400700 1 --2- 172.21.15.53:0/3434693720 >> [v2:172.21.15.53:6810/570672219,v1:172.21.15.53:6811/570672219] conn(0x7f0f300ad530 0x7f0f300af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:50.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.202+0000 7f0f43400700 1 -- 172.21.15.53:0/3434693720 --> [v2:172.21.15.53:6810/570672219,v1:172.21.15.53:6811/570672219] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0f3c0020b0 con 0x7f0f300ad530 2024-04-25T14:38:50.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.202+0000 7f0f43400700 1 -- 172.21.15.53:0/3434693720 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f0f3c0d1e80 con 0x7f0f4c102a90 2024-04-25T14:38:50.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.203+0000 7f0f4a800700 1 --2- 172.21.15.53:0/3434693720 >> [v2:172.21.15.53:6810/570672219,v1:172.21.15.53:6811/570672219] conn(0x7f0f300ad530 0x7f0f300af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:50.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.203+0000 7f0f41400700 1 --2- 172.21.15.53:0/3434693720 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f0f300a3d50 0x7f0f300a6210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c101180 tx=0x7f0f34009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:50.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.203+0000 7f0f4a800700 1 --2- 172.21.15.53:0/3434693720 >> [v2:172.21.15.53:6810/570672219,v1:172.21.15.53:6811/570672219] conn(0x7f0f300ad530 0x7f0f300af950 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:50.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.204+0000 7f0f43400700 1 -- 172.21.15.53:0/3434693720 <== osd.3 v2:172.21.15.53:6810/570672219 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0f3c0020b0 con 0x7f0f300ad530 2024-04-25T14:38:50.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.223+0000 7f0f50c00700 1 -- 172.21.15.53:0/3434693720 --> [v2:172.21.15.53:6810/570672219,v1:172.21.15.53:6811/570672219] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0f4c104180 con 0x7f0f300ad530 2024-04-25T14:38:50.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.224+0000 7f0f43400700 1 -- 172.21.15.53:0/3434693720 <== osd.3 v2:172.21.15.53:6810/570672219 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0f4c104180 con 0x7f0f300ad530 2024-04-25T14:38:50.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.225+0000 7f0f50c00700 1 -- 172.21.15.53:0/3434693720 >> [v2:172.21.15.53:6810/570672219,v1:172.21.15.53:6811/570672219] conn(0x7f0f300ad530 msgr2=0x7f0f300af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:50.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.225+0000 7f0f50c00700 1 --2- 172.21.15.53:0/3434693720 >> [v2:172.21.15.53:6810/570672219,v1:172.21.15.53:6811/570672219] conn(0x7f0f300ad530 0x7f0f300af950 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:50.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.225+0000 7f0f50c00700 1 -- 172.21.15.53:0/3434693720 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f0f300a3d50 msgr2=0x7f0f300a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:50.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.225+0000 7f0f50c00700 1 --2- 172.21.15.53:0/3434693720 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f0f300a3d50 0x7f0f300a6210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c101180 tx=0x7f0f34009260 comp rx=0 tx=0).stop 2024-04-25T14:38:50.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.225+0000 7f0f50c00700 1 -- 172.21.15.53:0/3434693720 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f4c102a90 msgr2=0x7f0f4c19b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:50.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.225+0000 7f0f50c00700 1 --2- 172.21.15.53:0/3434693720 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f4c102a90 0x7f0f4c19b300 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c00d010 tx=0x7f0f3c004d00 comp rx=0 tx=0).stop 2024-04-25T14:38:50.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.225+0000 7f0f4a800700 1 -- 172.21.15.53:0/3434693720 reap_dead start 2024-04-25T14:38:50.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.225+0000 7f0f50c00700 1 -- 172.21.15.53:0/3434693720 shutdown_connections 2024-04-25T14:38:50.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.225+0000 7f0f50c00700 1 -- 172.21.15.53:0/3434693720 >> 172.21.15.53:0/3434693720 conn(0x7f0f4c079ba0 msgr2=0x7f0f4c10de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:50.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.225+0000 7f0f50c00700 1 -- 172.21.15.53:0/3434693720 shutdown_connections 2024-04-25T14:38:50.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:50.225+0000 7f0f50c00700 1 -- 172.21.15.53:0/3434693720 wait complete. 2024-04-25T14:38:50.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:50 smithi183 ceph-mon[32584]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T14:38:50.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:50 smithi150 ceph-mon[32504]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T14:38:50.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:50 smithi053 ceph-mon[27773]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T14:38:50.991 INFO:teuthology.orchestra.run.smithi053.stdout:206158430256 2024-04-25T14:38:50.991 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-25T14:38:51.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.200+0000 7f4584400700 1 -- 172.21.15.53:0/1515474858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f457c107990 msgr2=0x7f457c107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:51.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.200+0000 7f4584400700 1 --2- 172.21.15.53:0/1515474858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f457c107990 0x7f457c107d70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4570009990 tx=0x7f4570009ca0 comp rx=0 tx=0).stop 2024-04-25T14:38:51.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.200+0000 7f4584400700 1 -- 172.21.15.53:0/1515474858 shutdown_connections 2024-04-25T14:38:51.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.200+0000 7f4584400700 1 --2- 172.21.15.53:0/1515474858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f457c102300 0x7f457c10e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:51.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.200+0000 7f4584400700 1 --2- 172.21.15.53:0/1515474858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f457c107990 0x7f457c107d70 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:51.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.200+0000 7f4584400700 1 --2- 172.21.15.53:0/1515474858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f457c101940 0x7f457c101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:51.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.200+0000 7f4584400700 1 -- 172.21.15.53:0/1515474858 >> 172.21.15.53:0/1515474858 conn(0x7f457c0788f0 msgr2=0x7f457c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:51.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.200+0000 7f4584400700 1 -- 172.21.15.53:0/1515474858 shutdown_connections 2024-04-25T14:38:51.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.200+0000 7f4584400700 1 -- 172.21.15.53:0/1515474858 wait complete. 2024-04-25T14:38:51.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.202+0000 7f4584400700 1 Processor -- start 2024-04-25T14:38:51.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.202+0000 7f4584400700 1 -- start start 2024-04-25T14:38:51.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.202+0000 7f4584400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f457c101940 0x7f457c19b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:51.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.202+0000 7f4584400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f457c102300 0x7f457c19b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:51.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.202+0000 7f4584400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f457c107990 0x7f457c19fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:51.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.202+0000 7f4584400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f457c111b00 con 0x7f457c101940 2024-04-25T14:38:51.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.202+0000 7f4584400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f457c111980 con 0x7f457c102300 2024-04-25T14:38:51.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.202+0000 7f4584400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f457c111c80 con 0x7f457c107990 2024-04-25T14:38:51.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.203+0000 7f4581000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f457c102300 0x7f457c19b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:51.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.203+0000 7f4581000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f457c102300 0x7f457c19b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.53:46034/0 (socket says 172.21.15.53:46034) 2024-04-25T14:38:51.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.203+0000 7f4581000700 1 -- 172.21.15.53:0/1673078144 learned_addr learned my addr 172.21.15.53:0/1673078144 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:38:51.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.203+0000 7f4582400700 1 --2- 172.21.15.53:0/1673078144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f457c107990 0x7f457c19fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:51.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.203+0000 7f4581a00700 1 --2- 172.21.15.53:0/1673078144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f457c101940 0x7f457c19b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:51.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.203+0000 7f4581000700 1 -- 172.21.15.53:0/1673078144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f457c107990 msgr2=0x7f457c19fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:51.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.203+0000 7f4581000700 1 --2- 172.21.15.53:0/1673078144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f457c107990 0x7f457c19fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:51.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.203+0000 7f4581000700 1 -- 172.21.15.53:0/1673078144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f457c101940 msgr2=0x7f457c19b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:51.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.203+0000 7f4581000700 1 --2- 172.21.15.53:0/1673078144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f457c101940 0x7f457c19b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:51.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.203+0000 7f4581000700 1 -- 172.21.15.53:0/1673078144 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f457c1a0320 con 0x7f457c102300 2024-04-25T14:38:51.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.203+0000 7f4582400700 1 --2- 172.21.15.53:0/1673078144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f457c107990 0x7f457c19fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:38:51.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.203+0000 7f4581a00700 1 --2- 172.21.15.53:0/1673078144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f457c101940 0x7f457c19b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:38:51.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.204+0000 7f4581000700 1 --2- 172.21.15.53:0/1673078144 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f457c102300 0x7f457c19b850 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f456800e9c0 tx=0x7f456800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:51.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.204+0000 7f457a000700 1 -- 172.21.15.53:0/1673078144 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456800cbd0 con 0x7f457c102300 2024-04-25T14:38:51.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.204+0000 7f457a000700 1 -- 172.21.15.53:0/1673078144 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f456800ee60 con 0x7f457c102300 2024-04-25T14:38:51.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.204+0000 7f4584400700 1 -- 172.21.15.53:0/1673078144 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f457c1a0610 con 0x7f457c102300 2024-04-25T14:38:51.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.204+0000 7f457a000700 1 -- 172.21.15.53:0/1673078144 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45680055c0 con 0x7f457c102300 2024-04-25T14:38:51.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.204+0000 7f4584400700 1 -- 172.21.15.53:0/1673078144 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f457c1a7e90 con 0x7f457c102300 2024-04-25T14:38:51.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.205+0000 7f4584400700 1 -- 172.21.15.53:0/1673078144 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f457c06b190 con 0x7f457c102300 2024-04-25T14:38:51.208 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.207+0000 7f457a000700 1 -- 172.21.15.53:0/1673078144 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4568004750 con 0x7f457c102300 2024-04-25T14:38:51.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.209+0000 7f457a000700 1 --2- 172.21.15.53:0/1673078144 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f45640a3d50 0x7f45640a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:51.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.209+0000 7f457a000700 1 -- 172.21.15.53:0/1673078144 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f45680d06c0 con 0x7f457c102300 2024-04-25T14:38:51.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.209+0000 7f4581a00700 1 --2- 172.21.15.53:0/1673078144 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f45640a3d50 0x7f45640a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:51.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.209+0000 7f457a000700 1 --2- 172.21.15.53:0/1673078144 >> [v2:172.21.15.183:6800/574509311,v1:172.21.15.183:6801/574509311] conn(0x7f45640ad530 0x7f45640af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:51.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.209+0000 7f457a000700 1 -- 172.21.15.53:0/1673078144 --> [v2:172.21.15.183:6800/574509311,v1:172.21.15.183:6801/574509311] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f45640b0000 con 0x7f45640ad530 2024-04-25T14:38:51.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.209+0000 7f457a000700 1 -- 172.21.15.53:0/1673078144 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f45680d7050 con 0x7f457c102300 2024-04-25T14:38:51.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.210+0000 7f4582400700 1 --2- 172.21.15.53:0/1673078144 >> [v2:172.21.15.183:6800/574509311,v1:172.21.15.183:6801/574509311] conn(0x7f45640ad530 0x7f45640af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:51.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.210+0000 7f4581a00700 1 --2- 172.21.15.53:0/1673078144 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f45640a3d50 0x7f45640a6210 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f457000b2e0 tx=0x7f457001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:51.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.210+0000 7f4582400700 1 --2- 172.21.15.53:0/1673078144 >> [v2:172.21.15.183:6800/574509311,v1:172.21.15.183:6801/574509311] conn(0x7f45640ad530 0x7f45640af950 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:51.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.211+0000 7f457a000700 1 -- 172.21.15.53:0/1673078144 <== osd.2 v2:172.21.15.183:6800/574509311 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f45640b0000 con 0x7f45640ad530 2024-04-25T14:38:51.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.230+0000 7f4584400700 1 -- 172.21.15.53:0/1673078144 --> [v2:172.21.15.183:6800/574509311,v1:172.21.15.183:6801/574509311] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f457c103060 con 0x7f45640ad530 2024-04-25T14:38:51.232 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.231+0000 7f457a000700 1 -- 172.21.15.53:0/1673078144 <== osd.2 v2:172.21.15.183:6800/574509311 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f457c103060 con 0x7f45640ad530 2024-04-25T14:38:51.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.231+0000 7f4584400700 1 -- 172.21.15.53:0/1673078144 >> [v2:172.21.15.183:6800/574509311,v1:172.21.15.183:6801/574509311] conn(0x7f45640ad530 msgr2=0x7f45640af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:51.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.231+0000 7f4584400700 1 --2- 172.21.15.53:0/1673078144 >> [v2:172.21.15.183:6800/574509311,v1:172.21.15.183:6801/574509311] conn(0x7f45640ad530 0x7f45640af950 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:51.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.232+0000 7f4584400700 1 -- 172.21.15.53:0/1673078144 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f45640a3d50 msgr2=0x7f45640a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:51.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.232+0000 7f4584400700 1 --2- 172.21.15.53:0/1673078144 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f45640a3d50 0x7f45640a6210 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f457000b2e0 tx=0x7f457001a040 comp rx=0 tx=0).stop 2024-04-25T14:38:51.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.232+0000 7f4584400700 1 -- 172.21.15.53:0/1673078144 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f457c102300 msgr2=0x7f457c19b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:51.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.232+0000 7f4584400700 1 --2- 172.21.15.53:0/1673078144 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f457c102300 0x7f457c19b850 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f456800e9c0 tx=0x7f456800ecd0 comp rx=0 tx=0).stop 2024-04-25T14:38:51.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.232+0000 7f4582400700 1 -- 172.21.15.53:0/1673078144 reap_dead start 2024-04-25T14:38:51.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.232+0000 7f4584400700 1 -- 172.21.15.53:0/1673078144 shutdown_connections 2024-04-25T14:38:51.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.232+0000 7f4584400700 1 -- 172.21.15.53:0/1673078144 >> 172.21.15.53:0/1673078144 conn(0x7f457c0788f0 msgr2=0x7f457c06a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:51.234 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.232+0000 7f4584400700 1 -- 172.21.15.53:0/1673078144 shutdown_connections 2024-04-25T14:38:51.234 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:51.232+0000 7f4584400700 1 -- 172.21.15.53:0/1673078144 wait complete. 2024-04-25T14:38:52.374 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:52 smithi053 ceph-mon[27773]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T14:38:52.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:52 smithi183 ceph-mon[32584]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T14:38:52.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:52 smithi150 ceph-mon[32504]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T14:38:54.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.213+0000 7f5fd0600700 1 -- 172.21.15.53:0/2654368524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fc8108a30 msgr2=0x7f5fc8108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:54.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.213+0000 7f5fd0600700 1 --2- 172.21.15.53:0/2654368524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fc8108a30 0x7f5fc8108e10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc009990 tx=0x7f5fbc009ca0 comp rx=0 tx=0).stop 2024-04-25T14:38:54.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.213+0000 7f5fd0600700 1 -- 172.21.15.53:0/2654368524 shutdown_connections 2024-04-25T14:38:54.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.213+0000 7f5fd0600700 1 --2- 172.21.15.53:0/2654368524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5fc806b0f0 0x7f5fc810f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:54.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.213+0000 7f5fd0600700 1 --2- 172.21.15.53:0/2654368524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fc8108a30 0x7f5fc8108e10 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:54.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.213+0000 7f5fd0600700 1 --2- 172.21.15.53:0/2654368524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fc806a750 0x7f5fc806abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:54.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.213+0000 7f5fd0600700 1 -- 172.21.15.53:0/2654368524 >> 172.21.15.53:0/2654368524 conn(0x7f5fc80788f0 msgr2=0x7f5fc8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:54.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.213+0000 7f5fd0600700 1 -- 172.21.15.53:0/2654368524 shutdown_connections 2024-04-25T14:38:54.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.213+0000 7f5fd0600700 1 -- 172.21.15.53:0/2654368524 wait complete. 2024-04-25T14:38:54.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.214+0000 7f5fd0600700 1 Processor -- start 2024-04-25T14:38:54.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.214+0000 7f5fd0600700 1 -- start start 2024-04-25T14:38:54.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.215+0000 7f5fd0600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fc806a750 0x7f5fc8101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:54.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.215+0000 7f5fd0600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5fc806b0f0 0x7f5fc8101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:54.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.215+0000 7f5fd0600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fc8108a30 0x7f5fc8102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:54.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.215+0000 7f5fd0600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc8112c00 con 0x7f5fc8108a30 2024-04-25T14:38:54.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.215+0000 7f5fd0600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc8112a80 con 0x7f5fc806b0f0 2024-04-25T14:38:54.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.215+0000 7f5fd0600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc8112d80 con 0x7f5fc806a750 2024-04-25T14:38:54.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.215+0000 7f5fce600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fc8108a30 0x7f5fc8102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:54.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.215+0000 7f5fcd200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5fc806b0f0 0x7f5fc8101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:54.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.215+0000 7f5fcdc00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fc806a750 0x7f5fc8101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:54.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.215+0000 7f5fcdc00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fc806a750 0x7f5fc8101a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:54000/0 (socket says 172.21.15.53:54000) 2024-04-25T14:38:54.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.215+0000 7f5fcd200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5fc806b0f0 0x7f5fc8101f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.53:57026/0 (socket says 172.21.15.53:57026) 2024-04-25T14:38:54.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.215+0000 7f5fcdc00700 1 -- 172.21.15.53:0/2722457054 learned_addr learned my addr 172.21.15.53:0/2722457054 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:38:54.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.215+0000 7f5fce600700 1 --2- 172.21.15.53:0/2722457054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fc8108a30 0x7f5fc8102490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55596/0 (socket says 172.21.15.53:55596) 2024-04-25T14:38:54.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.216+0000 7f5fcdc00700 1 -- 172.21.15.53:0/2722457054 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5fc806b0f0 msgr2=0x7f5fc8101f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:54.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.216+0000 7f5fcdc00700 1 --2- 172.21.15.53:0/2722457054 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5fc806b0f0 0x7f5fc8101f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:54.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.216+0000 7f5fcdc00700 1 -- 172.21.15.53:0/2722457054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fc8108a30 msgr2=0x7f5fc8102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:54.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.216+0000 7f5fcdc00700 1 --2- 172.21.15.53:0/2722457054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fc8108a30 0x7f5fc8102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:54.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.216+0000 7f5fcdc00700 1 -- 172.21.15.53:0/2722457054 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fc81a7680 con 0x7f5fc806a750 2024-04-25T14:38:54.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.216+0000 7f5fcdc00700 1 --2- 172.21.15.53:0/2722457054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fc806a750 0x7f5fc8101a10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc000c00 tx=0x7f5fbc0057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:54.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.216+0000 7f5fc6000700 1 -- 172.21.15.53:0/2722457054 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc027070 con 0x7f5fc806a750 2024-04-25T14:38:54.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.216+0000 7f5fc6000700 1 -- 172.21.15.53:0/2722457054 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fbc022470 con 0x7f5fc806a750 2024-04-25T14:38:54.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.216+0000 7f5fc6000700 1 -- 172.21.15.53:0/2722457054 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc017650 con 0x7f5fc806a750 2024-04-25T14:38:54.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.216+0000 7f5fd0600700 1 -- 172.21.15.53:0/2722457054 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fc81a7910 con 0x7f5fc806a750 2024-04-25T14:38:54.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.217+0000 7f5fd0600700 1 -- 172.21.15.53:0/2722457054 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fc81a7d60 con 0x7f5fc806a750 2024-04-25T14:38:54.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.218+0000 7f5fd0600700 1 -- 172.21.15.53:0/2722457054 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5fc8063150 con 0x7f5fc806a750 2024-04-25T14:38:54.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.220+0000 7f5fc6000700 1 -- 172.21.15.53:0/2722457054 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5fbc02a070 con 0x7f5fc806a750 2024-04-25T14:38:54.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.221+0000 7f5fc6000700 1 --2- 172.21.15.53:0/2722457054 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5fb00a3cd0 0x7f5fb00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:54.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.221+0000 7f5fc6000700 1 -- 172.21.15.53:0/2722457054 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f5fbc0d1790 con 0x7f5fc806a750 2024-04-25T14:38:54.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.221+0000 7f5fcd200700 1 --2- 172.21.15.53:0/2722457054 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5fb00a3cd0 0x7f5fb00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:54.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.222+0000 7f5fc6000700 1 --2- 172.21.15.53:0/2722457054 >> [v2:172.21.15.183:6816/301003844,v1:172.21.15.183:6817/301003844] conn(0x7f5fb00ad4b0 0x7f5fb00af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:54.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.222+0000 7f5fc6000700 1 -- 172.21.15.53:0/2722457054 --> [v2:172.21.15.183:6816/301003844,v1:172.21.15.183:6817/301003844] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5fbc031e50 con 0x7f5fb00ad4b0 2024-04-25T14:38:54.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.222+0000 7f5fc6000700 1 -- 172.21.15.53:0/2722457054 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f5fbc0d1b40 con 0x7f5fc806a750 2024-04-25T14:38:54.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.222+0000 7f5fce600700 1 --2- 172.21.15.53:0/2722457054 >> [v2:172.21.15.183:6816/301003844,v1:172.21.15.183:6817/301003844] conn(0x7f5fb00ad4b0 0x7f5fb00af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:54.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.222+0000 7f5fcd200700 1 --2- 172.21.15.53:0/2722457054 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5fb00a3cd0 0x7f5fb00a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4009680 tx=0x7f5fb4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:54.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.223+0000 7f5fce600700 1 --2- 172.21.15.53:0/2722457054 >> [v2:172.21.15.183:6816/301003844,v1:172.21.15.183:6817/301003844] conn(0x7f5fb00ad4b0 0x7f5fb00af8d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:54.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.224+0000 7f5fc6000700 1 -- 172.21.15.53:0/2722457054 <== osd.8 v2:172.21.15.183:6816/301003844 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5fbc031e50 con 0x7f5fb00ad4b0 2024-04-25T14:38:54.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.242+0000 7f5fd0600700 1 -- 172.21.15.53:0/2722457054 --> [v2:172.21.15.183:6816/301003844,v1:172.21.15.183:6817/301003844] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5fc80635d0 con 0x7f5fb00ad4b0 2024-04-25T14:38:54.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.242+0000 7f5fc6000700 1 -- 172.21.15.53:0/2722457054 <== osd.8 v2:172.21.15.183:6816/301003844 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5fc80635d0 con 0x7f5fb00ad4b0 2024-04-25T14:38:54.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.243+0000 7f5fd0600700 1 -- 172.21.15.53:0/2722457054 >> [v2:172.21.15.183:6816/301003844,v1:172.21.15.183:6817/301003844] conn(0x7f5fb00ad4b0 msgr2=0x7f5fb00af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:54.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.243+0000 7f5fd0600700 1 --2- 172.21.15.53:0/2722457054 >> [v2:172.21.15.183:6816/301003844,v1:172.21.15.183:6817/301003844] conn(0x7f5fb00ad4b0 0x7f5fb00af8d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:54.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.243+0000 7f5fd0600700 1 -- 172.21.15.53:0/2722457054 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5fb00a3cd0 msgr2=0x7f5fb00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:54.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.243+0000 7f5fd0600700 1 --2- 172.21.15.53:0/2722457054 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5fb00a3cd0 0x7f5fb00a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4009680 tx=0x7f5fb4009260 comp rx=0 tx=0).stop 2024-04-25T14:38:54.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.243+0000 7f5fd0600700 1 -- 172.21.15.53:0/2722457054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fc806a750 msgr2=0x7f5fc8101a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:54.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.243+0000 7f5fd0600700 1 --2- 172.21.15.53:0/2722457054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fc806a750 0x7f5fc8101a10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc000c00 tx=0x7f5fbc0057c0 comp rx=0 tx=0).stop 2024-04-25T14:38:54.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.243+0000 7f5fce600700 1 -- 172.21.15.53:0/2722457054 reap_dead start 2024-04-25T14:38:54.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.243+0000 7f5fd0600700 1 -- 172.21.15.53:0/2722457054 shutdown_connections 2024-04-25T14:38:54.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.243+0000 7f5fd0600700 1 -- 172.21.15.53:0/2722457054 >> 172.21.15.53:0/2722457054 conn(0x7f5fc80788f0 msgr2=0x7f5fc80ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:54.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.244+0000 7f5fd0600700 1 -- 172.21.15.53:0/2722457054 shutdown_connections 2024-04-25T14:38:54.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:54.244+0000 7f5fd0600700 1 -- 172.21.15.53:0/2722457054 wait complete. 2024-04-25T14:38:54.387 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:54 smithi053 ceph-mon[27773]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T14:38:54.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:54 smithi183 ceph-mon[32584]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T14:38:54.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:54 smithi150 ceph-mon[32504]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T14:38:55.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.289+0000 7fdd72a00700 1 -- 172.21.15.53:0/3449320932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd6c100ec0 msgr2=0x7fdd6c1012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:55.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.289+0000 7fdd72a00700 1 --2- 172.21.15.53:0/3449320932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd6c100ec0 0x7fdd6c1012a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c0099c0 tx=0x7fdd5c009cd0 comp rx=0 tx=0).stop 2024-04-25T14:38:55.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.289+0000 7fdd72a00700 1 -- 172.21.15.53:0/3449320932 shutdown_connections 2024-04-25T14:38:55.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.289+0000 7fdd72a00700 1 --2- 172.21.15.53:0/3449320932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd6c105fa0 0x7fdd6c10db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:55.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.289+0000 7fdd72a00700 1 --2- 172.21.15.53:0/3449320932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd6c100ec0 0x7fdd6c1012a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:55.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.289+0000 7fdd72a00700 1 --2- 172.21.15.53:0/3449320932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd6c101870 0x7fdd6c1058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:55.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.289+0000 7fdd72a00700 1 -- 172.21.15.53:0/3449320932 >> 172.21.15.53:0/3449320932 conn(0x7fdd6c077b40 msgr2=0x7fdd6c0feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:55.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.290+0000 7fdd72a00700 1 -- 172.21.15.53:0/3449320932 shutdown_connections 2024-04-25T14:38:55.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.290+0000 7fdd72a00700 1 -- 172.21.15.53:0/3449320932 wait complete. 2024-04-25T14:38:55.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.291+0000 7fdd72a00700 1 Processor -- start 2024-04-25T14:38:55.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.291+0000 7fdd72a00700 1 -- start start 2024-04-25T14:38:55.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.291+0000 7fdd72a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd6c100ec0 0x7fdd6c0fcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:55.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.291+0000 7fdd72a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd6c101870 0x7fdd6c0fd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:55.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.291+0000 7fdd72a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd6c105fa0 0x7fdd6c0fda30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:55.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.291+0000 7fdd72a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6c111b70 con 0x7fdd6c100ec0 2024-04-25T14:38:55.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.291+0000 7fdd72a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6c1119f0 con 0x7fdd6c105fa0 2024-04-25T14:38:55.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.291+0000 7fdd72a00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6c111cf0 con 0x7fdd6c101870 2024-04-25T14:38:55.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.292+0000 7fdd6be00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd6c100ec0 0x7fdd6c0fcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:55.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.292+0000 7fdd6be00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd6c100ec0 0x7fdd6c0fcfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55600/0 (socket says 172.21.15.53:55600) 2024-04-25T14:38:55.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.292+0000 7fdd6be00700 1 -- 172.21.15.53:0/344108804 learned_addr learned my addr 172.21.15.53:0/344108804 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:38:55.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.292+0000 7fdd6b400700 1 --2- 172.21.15.53:0/344108804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd6c101870 0x7fdd6c0fd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:55.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.292+0000 7fdd70a00700 1 --2- 172.21.15.53:0/344108804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd6c105fa0 0x7fdd6c0fda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:55.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.292+0000 7fdd6be00700 1 -- 172.21.15.53:0/344108804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd6c101870 msgr2=0x7fdd6c0fd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:55.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.292+0000 7fdd6be00700 1 --2- 172.21.15.53:0/344108804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd6c101870 0x7fdd6c0fd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:55.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.292+0000 7fdd6be00700 1 -- 172.21.15.53:0/344108804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd6c105fa0 msgr2=0x7fdd6c0fda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:55.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.292+0000 7fdd6be00700 1 --2- 172.21.15.53:0/344108804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd6c105fa0 0x7fdd6c0fda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:55.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.292+0000 7fdd6be00700 1 -- 172.21.15.53:0/344108804 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd6c1a7790 con 0x7fdd6c100ec0 2024-04-25T14:38:55.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.292+0000 7fdd6b400700 1 --2- 172.21.15.53:0/344108804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd6c101870 0x7fdd6c0fd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:38:55.295 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.293+0000 7fdd6be00700 1 --2- 172.21.15.53:0/344108804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd6c100ec0 0x7fdd6c0fcfb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c005910 tx=0x7fdd5c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:55.295 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.293+0000 7fdd68c00700 1 -- 172.21.15.53:0/344108804 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c026070 con 0x7fdd6c100ec0 2024-04-25T14:38:55.295 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.293+0000 7fdd68c00700 1 -- 172.21.15.53:0/344108804 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd5c003c70 con 0x7fdd6c100ec0 2024-04-25T14:38:55.295 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.293+0000 7fdd68c00700 1 -- 172.21.15.53:0/344108804 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c00fa40 con 0x7fdd6c100ec0 2024-04-25T14:38:55.295 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.293+0000 7fdd72a00700 1 -- 172.21.15.53:0/344108804 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd6c1a7a20 con 0x7fdd6c100ec0 2024-04-25T14:38:55.295 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.293+0000 7fdd72a00700 1 -- 172.21.15.53:0/344108804 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd6c1a7e20 con 0x7fdd6c100ec0 2024-04-25T14:38:55.295 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.294+0000 7fdd72a00700 1 -- 172.21.15.53:0/344108804 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdd6c10ed60 con 0x7fdd6c100ec0 2024-04-25T14:38:55.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.296+0000 7fdd68c00700 1 -- 172.21.15.53:0/344108804 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdd5c029070 con 0x7fdd6c100ec0 2024-04-25T14:38:55.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.298+0000 7fdd68c00700 1 --2- 172.21.15.53:0/344108804 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fdd500a3cd0 0x7fdd500a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:55.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.298+0000 7fdd68c00700 1 -- 172.21.15.53:0/344108804 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fdd5c0d13e0 con 0x7fdd6c100ec0 2024-04-25T14:38:55.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.298+0000 7fdd6b400700 1 --2- 172.21.15.53:0/344108804 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fdd500a3cd0 0x7fdd500a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:55.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.298+0000 7fdd68c00700 1 --2- 172.21.15.53:0/344108804 >> [v2:172.21.15.150:6800/1304621982,v1:172.21.15.150:6801/1304621982] conn(0x7fdd500ad4b0 0x7fdd500af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:55.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.298+0000 7fdd68c00700 1 -- 172.21.15.53:0/344108804 --> [v2:172.21.15.150:6800/1304621982,v1:172.21.15.150:6801/1304621982] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdd500aff80 con 0x7fdd500ad4b0 2024-04-25T14:38:55.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.298+0000 7fdd68c00700 1 -- 172.21.15.53:0/344108804 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fdd5c0d1790 con 0x7fdd6c100ec0 2024-04-25T14:38:55.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.299+0000 7fdd70a00700 1 --2- 172.21.15.53:0/344108804 >> [v2:172.21.15.150:6800/1304621982,v1:172.21.15.150:6801/1304621982] conn(0x7fdd500ad4b0 0x7fdd500af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:55.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.299+0000 7fdd6b400700 1 --2- 172.21.15.53:0/344108804 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fdd500a3cd0 0x7fdd500a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c100140 tx=0x7fdd58009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:55.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.299+0000 7fdd70a00700 1 --2- 172.21.15.53:0/344108804 >> [v2:172.21.15.150:6800/1304621982,v1:172.21.15.150:6801/1304621982] conn(0x7fdd500ad4b0 0x7fdd500af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:55.301 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.300+0000 7fdd68c00700 1 -- 172.21.15.53:0/344108804 <== osd.1 v2:172.21.15.150:6800/1304621982 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdd500aff80 con 0x7fdd500ad4b0 2024-04-25T14:38:55.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.318+0000 7fdd72a00700 1 -- 172.21.15.53:0/344108804 --> [v2:172.21.15.150:6800/1304621982,v1:172.21.15.150:6801/1304621982] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdd6c1023a0 con 0x7fdd500ad4b0 2024-04-25T14:38:55.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.319+0000 7fdd68c00700 1 -- 172.21.15.53:0/344108804 <== osd.1 v2:172.21.15.150:6800/1304621982 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fdd6c1023a0 con 0x7fdd500ad4b0 2024-04-25T14:38:55.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.320+0000 7fdd72a00700 1 -- 172.21.15.53:0/344108804 >> [v2:172.21.15.150:6800/1304621982,v1:172.21.15.150:6801/1304621982] conn(0x7fdd500ad4b0 msgr2=0x7fdd500af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:55.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.320+0000 7fdd72a00700 1 --2- 172.21.15.53:0/344108804 >> [v2:172.21.15.150:6800/1304621982,v1:172.21.15.150:6801/1304621982] conn(0x7fdd500ad4b0 0x7fdd500af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:55.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.320+0000 7fdd72a00700 1 -- 172.21.15.53:0/344108804 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fdd500a3cd0 msgr2=0x7fdd500a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:55.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.320+0000 7fdd72a00700 1 --2- 172.21.15.53:0/344108804 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fdd500a3cd0 0x7fdd500a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c100140 tx=0x7fdd58009260 comp rx=0 tx=0).stop 2024-04-25T14:38:55.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.320+0000 7fdd72a00700 1 -- 172.21.15.53:0/344108804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd6c100ec0 msgr2=0x7fdd6c0fcfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:55.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.320+0000 7fdd72a00700 1 --2- 172.21.15.53:0/344108804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd6c100ec0 0x7fdd6c0fcfb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c005910 tx=0x7fdd5c004270 comp rx=0 tx=0).stop 2024-04-25T14:38:55.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.320+0000 7fdd70a00700 1 -- 172.21.15.53:0/344108804 reap_dead start 2024-04-25T14:38:55.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.320+0000 7fdd72a00700 1 -- 172.21.15.53:0/344108804 shutdown_connections 2024-04-25T14:38:55.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.320+0000 7fdd72a00700 1 -- 172.21.15.53:0/344108804 >> 172.21.15.53:0/344108804 conn(0x7fdd6c077b40 msgr2=0x7fdd6c104860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:55.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.320+0000 7fdd72a00700 1 -- 172.21.15.53:0/344108804 shutdown_connections 2024-04-25T14:38:55.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:55.320+0000 7fdd72a00700 1 -- 172.21.15.53:0/344108804 wait complete. 2024-04-25T14:38:55.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.395+0000 7f0260800700 1 -- 172.21.15.53:0/2869013218 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0258113c10 msgr2=0x7f0258116000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:55.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.395+0000 7f0260800700 1 --2- 172.21.15.53:0/2869013218 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0258113c10 0x7f0258116000 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0254009f90 tx=0x7f025400b330 comp rx=0 tx=0).stop 2024-04-25T14:38:55.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.395+0000 7f0260800700 1 -- 172.21.15.53:0/2869013218 shutdown_connections 2024-04-25T14:38:55.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.395+0000 7f0260800700 1 --2- 172.21.15.53:0/2869013218 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0258113c10 0x7f0258116000 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:55.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.395+0000 7f0260800700 1 --2- 172.21.15.53:0/2869013218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0258074c70 0x7f0258075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:55.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.395+0000 7f0260800700 1 --2- 172.21.15.53:0/2869013218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0258075590 0x7f02581136d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:55.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.395+0000 7f0260800700 1 -- 172.21.15.53:0/2869013218 >> 172.21.15.53:0/2869013218 conn(0x7f0258079ba0 msgr2=0x7f0258079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:55.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.395+0000 7f0260800700 1 -- 172.21.15.53:0/2869013218 shutdown_connections 2024-04-25T14:38:55.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.395+0000 7f0260800700 1 -- 172.21.15.53:0/2869013218 wait complete. 2024-04-25T14:38:55.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.397+0000 7f0260800700 1 Processor -- start 2024-04-25T14:38:55.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.397+0000 7f0260800700 1 -- start start 2024-04-25T14:38:55.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.397+0000 7f0260800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0258074c70 0x7f025819f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:55.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.397+0000 7f0260800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0258075590 0x7f025819fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:55.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.397+0000 7f0260800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0258113c10 0x7f02581a3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:55.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.397+0000 7f0260800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0258119240 con 0x7f0258075590 2024-04-25T14:38:55.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.397+0000 7f0260800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f02581190c0 con 0x7f0258074c70 2024-04-25T14:38:55.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.397+0000 7f0260800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f02581193c0 con 0x7f0258113c10 2024-04-25T14:38:55.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.398+0000 7f025d400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0258075590 0x7f025819fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:55.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.398+0000 7f025de00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0258074c70 0x7f025819f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:55.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.398+0000 7f025de00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0258074c70 0x7f025819f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.53:57004/0 (socket says 172.21.15.53:57004) 2024-04-25T14:38:55.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.398+0000 7f025de00700 1 -- 172.21.15.53:0/3081783294 learned_addr learned my addr 172.21.15.53:0/3081783294 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:38:55.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.398+0000 7f025d400700 1 --2- 172.21.15.53:0/3081783294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0258075590 0x7f025819fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55582/0 (socket says 172.21.15.53:55582) 2024-04-25T14:38:55.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.398+0000 7f025e800700 1 --2- 172.21.15.53:0/3081783294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0258113c10 0x7f02581a3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:55.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.398+0000 7f025d400700 1 -- 172.21.15.53:0/3081783294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0258113c10 msgr2=0x7f02581a3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:55.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.398+0000 7f025d400700 1 --2- 172.21.15.53:0/3081783294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0258113c10 0x7f02581a3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:55.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.398+0000 7f025d400700 1 -- 172.21.15.53:0/3081783294 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0258074c70 msgr2=0x7f025819f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:55.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.398+0000 7f025d400700 1 --2- 172.21.15.53:0/3081783294 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0258074c70 0x7f025819f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:55.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.398+0000 7f025d400700 1 -- 172.21.15.53:0/3081783294 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02581a4720 con 0x7f0258075590 2024-04-25T14:38:55.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.398+0000 7f025e800700 1 --2- 172.21.15.53:0/3081783294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0258113c10 0x7f02581a3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:38:55.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.398+0000 7f025de00700 1 --2- 172.21.15.53:0/3081783294 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0258074c70 0x7f025819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:38:55.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.398+0000 7f025d400700 1 --2- 172.21.15.53:0/3081783294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0258075590 0x7f025819fc50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f024800eb90 tx=0x7f024800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:55.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.399+0000 7f0252a00700 1 -- 172.21.15.53:0/3081783294 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024800cc90 con 0x7f0258075590 2024-04-25T14:38:55.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.399+0000 7f0252a00700 1 -- 172.21.15.53:0/3081783294 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f024800ce30 con 0x7f0258075590 2024-04-25T14:38:55.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.399+0000 7f0260800700 1 -- 172.21.15.53:0/3081783294 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02581a4a10 con 0x7f0258075590 2024-04-25T14:38:55.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.399+0000 7f0252a00700 1 -- 172.21.15.53:0/3081783294 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0248010430 con 0x7f0258075590 2024-04-25T14:38:55.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.399+0000 7f0260800700 1 -- 172.21.15.53:0/3081783294 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02581ac290 con 0x7f0258075590 2024-04-25T14:38:55.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.400+0000 7f0260800700 1 -- 172.21.15.53:0/3081783294 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0258110110 con 0x7f0258075590 2024-04-25T14:38:55.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.402+0000 7f0252a00700 1 -- 172.21.15.53:0/3081783294 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0248003ef0 con 0x7f0258075590 2024-04-25T14:38:55.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.404+0000 7f0252a00700 1 --2- 172.21.15.53:0/3081783294 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f02400a3cd0 0x7f02400a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:55.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.404+0000 7f0252a00700 1 -- 172.21.15.53:0/3081783294 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f02480d0a10 con 0x7f0258075590 2024-04-25T14:38:55.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.404+0000 7f025de00700 1 --2- 172.21.15.53:0/3081783294 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f02400a3cd0 0x7f02400a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:55.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.404+0000 7f0252a00700 1 --2- 172.21.15.53:0/3081783294 >> [v2:172.21.15.150:6808/1526850103,v1:172.21.15.150:6809/1526850103] conn(0x7f02400ad4b0 0x7f02400af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:55.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.404+0000 7f0252a00700 1 -- 172.21.15.53:0/3081783294 --> [v2:172.21.15.150:6808/1526850103,v1:172.21.15.150:6809/1526850103] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f024800e1a0 con 0x7f02400ad4b0 2024-04-25T14:38:55.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.404+0000 7f0252a00700 1 -- 172.21.15.53:0/3081783294 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f02480d0dc0 con 0x7f0258075590 2024-04-25T14:38:55.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.404+0000 7f025e800700 1 --2- 172.21.15.53:0/3081783294 >> [v2:172.21.15.150:6808/1526850103,v1:172.21.15.150:6809/1526850103] conn(0x7f02400ad4b0 0x7f02400af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:55.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.404+0000 7f025de00700 1 --2- 172.21.15.53:0/3081783294 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f02400a3cd0 0x7f02400a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f024c011590 tx=0x7f024c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:55.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.405+0000 7f025e800700 1 --2- 172.21.15.53:0/3081783294 >> [v2:172.21.15.150:6808/1526850103,v1:172.21.15.150:6809/1526850103] conn(0x7f02400ad4b0 0x7f02400af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:55.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.406+0000 7f0252a00700 1 -- 172.21.15.53:0/3081783294 <== osd.4 v2:172.21.15.150:6808/1526850103 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f024800e1a0 con 0x7f02400ad4b0 2024-04-25T14:38:55.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.424+0000 7f0260800700 1 -- 172.21.15.53:0/3081783294 --> [v2:172.21.15.150:6808/1526850103,v1:172.21.15.150:6809/1526850103] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0258076360 con 0x7f02400ad4b0 2024-04-25T14:38:55.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.425+0000 7f0252a00700 1 -- 172.21.15.53:0/3081783294 <== osd.4 v2:172.21.15.150:6808/1526850103 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0258076360 con 0x7f02400ad4b0 2024-04-25T14:38:55.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.426+0000 7f0260800700 1 -- 172.21.15.53:0/3081783294 >> [v2:172.21.15.150:6808/1526850103,v1:172.21.15.150:6809/1526850103] conn(0x7f02400ad4b0 msgr2=0x7f02400af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:55.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.426+0000 7f0260800700 1 --2- 172.21.15.53:0/3081783294 >> [v2:172.21.15.150:6808/1526850103,v1:172.21.15.150:6809/1526850103] conn(0x7f02400ad4b0 0x7f02400af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:55.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.426+0000 7f0260800700 1 -- 172.21.15.53:0/3081783294 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f02400a3cd0 msgr2=0x7f02400a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:55.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.426+0000 7f0260800700 1 --2- 172.21.15.53:0/3081783294 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f02400a3cd0 0x7f02400a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f024c011590 tx=0x7f024c009260 comp rx=0 tx=0).stop 2024-04-25T14:38:55.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.426+0000 7f0260800700 1 -- 172.21.15.53:0/3081783294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0258075590 msgr2=0x7f025819fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:55.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.426+0000 7f0260800700 1 --2- 172.21.15.53:0/3081783294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0258075590 0x7f025819fc50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f024800eb90 tx=0x7f024800eea0 comp rx=0 tx=0).stop 2024-04-25T14:38:55.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.426+0000 7f025e800700 1 -- 172.21.15.53:0/3081783294 reap_dead start 2024-04-25T14:38:55.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.426+0000 7f0260800700 1 -- 172.21.15.53:0/3081783294 shutdown_connections 2024-04-25T14:38:55.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.426+0000 7f0260800700 1 -- 172.21.15.53:0/3081783294 >> 172.21.15.53:0/3081783294 conn(0x7f0258079ba0 msgr2=0x7f025810f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:55.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.426+0000 7f0260800700 1 -- 172.21.15.53:0/3081783294 shutdown_connections 2024-04-25T14:38:55.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.426+0000 7f0260800700 1 -- 172.21.15.53:0/3081783294 wait complete. 2024-04-25T14:38:56.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:56 smithi183 ceph-mon[32584]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T14:38:56.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:56 smithi150 ceph-mon[32504]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T14:38:56.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:56 smithi053 ceph-mon[27773]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T14:38:56.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.891+0000 7f042a600700 1 -- 172.21.15.53:0/3109459179 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0424108a30 msgr2=0x7f0424108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:56.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.891+0000 7f042a600700 1 --2- 172.21.15.53:0/3109459179 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0424108a30 0x7f0424108e10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f04140098f0 tx=0x7f0414009c00 comp rx=0 tx=0).stop 2024-04-25T14:38:56.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.891+0000 7f042a600700 1 -- 172.21.15.53:0/3109459179 shutdown_connections 2024-04-25T14:38:56.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.891+0000 7f042a600700 1 --2- 172.21.15.53:0/3109459179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f042406b0f0 0x7f042410f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:56.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.891+0000 7f042a600700 1 --2- 172.21.15.53:0/3109459179 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0424108a30 0x7f0424108e10 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:56.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.891+0000 7f042a600700 1 --2- 172.21.15.53:0/3109459179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f042406a750 0x7f042406abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:56.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.891+0000 7f042a600700 1 -- 172.21.15.53:0/3109459179 >> 172.21.15.53:0/3109459179 conn(0x7f04240788f0 msgr2=0x7f0424078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:56.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.891+0000 7f042a600700 1 -- 172.21.15.53:0/3109459179 shutdown_connections 2024-04-25T14:38:56.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.892+0000 7f042a600700 1 -- 172.21.15.53:0/3109459179 wait complete. 2024-04-25T14:38:56.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.893+0000 7f042a600700 1 Processor -- start 2024-04-25T14:38:56.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.893+0000 7f042a600700 1 -- start start 2024-04-25T14:38:56.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.893+0000 7f042a600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f042406a750 0x7f0424199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:56.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.893+0000 7f042a600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f042406b0f0 0x7f0424199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:56.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.893+0000 7f042a600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0424108a30 0x7f042419d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:56.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.893+0000 7f042a600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0424112bd0 con 0x7f0424108a30 2024-04-25T14:38:56.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.893+0000 7f042a600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0424112a50 con 0x7f042406a750 2024-04-25T14:38:56.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.893+0000 7f042a600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0424112d50 con 0x7f042406b0f0 2024-04-25T14:38:56.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.894+0000 7f0423e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0424108a30 0x7f042419d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:56.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.894+0000 7f0422a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f042406b0f0 0x7f0424199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:56.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.894+0000 7f0423400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f042406a750 0x7f0424199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:56.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.894+0000 7f0422a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f042406b0f0 0x7f0424199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:54032/0 (socket says 172.21.15.53:54032) 2024-04-25T14:38:56.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.894+0000 7f0422a00700 1 -- 172.21.15.53:0/3293772916 learned_addr learned my addr 172.21.15.53:0/3293772916 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:38:56.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.894+0000 7f0422a00700 1 -- 172.21.15.53:0/3293772916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f042406a750 msgr2=0x7f0424199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:56.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.894+0000 7f0422a00700 1 --2- 172.21.15.53:0/3293772916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f042406a750 0x7f0424199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:56.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.894+0000 7f0422a00700 1 -- 172.21.15.53:0/3293772916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0424108a30 msgr2=0x7f042419d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:56.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.894+0000 7f0422a00700 1 --2- 172.21.15.53:0/3293772916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0424108a30 0x7f042419d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:56.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.894+0000 7f0422a00700 1 -- 172.21.15.53:0/3293772916 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f042419e110 con 0x7f042406b0f0 2024-04-25T14:38:56.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.895+0000 7f0422a00700 1 --2- 172.21.15.53:0/3293772916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f042406b0f0 0x7f0424199640 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f041000ec10 tx=0x7f041000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:56.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.895+0000 7f041be00700 1 -- 172.21.15.53:0/3293772916 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041000cc90 con 0x7f042406b0f0 2024-04-25T14:38:56.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.895+0000 7f042a600700 1 -- 172.21.15.53:0/3293772916 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f042419e400 con 0x7f042406b0f0 2024-04-25T14:38:56.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.895+0000 7f041be00700 1 -- 172.21.15.53:0/3293772916 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f041000ce30 con 0x7f042406b0f0 2024-04-25T14:38:56.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.895+0000 7f041be00700 1 -- 172.21.15.53:0/3293772916 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0410010430 con 0x7f042406b0f0 2024-04-25T14:38:56.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.895+0000 7f042a600700 1 -- 172.21.15.53:0/3293772916 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04241a5c80 con 0x7f042406b0f0 2024-04-25T14:38:56.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.896+0000 7f042a600700 1 -- 172.21.15.53:0/3293772916 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f04240631d0 con 0x7f042406b0f0 2024-04-25T14:38:56.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.898+0000 7f041be00700 1 -- 172.21.15.53:0/3293772916 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0410004000 con 0x7f042406b0f0 2024-04-25T14:38:56.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.899+0000 7f041be00700 1 --2- 172.21.15.53:0/3293772916 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f04080a3cd0 0x7f04080a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:56.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.900+0000 7f041be00700 1 -- 172.21.15.53:0/3293772916 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f04100d0970 con 0x7f042406b0f0 2024-04-25T14:38:56.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.900+0000 7f0423400700 1 --2- 172.21.15.53:0/3293772916 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f04080a3cd0 0x7f04080a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:56.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.900+0000 7f041be00700 1 --2- 172.21.15.53:0/3293772916 >> [v2:172.21.15.53:6826/3449260978,v1:172.21.15.53:6827/3449260978] conn(0x7f04080ad4b0 0x7f04080af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:56.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.900+0000 7f041be00700 1 -- 172.21.15.53:0/3293772916 --> [v2:172.21.15.53:6826/3449260978,v1:172.21.15.53:6827/3449260978] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f04080aff80 con 0x7f04080ad4b0 2024-04-25T14:38:56.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.900+0000 7f041be00700 1 -- 172.21.15.53:0/3293772916 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f04100d0d20 con 0x7f042406b0f0 2024-04-25T14:38:56.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.900+0000 7f0423e00700 1 --2- 172.21.15.53:0/3293772916 >> [v2:172.21.15.53:6826/3449260978,v1:172.21.15.53:6827/3449260978] conn(0x7f04080ad4b0 0x7f04080af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:56.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.901+0000 7f0423400700 1 --2- 172.21.15.53:0/3293772916 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f04080a3cd0 0x7f04080a6190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0414009fd0 tx=0x7f0414019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:56.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.901+0000 7f0423e00700 1 --2- 172.21.15.53:0/3293772916 >> [v2:172.21.15.53:6826/3449260978,v1:172.21.15.53:6827/3449260978] conn(0x7f04080ad4b0 0x7f04080af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:56.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.902+0000 7f041be00700 1 -- 172.21.15.53:0/3293772916 <== osd.9 v2:172.21.15.53:6826/3449260978 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f04080aff80 con 0x7f04080ad4b0 2024-04-25T14:38:56.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.922+0000 7f042a600700 1 -- 172.21.15.53:0/3293772916 --> [v2:172.21.15.53:6826/3449260978,v1:172.21.15.53:6827/3449260978] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f04240635d0 con 0x7f04080ad4b0 2024-04-25T14:38:56.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.923+0000 7f041be00700 1 -- 172.21.15.53:0/3293772916 <== osd.9 v2:172.21.15.53:6826/3449260978 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f04240635d0 con 0x7f04080ad4b0 2024-04-25T14:38:56.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.923+0000 7f042a600700 1 -- 172.21.15.53:0/3293772916 >> [v2:172.21.15.53:6826/3449260978,v1:172.21.15.53:6827/3449260978] conn(0x7f04080ad4b0 msgr2=0x7f04080af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:56.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.923+0000 7f042a600700 1 --2- 172.21.15.53:0/3293772916 >> [v2:172.21.15.53:6826/3449260978,v1:172.21.15.53:6827/3449260978] conn(0x7f04080ad4b0 0x7f04080af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:56.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.923+0000 7f042a600700 1 -- 172.21.15.53:0/3293772916 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f04080a3cd0 msgr2=0x7f04080a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:56.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.923+0000 7f042a600700 1 --2- 172.21.15.53:0/3293772916 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f04080a3cd0 0x7f04080a6190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0414009fd0 tx=0x7f0414019040 comp rx=0 tx=0).stop 2024-04-25T14:38:56.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.923+0000 7f042a600700 1 -- 172.21.15.53:0/3293772916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f042406b0f0 msgr2=0x7f0424199640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:56.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.923+0000 7f042a600700 1 --2- 172.21.15.53:0/3293772916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f042406b0f0 0x7f0424199640 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f041000ec10 tx=0x7f041000ef20 comp rx=0 tx=0).stop 2024-04-25T14:38:56.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.923+0000 7f0423e00700 1 -- 172.21.15.53:0/3293772916 reap_dead start 2024-04-25T14:38:56.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.923+0000 7f042a600700 1 -- 172.21.15.53:0/3293772916 shutdown_connections 2024-04-25T14:38:56.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.923+0000 7f042a600700 1 -- 172.21.15.53:0/3293772916 >> 172.21.15.53:0/3293772916 conn(0x7f04240788f0 msgr2=0x7f04240ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:56.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.924+0000 7f042a600700 1 -- 172.21.15.53:0/3293772916 shutdown_connections 2024-04-25T14:38:56.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:56.924+0000 7f042a600700 1 -- 172.21.15.53:0/3293772916 wait complete. 2024-04-25T14:38:57.612 INFO:teuthology.orchestra.run.smithi053.stdout:111669149762 2024-04-25T14:38:57.613 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-25T14:38:57.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.591+0000 7f5bdf400700 1 -- 172.21.15.53:0/372764962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bd8074c70 msgr2=0x7f5bd8075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:57.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.591+0000 7f5bdf400700 1 --2- 172.21.15.53:0/372764962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bd8074c70 0x7f5bd8075050 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc0098f0 tx=0x7f5bcc009c00 comp rx=0 tx=0).stop 2024-04-25T14:38:57.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.592+0000 7f5bdf400700 1 -- 172.21.15.53:0/372764962 shutdown_connections 2024-04-25T14:38:57.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.592+0000 7f5bdf400700 1 --2- 172.21.15.53:0/372764962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5bd810f470 0x7f5bd8111860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:57.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.592+0000 7f5bdf400700 1 --2- 172.21.15.53:0/372764962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bd8074c70 0x7f5bd8075050 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:57.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.592+0000 7f5bdf400700 1 --2- 172.21.15.53:0/372764962 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd8075590 0x7f5bd810ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:57.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.592+0000 7f5bdf400700 1 -- 172.21.15.53:0/372764962 >> 172.21.15.53:0/372764962 conn(0x7f5bd8079b10 msgr2=0x7f5bd8079f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:57.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.592+0000 7f5bdf400700 1 -- 172.21.15.53:0/372764962 shutdown_connections 2024-04-25T14:38:57.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.592+0000 7f5bdf400700 1 -- 172.21.15.53:0/372764962 wait complete. 2024-04-25T14:38:57.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.593+0000 7f5bdf400700 1 Processor -- start 2024-04-25T14:38:57.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.593+0000 7f5bdf400700 1 -- start start 2024-04-25T14:38:57.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.593+0000 7f5bdf400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5bd8074c70 0x7f5bd819b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:57.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.593+0000 7f5bdf400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bd8075590 0x7f5bd819b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:57.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.593+0000 7f5bdf400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd810f470 0x7f5bd819fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:57.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.593+0000 7f5bdf400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd8114e80 con 0x7f5bd810f470 2024-04-25T14:38:57.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.593+0000 7f5bdf400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd8114d00 con 0x7f5bd8075590 2024-04-25T14:38:57.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.593+0000 7f5bdf400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd8115000 con 0x7f5bd8074c70 2024-04-25T14:38:57.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.594+0000 7f5bdd400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd810f470 0x7f5bd819fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:57.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.594+0000 7f5bdd400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd810f470 0x7f5bd819fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55640/0 (socket says 172.21.15.53:55640) 2024-04-25T14:38:57.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.594+0000 7f5bdca00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5bd8074c70 0x7f5bd819b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:57.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.594+0000 7f5bdd400700 1 -- 172.21.15.53:0/1511439093 learned_addr learned my addr 172.21.15.53:0/1511439093 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:38:57.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.594+0000 7f5bd7e00700 1 --2- 172.21.15.53:0/1511439093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bd8075590 0x7f5bd819b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:57.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.594+0000 7f5bdca00700 1 -- 172.21.15.53:0/1511439093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bd8075590 msgr2=0x7f5bd819b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:57.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.594+0000 7f5bdca00700 1 --2- 172.21.15.53:0/1511439093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bd8075590 0x7f5bd819b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:57.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.594+0000 7f5bdca00700 1 -- 172.21.15.53:0/1511439093 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd810f470 msgr2=0x7f5bd819fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:57.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.594+0000 7f5bdca00700 1 --2- 172.21.15.53:0/1511439093 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd810f470 0x7f5bd819fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:57.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.594+0000 7f5bdca00700 1 -- 172.21.15.53:0/1511439093 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bd81a0350 con 0x7f5bd8074c70 2024-04-25T14:38:57.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.594+0000 7f5bd7e00700 1 --2- 172.21.15.53:0/1511439093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bd8075590 0x7f5bd819b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:38:57.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.595+0000 7f5bdca00700 1 --2- 172.21.15.53:0/1511439093 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5bd8074c70 0x7f5bd819b390 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc00c010 tx=0x7f5bcc00fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:57.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.595+0000 7f5bd5600700 1 -- 172.21.15.53:0/1511439093 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc0050c0 con 0x7f5bd8074c70 2024-04-25T14:38:57.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.595+0000 7f5bd5600700 1 -- 172.21.15.53:0/1511439093 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bcc00b560 con 0x7f5bd8074c70 2024-04-25T14:38:57.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.595+0000 7f5bdf400700 1 -- 172.21.15.53:0/1511439093 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bd81a05e0 con 0x7f5bd8074c70 2024-04-25T14:38:57.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.595+0000 7f5bd5600700 1 -- 172.21.15.53:0/1511439093 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc003a00 con 0x7f5bd8074c70 2024-04-25T14:38:57.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.595+0000 7f5bdf400700 1 -- 172.21.15.53:0/1511439093 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bd81a7ec0 con 0x7f5bd8074c70 2024-04-25T14:38:57.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.596+0000 7f5bdf400700 1 -- 172.21.15.53:0/1511439093 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5bd8063150 con 0x7f5bd8074c70 2024-04-25T14:38:57.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.600+0000 7f5bd5600700 1 -- 172.21.15.53:0/1511439093 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5bcc00b710 con 0x7f5bd8074c70 2024-04-25T14:38:57.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.601+0000 7f5bd5600700 1 --2- 172.21.15.53:0/1511439093 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5bc00a3cd0 0x7f5bc00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:57.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.601+0000 7f5bd5600700 1 -- 172.21.15.53:0/1511439093 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f5bcc0d1690 con 0x7f5bd8074c70 2024-04-25T14:38:57.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.602+0000 7f5bd7e00700 1 --2- 172.21.15.53:0/1511439093 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5bc00a3cd0 0x7f5bc00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:57.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.602+0000 7f5bd5600700 1 --2- 172.21.15.53:0/1511439093 >> [v2:172.21.15.150:6824/3435992308,v1:172.21.15.150:6825/3435992308] conn(0x7f5bc00ad4b0 0x7f5bc00af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:57.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.602+0000 7f5bd5600700 1 -- 172.21.15.53:0/1511439093 --> [v2:172.21.15.150:6824/3435992308,v1:172.21.15.150:6825/3435992308] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5bc00aff80 con 0x7f5bc00ad4b0 2024-04-25T14:38:57.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.602+0000 7f5bd5600700 1 -- 172.21.15.53:0/1511439093 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f5bcc0d1a40 con 0x7f5bd8074c70 2024-04-25T14:38:57.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.602+0000 7f5bdd400700 1 --2- 172.21.15.53:0/1511439093 >> [v2:172.21.15.150:6824/3435992308,v1:172.21.15.150:6825/3435992308] conn(0x7f5bc00ad4b0 0x7f5bc00af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:57.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.603+0000 7f5bd7e00700 1 --2- 172.21.15.53:0/1511439093 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5bc00a3cd0 0x7f5bc00a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4009750 tx=0x7f5bc4009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:57.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.603+0000 7f5bdd400700 1 --2- 172.21.15.53:0/1511439093 >> [v2:172.21.15.150:6824/3435992308,v1:172.21.15.150:6825/3435992308] conn(0x7f5bc00ad4b0 0x7f5bc00af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:57.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.604+0000 7f5bd5600700 1 -- 172.21.15.53:0/1511439093 <== osd.10 v2:172.21.15.150:6824/3435992308 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5bc00aff80 con 0x7f5bc00ad4b0 2024-04-25T14:38:57.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.622+0000 7f5bdf400700 1 -- 172.21.15.53:0/1511439093 --> [v2:172.21.15.150:6824/3435992308,v1:172.21.15.150:6825/3435992308] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5bd80635d0 con 0x7f5bc00ad4b0 2024-04-25T14:38:57.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.623+0000 7f5bd5600700 1 -- 172.21.15.53:0/1511439093 <== osd.10 v2:172.21.15.150:6824/3435992308 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5bd80635d0 con 0x7f5bc00ad4b0 2024-04-25T14:38:57.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.624+0000 7f5bdf400700 1 -- 172.21.15.53:0/1511439093 >> [v2:172.21.15.150:6824/3435992308,v1:172.21.15.150:6825/3435992308] conn(0x7f5bc00ad4b0 msgr2=0x7f5bc00af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:57.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.624+0000 7f5bdf400700 1 --2- 172.21.15.53:0/1511439093 >> [v2:172.21.15.150:6824/3435992308,v1:172.21.15.150:6825/3435992308] conn(0x7f5bc00ad4b0 0x7f5bc00af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:57.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.624+0000 7f5bdf400700 1 -- 172.21.15.53:0/1511439093 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5bc00a3cd0 msgr2=0x7f5bc00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:57.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.624+0000 7f5bdf400700 1 --2- 172.21.15.53:0/1511439093 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5bc00a3cd0 0x7f5bc00a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4009750 tx=0x7f5bc4009310 comp rx=0 tx=0).stop 2024-04-25T14:38:57.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.624+0000 7f5bdf400700 1 -- 172.21.15.53:0/1511439093 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5bd8074c70 msgr2=0x7f5bd819b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:57.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.624+0000 7f5bdf400700 1 --2- 172.21.15.53:0/1511439093 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5bd8074c70 0x7f5bd819b390 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc00c010 tx=0x7f5bcc00fa00 comp rx=0 tx=0).stop 2024-04-25T14:38:57.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.624+0000 7f5bdd400700 1 -- 172.21.15.53:0/1511439093 reap_dead start 2024-04-25T14:38:57.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.624+0000 7f5bdf400700 1 -- 172.21.15.53:0/1511439093 shutdown_connections 2024-04-25T14:38:57.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.624+0000 7f5bdf400700 1 -- 172.21.15.53:0/1511439093 >> 172.21.15.53:0/1511439093 conn(0x7f5bd8079b10 msgr2=0x7f5bd80ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:57.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.624+0000 7f5bdf400700 1 -- 172.21.15.53:0/1511439093 shutdown_connections 2024-04-25T14:38:57.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:57.624+0000 7f5bdf400700 1 -- 172.21.15.53:0/1511439093 wait complete. 2024-04-25T14:38:58.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.301+0000 7feacb200700 1 -- 172.21.15.53:0/4128468608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feac406b0f0 msgr2=0x7feac410f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:58.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.301+0000 7feacb200700 1 --2- 172.21.15.53:0/4128468608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feac406b0f0 0x7feac410f900 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7feabc00b0a0 tx=0x7feabc00b3b0 comp rx=0 tx=0).stop 2024-04-25T14:38:58.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.301+0000 7feacb200700 1 -- 172.21.15.53:0/4128468608 shutdown_connections 2024-04-25T14:38:58.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.301+0000 7feacb200700 1 --2- 172.21.15.53:0/4128468608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feac406b0f0 0x7feac410f900 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:58.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.301+0000 7feacb200700 1 --2- 172.21.15.53:0/4128468608 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feac4108a30 0x7feac4108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:58.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.301+0000 7feacb200700 1 --2- 172.21.15.53:0/4128468608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feac406a750 0x7feac406abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:58.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.301+0000 7feacb200700 1 -- 172.21.15.53:0/4128468608 >> 172.21.15.53:0/4128468608 conn(0x7feac40788f0 msgr2=0x7feac4078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:58.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.301+0000 7feacb200700 1 -- 172.21.15.53:0/4128468608 shutdown_connections 2024-04-25T14:38:58.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.301+0000 7feacb200700 1 -- 172.21.15.53:0/4128468608 wait complete. 2024-04-25T14:38:58.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.302+0000 7feacb200700 1 Processor -- start 2024-04-25T14:38:58.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.302+0000 7feacb200700 1 -- start start 2024-04-25T14:38:58.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.303+0000 7feacb200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feac406a750 0x7feac4199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:58.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.303+0000 7feacb200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feac406b0f0 0x7feac41996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:58.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.303+0000 7feacb200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feac4108a30 0x7feac419da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:58.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.303+0000 7feacb200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7feac4112c20 con 0x7feac406a750 2024-04-25T14:38:58.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.303+0000 7feacb200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7feac4112aa0 con 0x7feac406b0f0 2024-04-25T14:38:58.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.303+0000 7feacb200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feac4112da0 con 0x7feac4108a30 2024-04-25T14:38:58.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.303+0000 7feac3e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feac406a750 0x7feac4199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:58.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.303+0000 7feac3e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feac406a750 0x7feac4199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55564/0 (socket says 172.21.15.53:55564) 2024-04-25T14:38:58.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.303+0000 7feac3e00700 1 -- 172.21.15.53:0/3597202087 learned_addr learned my addr 172.21.15.53:0/3597202087 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:38:58.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.303+0000 7feac9200700 1 --2- 172.21.15.53:0/3597202087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feac4108a30 0x7feac419da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:58.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.303+0000 7feac3400700 1 --2- 172.21.15.53:0/3597202087 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feac406b0f0 0x7feac41996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:58.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.304+0000 7feac3400700 1 -- 172.21.15.53:0/3597202087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feac4108a30 msgr2=0x7feac419da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:58.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.304+0000 7feac3400700 1 --2- 172.21.15.53:0/3597202087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feac4108a30 0x7feac419da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:58.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.304+0000 7feac3400700 1 -- 172.21.15.53:0/3597202087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feac406a750 msgr2=0x7feac4199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:58.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.304+0000 7feac3400700 1 --2- 172.21.15.53:0/3597202087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feac406a750 0x7feac4199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:58.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.304+0000 7feac3400700 1 -- 172.21.15.53:0/3597202087 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac419e150 con 0x7feac406b0f0 2024-04-25T14:38:58.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.304+0000 7feac3e00700 1 --2- 172.21.15.53:0/3597202087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feac406a750 0x7feac4199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:38:58.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.304+0000 7feac9200700 1 --2- 172.21.15.53:0/3597202087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feac4108a30 0x7feac419da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:38:58.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.304+0000 7feac3400700 1 --2- 172.21.15.53:0/3597202087 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feac406b0f0 0x7feac41996d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7feab000e9f0 tx=0x7feab000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:58.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.304+0000 7feac0c00700 1 -- 172.21.15.53:0/3597202087 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab000c9c0 con 0x7feac406b0f0 2024-04-25T14:38:58.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.304+0000 7feac0c00700 1 -- 172.21.15.53:0/3597202087 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feab000cb60 con 0x7feac406b0f0 2024-04-25T14:38:58.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.304+0000 7feacb200700 1 -- 172.21.15.53:0/3597202087 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feac419e440 con 0x7feac406b0f0 2024-04-25T14:38:58.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.304+0000 7feac0c00700 1 -- 172.21.15.53:0/3597202087 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab0018680 con 0x7feac406b0f0 2024-04-25T14:38:58.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.304+0000 7feacb200700 1 -- 172.21.15.53:0/3597202087 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feac41a5cc0 con 0x7feac406b0f0 2024-04-25T14:38:58.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.305+0000 7feacb200700 1 -- 172.21.15.53:0/3597202087 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7feac40631d0 con 0x7feac406b0f0 2024-04-25T14:38:58.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.307+0000 7feac0c00700 1 -- 172.21.15.53:0/3597202087 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7feab0010430 con 0x7feac406b0f0 2024-04-25T14:38:58.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.309+0000 7feac0c00700 1 --2- 172.21.15.53:0/3597202087 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7feaa80a3cd0 0x7feaa80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:58.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.309+0000 7feac0c00700 1 -- 172.21.15.53:0/3597202087 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7feab00d0bd0 con 0x7feac406b0f0 2024-04-25T14:38:58.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.309+0000 7feac3e00700 1 --2- 172.21.15.53:0/3597202087 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7feaa80a3cd0 0x7feaa80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:58.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.309+0000 7feac0c00700 1 --2- 172.21.15.53:0/3597202087 >> [v2:172.21.15.53:6802/464176015,v1:172.21.15.53:6803/464176015] conn(0x7feaa80ad4b0 0x7feaa80af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:38:58.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.309+0000 7feac0c00700 1 -- 172.21.15.53:0/3597202087 --> [v2:172.21.15.53:6802/464176015,v1:172.21.15.53:6803/464176015] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feab0019ec0 con 0x7feaa80ad4b0 2024-04-25T14:38:58.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.309+0000 7feac0c00700 1 -- 172.21.15.53:0/3597202087 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7feab00d0f80 con 0x7feac406b0f0 2024-04-25T14:38:58.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.309+0000 7feac9200700 1 --2- 172.21.15.53:0/3597202087 >> [v2:172.21.15.53:6802/464176015,v1:172.21.15.53:6803/464176015] conn(0x7feaa80ad4b0 0x7feaa80af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:38:58.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.310+0000 7feac3e00700 1 --2- 172.21.15.53:0/3597202087 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7feaa80a3cd0 0x7feaa80a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7feab4011550 tx=0x7feab4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:58.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.310+0000 7feac9200700 1 --2- 172.21.15.53:0/3597202087 >> [v2:172.21.15.53:6802/464176015,v1:172.21.15.53:6803/464176015] conn(0x7feaa80ad4b0 0x7feaa80af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:38:58.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.311+0000 7feac0c00700 1 -- 172.21.15.53:0/3597202087 <== osd.0 v2:172.21.15.53:6802/464176015 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feab0019ec0 con 0x7feaa80ad4b0 2024-04-25T14:38:58.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.329+0000 7feacb200700 1 -- 172.21.15.53:0/3597202087 --> [v2:172.21.15.53:6802/464176015,v1:172.21.15.53:6803/464176015] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7feac4063650 con 0x7feaa80ad4b0 2024-04-25T14:38:58.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.330+0000 7feac0c00700 1 -- 172.21.15.53:0/3597202087 <== osd.0 v2:172.21.15.53:6802/464176015 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7feac4063650 con 0x7feaa80ad4b0 2024-04-25T14:38:58.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.331+0000 7feacb200700 1 -- 172.21.15.53:0/3597202087 >> [v2:172.21.15.53:6802/464176015,v1:172.21.15.53:6803/464176015] conn(0x7feaa80ad4b0 msgr2=0x7feaa80af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:58.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.331+0000 7feacb200700 1 --2- 172.21.15.53:0/3597202087 >> [v2:172.21.15.53:6802/464176015,v1:172.21.15.53:6803/464176015] conn(0x7feaa80ad4b0 0x7feaa80af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:38:58.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.331+0000 7feacb200700 1 -- 172.21.15.53:0/3597202087 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7feaa80a3cd0 msgr2=0x7feaa80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:58.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.331+0000 7feacb200700 1 --2- 172.21.15.53:0/3597202087 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7feaa80a3cd0 0x7feaa80a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7feab4011550 tx=0x7feab4009260 comp rx=0 tx=0).stop 2024-04-25T14:38:58.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.331+0000 7feacb200700 1 -- 172.21.15.53:0/3597202087 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feac406b0f0 msgr2=0x7feac41996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:38:58.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.331+0000 7feacb200700 1 --2- 172.21.15.53:0/3597202087 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feac406b0f0 0x7feac41996d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7feab000e9f0 tx=0x7feab000ed00 comp rx=0 tx=0).stop 2024-04-25T14:38:58.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.331+0000 7feac9200700 1 -- 172.21.15.53:0/3597202087 reap_dead start 2024-04-25T14:38:58.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.331+0000 7feacb200700 1 -- 172.21.15.53:0/3597202087 shutdown_connections 2024-04-25T14:38:58.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.331+0000 7feacb200700 1 -- 172.21.15.53:0/3597202087 >> 172.21.15.53:0/3597202087 conn(0x7feac40788f0 msgr2=0x7feac40ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:38:58.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.331+0000 7feacb200700 1 -- 172.21.15.53:0/3597202087 shutdown_connections 2024-04-25T14:38:58.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:38:53.331+0000 7feacb200700 1 -- 172.21.15.53:0/3597202087 wait complete. 2024-04-25T14:38:58.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:38:58 smithi053 ceph-mon[27773]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-25T14:38:58.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:38:58 smithi183 ceph-mon[32584]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-25T14:38:58.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:38:58 smithi150 ceph-mon[32504]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-25T14:38:58.577 INFO:teuthology.orchestra.run.smithi053.stdout:146028888127 2024-04-25T14:38:58.577 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-25T14:38:59.797 INFO:teuthology.orchestra.run.smithi053.stdout:34359738451 2024-04-25T14:38:59.797 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-25T14:38:59.899 INFO:teuthology.orchestra.run.smithi053.stdout:77309411401 2024-04-25T14:38:59.899 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-25T14:39:00.170 INFO:teuthology.orchestra.run.smithi053.stdout:171798691896 2024-04-25T14:39:00.170 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-25T14:39:00.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:00 smithi183 ceph-mon[32584]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T14:39:00.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:00 smithi150 ceph-mon[32504]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T14:39:00.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:00 smithi053 ceph-mon[27773]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T14:39:01.150 INFO:teuthology.orchestra.run.smithi053.stdout:55834574925 2024-04-25T14:39:01.150 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-25T14:39:01.366 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:39:01.825 INFO:teuthology.orchestra.run.smithi053.stdout:322122547229 2024-04-25T14:39:01.826 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-25T14:39:01.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.841+0000 7f2ed8e00700 1 -- 172.21.15.53:0/476116843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ed4075590 msgr2=0x7f2ed410ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:01.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.841+0000 7f2ed8e00700 1 --2- 172.21.15.53:0/476116843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ed4075590 0x7f2ed410ef00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0009990 tx=0x7f2ec0009ca0 comp rx=0 tx=0).stop 2024-04-25T14:39:01.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.841+0000 7f2ed8e00700 1 -- 172.21.15.53:0/476116843 shutdown_connections 2024-04-25T14:39:01.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.841+0000 7f2ed8e00700 1 --2- 172.21.15.53:0/476116843 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ed410f440 0x7f2ed4111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:01.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.841+0000 7f2ed8e00700 1 --2- 172.21.15.53:0/476116843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ed4074c70 0x7f2ed4075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:01.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.841+0000 7f2ed8e00700 1 --2- 172.21.15.53:0/476116843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ed4075590 0x7f2ed410ef00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:01.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.841+0000 7f2ed8e00700 1 -- 172.21.15.53:0/476116843 >> 172.21.15.53:0/476116843 conn(0x7f2ed4079ba0 msgr2=0x7f2ed4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:01.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.841+0000 7f2ed8e00700 1 -- 172.21.15.53:0/476116843 shutdown_connections 2024-04-25T14:39:01.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.842+0000 7f2ed8e00700 1 -- 172.21.15.53:0/476116843 wait complete. 2024-04-25T14:39:01.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.843+0000 7f2ed8e00700 1 Processor -- start 2024-04-25T14:39:01.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.844+0000 7f2ed8e00700 1 -- start start 2024-04-25T14:39:01.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.844+0000 7f2ed8e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ed4074c70 0x7f2ed419b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:01.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.844+0000 7f2ed8e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ed4075590 0x7f2ed419b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:01.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.844+0000 7f2ed8e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ed410f440 0x7f2ed419fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:01.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.844+0000 7f2ed8e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed4114e70 con 0x7f2ed4074c70 2024-04-25T14:39:01.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.844+0000 7f2ed8e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed4114cf0 con 0x7f2ed410f440 2024-04-25T14:39:01.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.844+0000 7f2ed8e00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed4114ff0 con 0x7f2ed4075590 2024-04-25T14:39:01.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.844+0000 7f2ed1e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ed4074c70 0x7f2ed419b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:01.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.844+0000 7f2ed1e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ed4074c70 0x7f2ed419b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55654/0 (socket says 172.21.15.53:55654) 2024-04-25T14:39:01.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.844+0000 7f2ed1400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ed4075590 0x7f2ed419b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:01.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.844+0000 7f2ed1e00700 1 -- 172.21.15.53:0/2675417833 learned_addr learned my addr 172.21.15.53:0/2675417833 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:01.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.844+0000 7f2ed2800700 1 --2- 172.21.15.53:0/2675417833 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ed410f440 0x7f2ed419fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:01.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.845+0000 7f2ed1400700 1 -- 172.21.15.53:0/2675417833 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ed410f440 msgr2=0x7f2ed419fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:01.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.845+0000 7f2ed1400700 1 --2- 172.21.15.53:0/2675417833 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ed410f440 0x7f2ed419fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:01.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.845+0000 7f2ed1400700 1 -- 172.21.15.53:0/2675417833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ed4074c70 msgr2=0x7f2ed419b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:01.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.845+0000 7f2ed1400700 1 --2- 172.21.15.53:0/2675417833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ed4074c70 0x7f2ed419b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:01.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.845+0000 7f2ed1400700 1 -- 172.21.15.53:0/2675417833 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed41a0420 con 0x7f2ed4075590 2024-04-25T14:39:01.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.845+0000 7f2ed1e00700 1 --2- 172.21.15.53:0/2675417833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ed4074c70 0x7f2ed419b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:39:01.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.845+0000 7f2ed1400700 1 --2- 172.21.15.53:0/2675417833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ed4075590 0x7f2ed419b950 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2ec00098e0 tx=0x7f2ec00057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:01.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.846+0000 7f2ecaa00700 1 -- 172.21.15.53:0/2675417833 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec0027070 con 0x7f2ed4075590 2024-04-25T14:39:01.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.846+0000 7f2ecaa00700 1 -- 172.21.15.53:0/2675417833 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ec0022470 con 0x7f2ed4075590 2024-04-25T14:39:01.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.846+0000 7f2ed8e00700 1 -- 172.21.15.53:0/2675417833 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ed41a06b0 con 0x7f2ed4075590 2024-04-25T14:39:01.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.846+0000 7f2ecaa00700 1 -- 172.21.15.53:0/2675417833 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec0017650 con 0x7f2ed4075590 2024-04-25T14:39:01.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.846+0000 7f2ed8e00700 1 -- 172.21.15.53:0/2675417833 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ed41a7f90 con 0x7f2ed4075590 2024-04-25T14:39:01.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.847+0000 7f2ed8e00700 1 -- 172.21.15.53:0/2675417833 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2ed4063150 con 0x7f2ed4075590 2024-04-25T14:39:01.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.849+0000 7f2ecaa00700 1 -- 172.21.15.53:0/2675417833 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2ec002a070 con 0x7f2ed4075590 2024-04-25T14:39:01.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.850+0000 7f2ecaa00700 1 --2- 172.21.15.53:0/2675417833 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f2eb80a3cd0 0x7f2eb80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:01.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.850+0000 7f2ecaa00700 1 -- 172.21.15.53:0/2675417833 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f2ec00d1160 con 0x7f2ed4075590 2024-04-25T14:39:01.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.850+0000 7f2ed1e00700 1 --2- 172.21.15.53:0/2675417833 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f2eb80a3cd0 0x7f2eb80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:01.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.851+0000 7f2ecaa00700 1 --2- 172.21.15.53:0/2675417833 >> [v2:172.21.15.183:6824/1957842686,v1:172.21.15.183:6825/1957842686] conn(0x7f2eb80ad510 0x7f2eb80af930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:01.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.851+0000 7f2ecaa00700 1 -- 172.21.15.53:0/2675417833 --> [v2:172.21.15.183:6824/1957842686,v1:172.21.15.183:6825/1957842686] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2ec0001d30 con 0x7f2eb80ad510 2024-04-25T14:39:01.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.851+0000 7f2ecaa00700 1 -- 172.21.15.53:0/2675417833 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f2ec00d1510 con 0x7f2ed4075590 2024-04-25T14:39:01.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.851+0000 7f2ed1e00700 1 --2- 172.21.15.53:0/2675417833 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f2eb80a3cd0 0x7f2eb80a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4011550 tx=0x7f2ec4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:01.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.851+0000 7f2ed2800700 1 --2- 172.21.15.53:0/2675417833 >> [v2:172.21.15.183:6824/1957842686,v1:172.21.15.183:6825/1957842686] conn(0x7f2eb80ad510 0x7f2eb80af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:01.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.852+0000 7f2ed2800700 1 --2- 172.21.15.53:0/2675417833 >> [v2:172.21.15.183:6824/1957842686,v1:172.21.15.183:6825/1957842686] conn(0x7f2eb80ad510 0x7f2eb80af930 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:01.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.853+0000 7f2ecaa00700 1 -- 172.21.15.53:0/2675417833 <== osd.11 v2:172.21.15.183:6824/1957842686 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2ec0001d30 con 0x7f2eb80ad510 2024-04-25T14:39:01.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.872+0000 7f2ed8e00700 1 -- 172.21.15.53:0/2675417833 --> [v2:172.21.15.183:6824/1957842686,v1:172.21.15.183:6825/1957842686] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2ed40635d0 con 0x7f2eb80ad510 2024-04-25T14:39:01.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.872+0000 7f2ecaa00700 1 -- 172.21.15.53:0/2675417833 <== osd.11 v2:172.21.15.183:6824/1957842686 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2ed40635d0 con 0x7f2eb80ad510 2024-04-25T14:39:01.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.873+0000 7f2ed8e00700 1 -- 172.21.15.53:0/2675417833 >> [v2:172.21.15.183:6824/1957842686,v1:172.21.15.183:6825/1957842686] conn(0x7f2eb80ad510 msgr2=0x7f2eb80af930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:01.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.873+0000 7f2ed8e00700 1 --2- 172.21.15.53:0/2675417833 >> [v2:172.21.15.183:6824/1957842686,v1:172.21.15.183:6825/1957842686] conn(0x7f2eb80ad510 0x7f2eb80af930 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:01.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.873+0000 7f2ed8e00700 1 -- 172.21.15.53:0/2675417833 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f2eb80a3cd0 msgr2=0x7f2eb80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:01.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.873+0000 7f2ed8e00700 1 --2- 172.21.15.53:0/2675417833 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f2eb80a3cd0 0x7f2eb80a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4011550 tx=0x7f2ec4009260 comp rx=0 tx=0).stop 2024-04-25T14:39:01.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.873+0000 7f2ed8e00700 1 -- 172.21.15.53:0/2675417833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ed4075590 msgr2=0x7f2ed419b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:01.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.873+0000 7f2ed8e00700 1 --2- 172.21.15.53:0/2675417833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ed4075590 0x7f2ed419b950 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2ec00098e0 tx=0x7f2ec00057c0 comp rx=0 tx=0).stop 2024-04-25T14:39:01.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.873+0000 7f2ed2800700 1 -- 172.21.15.53:0/2675417833 reap_dead start 2024-04-25T14:39:01.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.873+0000 7f2ed8e00700 1 -- 172.21.15.53:0/2675417833 shutdown_connections 2024-04-25T14:39:01.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.873+0000 7f2ed8e00700 1 -- 172.21.15.53:0/2675417833 >> 172.21.15.53:0/2675417833 conn(0x7f2ed4079ba0 msgr2=0x7f2ed4101060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:01.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.873+0000 7f2ed8e00700 1 -- 172.21.15.53:0/2675417833 shutdown_connections 2024-04-25T14:39:01.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:01.873+0000 7f2ed8e00700 1 -- 172.21.15.53:0/2675417833 wait complete. 2024-04-25T14:39:02.214 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:02 smithi053 ceph-mon[27773]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T14:39:02.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:02 smithi183 ceph-mon[32584]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T14:39:02.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:02 smithi150 ceph-mon[32504]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T14:39:03.301 INFO:teuthology.orchestra.run.smithi053.stdout:356482285593 2024-04-25T14:39:03.302 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-25T14:39:03.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:03 smithi183 ceph-mon[32584]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T14:39:03.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:03 smithi150 ceph-mon[32504]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T14:39:03.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:03 smithi053 ceph-mon[27773]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T14:39:04.541 INFO:teuthology.orchestra.run.smithi053.stdout:266287972393 2024-04-25T14:39:04.541 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-25T14:39:04.652 INFO:teuthology.orchestra.run.smithi053.stdout:300647710754 2024-04-25T14:39:04.652 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-25T14:39:05.363 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:39:05.398 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:39:05.410 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:39:05.508 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:39:05.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:05 smithi053 ceph-mon[27773]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T14:39:05.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:05 smithi183 ceph-mon[32584]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T14:39:05.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:05 smithi150 ceph-mon[32504]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T14:39:06.250 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:39:06.251 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:39:06.253 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:39:07.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:07 smithi183 ceph-mon[32584]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T14:39:07.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:07 smithi150 ceph-mon[32504]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T14:39:08.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:07 smithi053 ceph-mon[27773]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T14:39:08.295 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.294+0000 7f5d40800700 1 -- 172.21.15.53:0/2708724837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d3806b980 msgr2=0x7f5d3810f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:08.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.294+0000 7f5d40800700 1 --2- 172.21.15.53:0/2708724837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d3806b980 0x7f5d3810f8f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400b0a0 tx=0x7f5d3400b3b0 comp rx=0 tx=0).stop 2024-04-25T14:39:08.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.294+0000 7f5d40800700 1 -- 172.21.15.53:0/2708724837 shutdown_connections 2024-04-25T14:39:08.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.294+0000 7f5d40800700 1 --2- 172.21.15.53:0/2708724837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d3806b980 0x7f5d3810f8f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:08.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.294+0000 7f5d40800700 1 --2- 172.21.15.53:0/2708724837 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d38108a20 0x7f5d38108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:08.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.294+0000 7f5d40800700 1 --2- 172.21.15.53:0/2708724837 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d3806afe0 0x7f5d3806b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:08.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.294+0000 7f5d40800700 1 -- 172.21.15.53:0/2708724837 >> 172.21.15.53:0/2708724837 conn(0x7f5d380788b0 msgr2=0x7f5d38078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:08.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.294+0000 7f5d40800700 1 -- 172.21.15.53:0/2708724837 shutdown_connections 2024-04-25T14:39:08.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.294+0000 7f5d40800700 1 -- 172.21.15.53:0/2708724837 wait complete. 2024-04-25T14:39:08.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.296+0000 7f5d40800700 1 Processor -- start 2024-04-25T14:39:08.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.296+0000 7f5d40800700 1 -- start start 2024-04-25T14:39:08.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.296+0000 7f5d40800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d3806afe0 0x7f5d381991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:08.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.296+0000 7f5d40800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d3806b980 0x7f5d38199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:08.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.296+0000 7f5d40800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d38108a20 0x7f5d3819dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:08.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.296+0000 7f5d40800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5d38112c10 con 0x7f5d3806afe0 2024-04-25T14:39:08.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.296+0000 7f5d40800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5d38112a90 con 0x7f5d38108a20 2024-04-25T14:39:08.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.296+0000 7f5d40800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5d38112d90 con 0x7f5d3806b980 2024-04-25T14:39:08.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.297+0000 7f5d3de00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d3806afe0 0x7f5d381991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:08.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.297+0000 7f5d3d400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d3806b980 0x7f5d38199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:08.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.297+0000 7f5d3de00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d3806afe0 0x7f5d381991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41346/0 (socket says 172.21.15.53:41346) 2024-04-25T14:39:08.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.297+0000 7f5d3de00700 1 -- 172.21.15.53:0/1857560535 learned_addr learned my addr 172.21.15.53:0/1857560535 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:08.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.297+0000 7f5d3e800700 1 --2- 172.21.15.53:0/1857560535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d38108a20 0x7f5d3819dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:08.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.297+0000 7f5d3d400700 1 -- 172.21.15.53:0/1857560535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d38108a20 msgr2=0x7f5d3819dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:08.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.297+0000 7f5d3d400700 1 --2- 172.21.15.53:0/1857560535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d38108a20 0x7f5d3819dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:08.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.297+0000 7f5d3d400700 1 -- 172.21.15.53:0/1857560535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d3806afe0 msgr2=0x7f5d381991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:08.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.297+0000 7f5d3d400700 1 --2- 172.21.15.53:0/1857560535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d3806afe0 0x7f5d381991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:08.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.297+0000 7f5d3d400700 1 -- 172.21.15.53:0/1857560535 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d3819e1f0 con 0x7f5d3806b980 2024-04-25T14:39:08.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.297+0000 7f5d3de00700 1 --2- 172.21.15.53:0/1857560535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d3806afe0 0x7f5d381991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:39:08.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.298+0000 7f5d3d400700 1 --2- 172.21.15.53:0/1857560535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d3806b980 0x7f5d38199720 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5d2800e9f0 tx=0x7f5d2800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:08.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.298+0000 7f5d32a00700 1 -- 172.21.15.53:0/1857560535 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2800cb70 con 0x7f5d3806b980 2024-04-25T14:39:08.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.298+0000 7f5d40800700 1 -- 172.21.15.53:0/1857560535 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d3819e4e0 con 0x7f5d3806b980 2024-04-25T14:39:08.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.298+0000 7f5d32a00700 1 -- 172.21.15.53:0/1857560535 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d280044d0 con 0x7f5d3806b980 2024-04-25T14:39:08.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.298+0000 7f5d32a00700 1 -- 172.21.15.53:0/1857560535 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d28003e00 con 0x7f5d3806b980 2024-04-25T14:39:08.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.298+0000 7f5d40800700 1 -- 172.21.15.53:0/1857560535 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d381a5d60 con 0x7f5d3806b980 2024-04-25T14:39:08.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.299+0000 7f5d40800700 1 -- 172.21.15.53:0/1857560535 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5d38063150 con 0x7f5d3806b980 2024-04-25T14:39:08.302 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.301+0000 7f5d32a00700 1 -- 172.21.15.53:0/1857560535 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5d2800cd10 con 0x7f5d3806b980 2024-04-25T14:39:08.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.303+0000 7f5d32a00700 1 --2- 172.21.15.53:0/1857560535 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5d200a3cd0 0x7f5d200a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:08.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.303+0000 7f5d32a00700 1 -- 172.21.15.53:0/1857560535 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f5d280d0950 con 0x7f5d3806b980 2024-04-25T14:39:08.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.303+0000 7f5d3de00700 1 --2- 172.21.15.53:0/1857560535 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5d200a3cd0 0x7f5d200a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:08.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.303+0000 7f5d32a00700 1 --2- 172.21.15.53:0/1857560535 >> [v2:172.21.15.150:6816/229022562,v1:172.21.15.150:6817/229022562] conn(0x7f5d200ad4b0 0x7f5d200af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:08.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.303+0000 7f5d32a00700 1 -- 172.21.15.53:0/1857560535 --> [v2:172.21.15.150:6816/229022562,v1:172.21.15.150:6817/229022562] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5d200aff80 con 0x7f5d200ad4b0 2024-04-25T14:39:08.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.303+0000 7f5d32a00700 1 -- 172.21.15.53:0/1857560535 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f5d280d0d00 con 0x7f5d3806b980 2024-04-25T14:39:08.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.304+0000 7f5d3e800700 1 --2- 172.21.15.53:0/1857560535 >> [v2:172.21.15.150:6816/229022562,v1:172.21.15.150:6817/229022562] conn(0x7f5d200ad4b0 0x7f5d200af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:08.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.304+0000 7f5d3de00700 1 --2- 172.21.15.53:0/1857560535 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5d200a3cd0 0x7f5d200a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c011550 tx=0x7f5d2c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:08.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.304+0000 7f5d3e800700 1 --2- 172.21.15.53:0/1857560535 >> [v2:172.21.15.150:6816/229022562,v1:172.21.15.150:6817/229022562] conn(0x7f5d200ad4b0 0x7f5d200af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:08.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.305+0000 7f5d32a00700 1 -- 172.21.15.53:0/1857560535 <== osd.7 v2:172.21.15.150:6816/229022562 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5d200aff80 con 0x7f5d200ad4b0 2024-04-25T14:39:08.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.324+0000 7f5d40800700 1 -- 172.21.15.53:0/1857560535 --> [v2:172.21.15.150:6816/229022562,v1:172.21.15.150:6817/229022562] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5d38063550 con 0x7f5d200ad4b0 2024-04-25T14:39:08.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.325+0000 7f5d32a00700 1 -- 172.21.15.53:0/1857560535 <== osd.7 v2:172.21.15.150:6816/229022562 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5d38063550 con 0x7f5d200ad4b0 2024-04-25T14:39:08.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.325+0000 7f5d40800700 1 -- 172.21.15.53:0/1857560535 >> [v2:172.21.15.150:6816/229022562,v1:172.21.15.150:6817/229022562] conn(0x7f5d200ad4b0 msgr2=0x7f5d200af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:08.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.325+0000 7f5d40800700 1 --2- 172.21.15.53:0/1857560535 >> [v2:172.21.15.150:6816/229022562,v1:172.21.15.150:6817/229022562] conn(0x7f5d200ad4b0 0x7f5d200af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:08.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.325+0000 7f5d40800700 1 -- 172.21.15.53:0/1857560535 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5d200a3cd0 msgr2=0x7f5d200a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:08.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.325+0000 7f5d40800700 1 --2- 172.21.15.53:0/1857560535 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5d200a3cd0 0x7f5d200a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c011550 tx=0x7f5d2c009260 comp rx=0 tx=0).stop 2024-04-25T14:39:08.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.325+0000 7f5d40800700 1 -- 172.21.15.53:0/1857560535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d3806b980 msgr2=0x7f5d38199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:08.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.325+0000 7f5d40800700 1 --2- 172.21.15.53:0/1857560535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d3806b980 0x7f5d38199720 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5d2800e9f0 tx=0x7f5d2800ed00 comp rx=0 tx=0).stop 2024-04-25T14:39:08.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.326+0000 7f5d3e800700 1 -- 172.21.15.53:0/1857560535 reap_dead start 2024-04-25T14:39:08.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.326+0000 7f5d40800700 1 -- 172.21.15.53:0/1857560535 shutdown_connections 2024-04-25T14:39:08.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.326+0000 7f5d40800700 1 -- 172.21.15.53:0/1857560535 >> 172.21.15.53:0/1857560535 conn(0x7f5d380788b0 msgr2=0x7f5d380ffa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:08.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.326+0000 7f5d40800700 1 -- 172.21.15.53:0/1857560535 shutdown_connections 2024-04-25T14:39:08.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:08.326+0000 7f5d40800700 1 -- 172.21.15.53:0/1857560535 wait complete. 2024-04-25T14:39:09.776 INFO:teuthology.orchestra.run.smithi053.stdout:227633266735 2024-04-25T14:39:09.776 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-25T14:39:09.860 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:09 smithi053 ceph-mon[27773]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T14:39:09.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:09 smithi183 ceph-mon[32584]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T14:39:09.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:09 smithi150 ceph-mon[32504]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T14:39:10.029 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:39:10.030 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:39:10.030 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:39:10.036 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:39:11.945 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:11 smithi053 ceph-mon[27773]: pgmap v346: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-25T14:39:11.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:11 smithi183 ceph-mon[32584]: pgmap v346: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-25T14:39:11.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:11 smithi150 ceph-mon[32504]: pgmap v346: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-25T14:39:12.925 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:12 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:39:12.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:12 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:39:12.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:12 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:39:13.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:13 smithi183 ceph-mon[32584]: pgmap v347: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T14:39:13.959 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:13 smithi053 ceph-mon[27773]: pgmap v347: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T14:39:13.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:13 smithi150 ceph-mon[32504]: pgmap v347: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T14:39:15.035 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:14 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:39:15.035 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:14 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:39:15.035 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:14 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:39:15.035 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:14 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:39:15.035 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:14 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-04-25T14:39:15.035 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:14 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-04-25T14:39:15.035 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:14 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-04-25T14:39:15.035 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:14 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-04-25T14:39:15.035 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:14 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2024-04-25T14:39:15.035 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:14 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-04-25T14:39:15.035 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:14 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9, 5, 6]}]: dispatch 2024-04-25T14:39:15.035 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:14 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [5, 7]}]: dispatch 2024-04-25T14:39:15.036 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:14 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-04-25T14:39:15.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:14 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:39:15.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:14 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:39:15.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:14 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:39:15.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:14 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:39:15.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:14 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-04-25T14:39:15.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:14 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-04-25T14:39:15.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:14 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-04-25T14:39:15.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:14 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-04-25T14:39:15.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:14 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2024-04-25T14:39:15.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:14 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-04-25T14:39:15.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:14 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9, 5, 6]}]: dispatch 2024-04-25T14:39:15.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:14 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [5, 7]}]: dispatch 2024-04-25T14:39:15.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:14 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-04-25T14:39:15.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:14 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:39:15.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:14 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:39:15.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:14 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:39:15.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:14 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:39:15.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:14 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-04-25T14:39:15.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:14 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-04-25T14:39:15.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:14 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-04-25T14:39:15.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:14 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-04-25T14:39:15.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:14 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2024-04-25T14:39:15.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:14 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-04-25T14:39:15.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:14 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9, 5, 6]}]: dispatch 2024-04-25T14:39:15.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:14 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [5, 7]}]: dispatch 2024-04-25T14:39:15.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:14 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-04-25T14:39:16.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:15 smithi053 ceph-mon[27773]: pgmap v348: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T14:39:16.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:15 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-04-25T14:39:16.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:15 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-04-25T14:39:16.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:15 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-04-25T14:39:16.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:15 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-04-25T14:39:16.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:15 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2024-04-25T14:39:16.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:15 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-04-25T14:39:16.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:15 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9, 5, 6]}]': finished 2024-04-25T14:39:16.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:15 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [5, 7]}]': finished 2024-04-25T14:39:16.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:15 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-04-25T14:39:16.284 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:15 smithi053 ceph-mon[27773]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T14:39:16.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:15 smithi183 ceph-mon[32584]: pgmap v348: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T14:39:16.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:15 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-04-25T14:39:16.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:15 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-04-25T14:39:16.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:15 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-04-25T14:39:16.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:15 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-04-25T14:39:16.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:15 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2024-04-25T14:39:16.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:15 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-04-25T14:39:16.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:15 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9, 5, 6]}]': finished 2024-04-25T14:39:16.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:15 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [5, 7]}]': finished 2024-04-25T14:39:16.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:15 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-04-25T14:39:16.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:15 smithi183 ceph-mon[32584]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T14:39:16.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:15 smithi150 ceph-mon[32504]: pgmap v348: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T14:39:16.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:15 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-04-25T14:39:16.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:15 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-04-25T14:39:16.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:15 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-04-25T14:39:16.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:15 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-04-25T14:39:16.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:15 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2024-04-25T14:39:16.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:15 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-04-25T14:39:16.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:15 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9, 5, 6]}]': finished 2024-04-25T14:39:16.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:15 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [5, 7]}]': finished 2024-04-25T14:39:16.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:15 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-04-25T14:39:16.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:15 smithi150 ceph-mon[32504]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T14:39:17.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:16 smithi053 ceph-mon[27773]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T14:39:17.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:16 smithi183 ceph-mon[32584]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T14:39:17.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:16 smithi150 ceph-mon[32504]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T14:39:18.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:17 smithi053 ceph-mon[27773]: pgmap v351: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T14:39:18.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:17 smithi053 ceph-mon[27773]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-25T14:39:18.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:17 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/3991338222' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T14:39:18.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:17 smithi183 ceph-mon[32584]: pgmap v351: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T14:39:18.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:17 smithi183 ceph-mon[32584]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-25T14:39:18.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:17 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/3991338222' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T14:39:18.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:17 smithi150 ceph-mon[32504]: pgmap v351: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T14:39:18.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:17 smithi150 ceph-mon[32504]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-25T14:39:18.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:17 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/3991338222' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T14:39:20.128 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:19 smithi053 ceph-mon[27773]: pgmap v352: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 129 B/s, 0 objects/s recovering 2024-04-25T14:39:20.129 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:19 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1346741801' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T14:39:20.262 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:19 smithi150 ceph-mon[32504]: pgmap v352: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 129 B/s, 0 objects/s recovering 2024-04-25T14:39:20.262 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:19 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/1346741801' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T14:39:20.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:19 smithi183 ceph-mon[32584]: pgmap v352: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 129 B/s, 0 objects/s recovering 2024-04-25T14:39:20.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:19 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/1346741801' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T14:39:20.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.400+0000 7f44bb600700 1 -- 172.21.15.53:0/2907661431 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44b4109fa0 msgr2=0x7f44b4111b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:20.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.400+0000 7f44bb600700 1 --2- 172.21.15.53:0/2907661431 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44b4109fa0 0x7f44b4111b70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f44ac009fc0 tx=0x7f44ac00b330 comp rx=0 tx=0).stop 2024-04-25T14:39:20.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.400+0000 7f44bb600700 1 -- 172.21.15.53:0/2907661431 shutdown_connections 2024-04-25T14:39:20.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.400+0000 7f44bb600700 1 --2- 172.21.15.53:0/2907661431 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44b4109fa0 0x7f44b4111b70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:20.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.400+0000 7f44bb600700 1 --2- 172.21.15.53:0/2907661431 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44b4104ec0 0x7f44b41052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:20.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.400+0000 7f44bb600700 1 --2- 172.21.15.53:0/2907661431 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44b4105870 0x7f44b41098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:20.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.400+0000 7f44bb600700 1 -- 172.21.15.53:0/2907661431 >> 172.21.15.53:0/2907661431 conn(0x7f44b4100750 msgr2=0x7f44b4102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:20.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.400+0000 7f44bb600700 1 -- 172.21.15.53:0/2907661431 shutdown_connections 2024-04-25T14:39:20.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.401+0000 7f44bb600700 1 -- 172.21.15.53:0/2907661431 wait complete. 2024-04-25T14:39:20.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.402+0000 7f44bb600700 1 Processor -- start 2024-04-25T14:39:20.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.402+0000 7f44bb600700 1 -- start start 2024-04-25T14:39:20.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.403+0000 7f44bb600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44b4104ec0 0x7f44b419b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:20.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.403+0000 7f44bb600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44b4105870 0x7f44b419b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:20.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.403+0000 7f44bb600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44b4109fa0 0x7f44b419fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:20.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.403+0000 7f44bb600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f44b4114eb0 con 0x7f44b4105870 2024-04-25T14:39:20.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.403+0000 7f44bb600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f44b4114d30 con 0x7f44b4109fa0 2024-04-25T14:39:20.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.403+0000 7f44bb600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f44b4115030 con 0x7f44b4104ec0 2024-04-25T14:39:20.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.403+0000 7f44abe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44b4105870 0x7f44b419b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:20.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.403+0000 7f44b8c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44b4104ec0 0x7f44b419b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:20.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.403+0000 7f44b9600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44b4109fa0 0x7f44b419fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:20.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.403+0000 7f44b8c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44b4104ec0 0x7f44b419b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:41888/0 (socket says 172.21.15.53:41888) 2024-04-25T14:39:20.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.403+0000 7f44b8c00700 1 -- 172.21.15.53:0/248225752 learned_addr learned my addr 172.21.15.53:0/248225752 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:20.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.403+0000 7f44b8c00700 1 -- 172.21.15.53:0/248225752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44b4109fa0 msgr2=0x7f44b419fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:20.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.403+0000 7f44b8c00700 1 --2- 172.21.15.53:0/248225752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44b4109fa0 0x7f44b419fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:20.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.403+0000 7f44b8c00700 1 -- 172.21.15.53:0/248225752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44b4105870 msgr2=0x7f44b419b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:20.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.403+0000 7f44b8c00700 1 --2- 172.21.15.53:0/248225752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44b4105870 0x7f44b419b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:20.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.404+0000 7f44b8c00700 1 -- 172.21.15.53:0/248225752 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44b41a0310 con 0x7f44b4104ec0 2024-04-25T14:39:20.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.404+0000 7f44b9600700 1 --2- 172.21.15.53:0/248225752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44b4109fa0 0x7f44b419fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:39:20.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.404+0000 7f44abe00700 1 --2- 172.21.15.53:0/248225752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44b4105870 0x7f44b419b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:39:20.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.404+0000 7f44b8c00700 1 --2- 172.21.15.53:0/248225752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44b4104ec0 0x7f44b419b300 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f44a400e9c0 tx=0x7f44a400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:20.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.404+0000 7f44b2000700 1 -- 172.21.15.53:0/248225752 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a400cbd0 con 0x7f44b4104ec0 2024-04-25T14:39:20.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.404+0000 7f44b2000700 1 -- 172.21.15.53:0/248225752 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44a400ee50 con 0x7f44b4104ec0 2024-04-25T14:39:20.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.404+0000 7f44bb600700 1 -- 172.21.15.53:0/248225752 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44b41a0600 con 0x7f44b4104ec0 2024-04-25T14:39:20.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.404+0000 7f44b2000700 1 -- 172.21.15.53:0/248225752 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a4005550 con 0x7f44b4104ec0 2024-04-25T14:39:20.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.404+0000 7f44bb600700 1 -- 172.21.15.53:0/248225752 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44b41a7e80 con 0x7f44b4104ec0 2024-04-25T14:39:20.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.406+0000 7f44bb600700 1 -- 172.21.15.53:0/248225752 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44b4069730 con 0x7f44b4104ec0 2024-04-25T14:39:20.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.407+0000 7f44b2000700 1 -- 172.21.15.53:0/248225752 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f44a4004750 con 0x7f44b4104ec0 2024-04-25T14:39:20.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.409+0000 7f44b2000700 1 --2- 172.21.15.53:0/248225752 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f44940a3d50 0x7f44940a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:20.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.409+0000 7f44b2000700 1 -- 172.21.15.53:0/248225752 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f44a40d07d0 con 0x7f44b4104ec0 2024-04-25T14:39:20.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.409+0000 7f44abe00700 1 --2- 172.21.15.53:0/248225752 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f44940a3d50 0x7f44940a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:20.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.410+0000 7f44abe00700 1 --2- 172.21.15.53:0/248225752 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f44940a3d50 0x7f44940a6210 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f44b4103e40 tx=0x7f449c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:20.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.413+0000 7f44b2000700 1 -- 172.21.15.53:0/248225752 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f44a409cfe0 con 0x7f44b4104ec0 2024-04-25T14:39:20.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.659+0000 7f44bb600700 1 -- 172.21.15.53:0/248225752 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f44b402df80 con 0x7f44b4104ec0 2024-04-25T14:39:20.660 INFO:teuthology.orchestra.run.smithi053.stdout:77309411407 2024-04-25T14:39:20.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.660+0000 7f44b2000700 1 -- 172.21.15.53:0/248225752 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f44a409c8e0 con 0x7f44b4104ec0 2024-04-25T14:39:20.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.663+0000 7f44bb600700 1 -- 172.21.15.53:0/248225752 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f44940a3d50 msgr2=0x7f44940a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:20.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.663+0000 7f44bb600700 1 --2- 172.21.15.53:0/248225752 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f44940a3d50 0x7f44940a6210 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f44b4103e40 tx=0x7f449c009310 comp rx=0 tx=0).stop 2024-04-25T14:39:20.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.663+0000 7f44bb600700 1 -- 172.21.15.53:0/248225752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44b4104ec0 msgr2=0x7f44b419b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:20.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.663+0000 7f44bb600700 1 --2- 172.21.15.53:0/248225752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44b4104ec0 0x7f44b419b300 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f44a400e9c0 tx=0x7f44a400ecd0 comp rx=0 tx=0).stop 2024-04-25T14:39:20.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.664+0000 7f44bb600700 1 -- 172.21.15.53:0/248225752 shutdown_connections 2024-04-25T14:39:20.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.664+0000 7f44bb600700 1 --2- 172.21.15.53:0/248225752 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f44940a3d50 0x7f44940a6210 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:20.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.664+0000 7f44bb600700 1 --2- 172.21.15.53:0/248225752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44b4109fa0 0x7f44b419fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:20.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.664+0000 7f44bb600700 1 --2- 172.21.15.53:0/248225752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44b4104ec0 0x7f44b419b300 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:20.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.664+0000 7f44bb600700 1 --2- 172.21.15.53:0/248225752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44b4105870 0x7f44b419b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:20.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.664+0000 7f44bb600700 1 -- 172.21.15.53:0/248225752 >> 172.21.15.53:0/248225752 conn(0x7f44b4100750 msgr2=0x7f44b4101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:20.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.664+0000 7f44bb600700 1 -- 172.21.15.53:0/248225752 shutdown_connections 2024-04-25T14:39:20.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.664+0000 7f44bb600700 1 -- 172.21.15.53:0/248225752 wait complete. 2024-04-25T14:39:20.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.749+0000 7ff6a3e00700 1 -- 172.21.15.53:0/4066354730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff69c105890 msgr2=0x7ff69c1098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:20.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.749+0000 7ff6a3e00700 1 --2- 172.21.15.53:0/4066354730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff69c105890 0x7ff69c1098e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff6880098f0 tx=0x7ff688009c00 comp rx=0 tx=0).stop 2024-04-25T14:39:20.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.749+0000 7ff6a3e00700 1 -- 172.21.15.53:0/4066354730 shutdown_connections 2024-04-25T14:39:20.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.749+0000 7ff6a3e00700 1 --2- 172.21.15.53:0/4066354730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff69c109fc0 0x7ff69c111b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:20.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.749+0000 7ff6a3e00700 1 --2- 172.21.15.53:0/4066354730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff69c104ee0 0x7ff69c1052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:20.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.749+0000 7ff6a3e00700 1 --2- 172.21.15.53:0/4066354730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff69c105890 0x7ff69c1098e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:20.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.749+0000 7ff6a3e00700 1 -- 172.21.15.53:0/4066354730 >> 172.21.15.53:0/4066354730 conn(0x7ff69c100770 msgr2=0x7ff69c102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:20.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.749+0000 7ff6a3e00700 1 -- 172.21.15.53:0/4066354730 shutdown_connections 2024-04-25T14:39:20.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.749+0000 7ff6a3e00700 1 -- 172.21.15.53:0/4066354730 wait complete. 2024-04-25T14:39:20.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.750+0000 7ff6a3e00700 1 Processor -- start 2024-04-25T14:39:20.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.750+0000 7ff6a3e00700 1 -- start start 2024-04-25T14:39:20.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.751+0000 7ff6a3e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff69c104ee0 0x7ff69c19b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:20.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.751+0000 7ff6a3e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff69c105890 0x7ff69c19b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:20.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.751+0000 7ff6a3e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff69c109fc0 0x7ff69c19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:20.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.751+0000 7ff6a3e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c114e30 con 0x7ff69c109fc0 2024-04-25T14:39:20.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.751+0000 7ff6a3e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c114cb0 con 0x7ff69c104ee0 2024-04-25T14:39:20.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.751+0000 7ff6a3e00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c114fb0 con 0x7ff69c105890 2024-04-25T14:39:20.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.751+0000 7ff6a1e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff69c109fc0 0x7ff69c19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:20.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.751+0000 7ff6a1e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff69c109fc0 0x7ff69c19fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57532/0 (socket says 172.21.15.53:57532) 2024-04-25T14:39:20.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.751+0000 7ff6a1e00700 1 -- 172.21.15.53:0/3407772049 learned_addr learned my addr 172.21.15.53:0/3407772049 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:20.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.751+0000 7ff6a1400700 1 --2- 172.21.15.53:0/3407772049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff69c104ee0 0x7ff69c19b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:20.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.751+0000 7ff6a0a00700 1 --2- 172.21.15.53:0/3407772049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff69c105890 0x7ff69c19b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:20.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.752+0000 7ff6a1e00700 1 -- 172.21.15.53:0/3407772049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff69c105890 msgr2=0x7ff69c19b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:20.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.752+0000 7ff6a1e00700 1 --2- 172.21.15.53:0/3407772049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff69c105890 0x7ff69c19b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:20.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.752+0000 7ff6a1e00700 1 -- 172.21.15.53:0/3407772049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff69c104ee0 msgr2=0x7ff69c19b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:20.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.752+0000 7ff6a1e00700 1 --2- 172.21.15.53:0/3407772049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff69c104ee0 0x7ff69c19b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:20.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.752+0000 7ff6a1e00700 1 -- 172.21.15.53:0/3407772049 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff69c1a03d0 con 0x7ff69c109fc0 2024-04-25T14:39:20.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.752+0000 7ff6a1e00700 1 --2- 172.21.15.53:0/3407772049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff69c109fc0 0x7ff69c19fc90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff69400c110 tx=0x7ff69400c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:20.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.752+0000 7ff69a000700 1 -- 172.21.15.53:0/3407772049 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6940092e0 con 0x7ff69c109fc0 2024-04-25T14:39:20.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.752+0000 7ff69a000700 1 -- 172.21.15.53:0/3407772049 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff694004bb0 con 0x7ff69c109fc0 2024-04-25T14:39:20.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.752+0000 7ff69a000700 1 -- 172.21.15.53:0/3407772049 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff694007be0 con 0x7ff69c109fc0 2024-04-25T14:39:20.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.752+0000 7ff6a3e00700 1 -- 172.21.15.53:0/3407772049 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff69c1a0660 con 0x7ff69c109fc0 2024-04-25T14:39:20.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.752+0000 7ff6a3e00700 1 -- 172.21.15.53:0/3407772049 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff69c1a7f40 con 0x7ff69c109fc0 2024-04-25T14:39:20.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.754+0000 7ff6a3e00700 1 -- 172.21.15.53:0/3407772049 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff69c1063c0 con 0x7ff69c109fc0 2024-04-25T14:39:20.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.759+0000 7ff69a000700 1 -- 172.21.15.53:0/3407772049 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff69401a020 con 0x7ff69c109fc0 2024-04-25T14:39:20.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.760+0000 7ff69a000700 1 --2- 172.21.15.53:0/3407772049 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff6840a3d50 0x7ff6840a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:20.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.760+0000 7ff69a000700 1 -- 172.21.15.53:0/3407772049 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7ff6940d0a10 con 0x7ff69c109fc0 2024-04-25T14:39:20.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.760+0000 7ff6a1400700 1 --2- 172.21.15.53:0/3407772049 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff6840a3d50 0x7ff6840a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:20.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.761+0000 7ff6a1400700 1 --2- 172.21.15.53:0/3407772049 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff6840a3d50 0x7ff6840a6210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff690009c60 tx=0x7ff690009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:20.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:20.762+0000 7ff69a000700 1 -- 172.21.15.53:0/3407772049 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff694010040 con 0x7ff69c109fc0 2024-04-25T14:39:20.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:16.924+0000 7f5fb8400700 1 -- 172.21.15.53:0/222008904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fb0108a50 msgr2=0x7f5fb0108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:20.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:16.924+0000 7f5fb8400700 1 --2- 172.21.15.53:0/222008904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fb0108a50 0x7f5fb0108e30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4009990 tx=0x7f5fa4009ca0 comp rx=0 tx=0).stop 2024-04-25T14:39:20.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:16.924+0000 7f5fb8400700 1 -- 172.21.15.53:0/222008904 shutdown_connections 2024-04-25T14:39:20.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:16.924+0000 7f5fb8400700 1 --2- 172.21.15.53:0/222008904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5fb006bab0 0x7f5fb010f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:20.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:16.924+0000 7f5fb8400700 1 --2- 172.21.15.53:0/222008904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fb0108a50 0x7f5fb0108e30 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:20.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:16.924+0000 7f5fb8400700 1 --2- 172.21.15.53:0/222008904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fb006b110 0x7f5fb006b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:20.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:16.924+0000 7f5fb8400700 1 -- 172.21.15.53:0/222008904 >> 172.21.15.53:0/222008904 conn(0x7f5fb00788f0 msgr2=0x7f5fb0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:20.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:16.924+0000 7f5fb8400700 1 -- 172.21.15.53:0/222008904 shutdown_connections 2024-04-25T14:39:20.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:16.924+0000 7f5fb8400700 1 -- 172.21.15.53:0/222008904 wait complete. 2024-04-25T14:39:20.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:16.925+0000 7f5fb8400700 1 Processor -- start 2024-04-25T14:39:20.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:16.926+0000 7f5fb8400700 1 -- start start 2024-04-25T14:39:20.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:16.926+0000 7f5fb8400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5fb006b110 0x7f5fb019b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:20.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:16.926+0000 7f5fb8400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fb006bab0 0x7f5fb019b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:20.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:16.926+0000 7f5fb8400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fb0108a50 0x7f5fb019fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:20.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:16.926+0000 7f5fb8400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb0112c50 con 0x7f5fb0108a50 2024-04-25T14:39:20.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:16.926+0000 7f5fb8400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb0112ad0 con 0x7f5fb006b110 2024-04-25T14:39:20.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:16.926+0000 7f5fb8400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb0112dd0 con 0x7f5fb006bab0 2024-04-25T14:39:20.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:16.926+0000 7f5fb6400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fb0108a50 0x7f5fb019fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:20.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:16.926+0000 7f5fb6400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fb0108a50 0x7f5fb019fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57466/0 (socket says 172.21.15.53:57466) 2024-04-25T14:39:20.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:16.926+0000 7f5fb5a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5fb006b110 0x7f5fb019b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:20.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:16.926+0000 7f5fb6400700 1 -- 172.21.15.53:0/3991338222 learned_addr learned my addr 172.21.15.53:0/3991338222 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:20.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:16.926+0000 7f5fb5000700 1 --2- 172.21.15.53:0/3991338222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fb006bab0 0x7f5fb019b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:20.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:16.927+0000 7f5fb6400700 1 -- 172.21.15.53:0/3991338222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fb006bab0 msgr2=0x7f5fb019b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:20.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:16.927+0000 7f5fb6400700 1 --2- 172.21.15.53:0/3991338222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fb006bab0 0x7f5fb019b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:20.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:16.927+0000 7f5fb6400700 1 -- 172.21.15.53:0/3991338222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5fb006b110 msgr2=0x7f5fb019b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:20.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:16.927+0000 7f5fb6400700 1 --2- 172.21.15.53:0/3991338222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5fb006b110 0x7f5fb019b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:20.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:16.927+0000 7f5fb6400700 1 -- 172.21.15.53:0/3991338222 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb01a0420 con 0x7f5fb0108a50 2024-04-25T14:39:20.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:16.927+0000 7f5fb5a00700 1 --2- 172.21.15.53:0/3991338222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5fb006b110 0x7f5fb019b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:39:20.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:16.927+0000 7f5fb6400700 1 --2- 172.21.15.53:0/3991338222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fb0108a50 0x7f5fb019fce0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5fa800ef30 tx=0x7f5fa800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:20.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:16.927+0000 7f5fae000700 1 -- 172.21.15.53:0/3991338222 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa80092e0 con 0x7f5fb0108a50 2024-04-25T14:39:20.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:16.927+0000 7f5fae000700 1 -- 172.21.15.53:0/3991338222 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fa8004bb0 con 0x7f5fb0108a50 2024-04-25T14:39:20.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:16.927+0000 7f5fb8400700 1 -- 172.21.15.53:0/3991338222 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fb01a06b0 con 0x7f5fb0108a50 2024-04-25T14:39:20.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:16.927+0000 7f5fae000700 1 -- 172.21.15.53:0/3991338222 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8004020 con 0x7f5fb0108a50 2024-04-25T14:39:20.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:16.928+0000 7f5fb8400700 1 -- 172.21.15.53:0/3991338222 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fb01a7f90 con 0x7f5fb0108a50 2024-04-25T14:39:20.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:16.929+0000 7f5fb8400700 1 -- 172.21.15.53:0/3991338222 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fb006a9c0 con 0x7f5fb0108a50 2024-04-25T14:39:20.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:16.931+0000 7f5fae000700 1 -- 172.21.15.53:0/3991338222 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5fa801b030 con 0x7f5fb0108a50 2024-04-25T14:39:20.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:16.932+0000 7f5fae000700 1 --2- 172.21.15.53:0/3991338222 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5f980a3cd0 0x7f5f980a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:20.961 INFO:teuthology.orchestra.run.smithi053.stdout:34359738455 2024-04-25T14:39:20.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:16.932+0000 7f5fae000700 1 -- 172.21.15.53:0/3991338222 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f5fa80cffe0 con 0x7f5fb0108a50 2024-04-25T14:39:20.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:16.932+0000 7f5fb5a00700 1 --2- 172.21.15.53:0/3991338222 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5f980a3cd0 0x7f5f980a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:20.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:16.933+0000 7f5fb5a00700 1 --2- 172.21.15.53:0/3991338222 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5f980a3cd0 0x7f5f980a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5fa400b2e0 tx=0x7f5fa401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:20.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:16.937+0000 7f5fae000700 1 -- 172.21.15.53:0/3991338222 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5fa8022020 con 0x7f5fb0108a50 2024-04-25T14:39:20.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:17.172+0000 7f5fb8400700 1 -- 172.21.15.53:0/3991338222 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f5fb019c770 con 0x7f5fb0108a50 2024-04-25T14:39:20.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:17.173+0000 7f5fae000700 1 -- 172.21.15.53:0/3991338222 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5fa8010040 con 0x7f5fb0108a50 2024-04-25T14:39:20.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:17.176+0000 7f5fb8400700 1 -- 172.21.15.53:0/3991338222 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5f980a3cd0 msgr2=0x7f5f980a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:20.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:17.176+0000 7f5fb8400700 1 --2- 172.21.15.53:0/3991338222 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5f980a3cd0 0x7f5f980a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5fa400b2e0 tx=0x7f5fa401a040 comp rx=0 tx=0).stop 2024-04-25T14:39:20.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:17.176+0000 7f5fb8400700 1 -- 172.21.15.53:0/3991338222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fb0108a50 msgr2=0x7f5fb019fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:20.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:17.176+0000 7f5fb8400700 1 --2- 172.21.15.53:0/3991338222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fb0108a50 0x7f5fb019fce0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5fa800ef30 tx=0x7f5fa800c3a0 comp rx=0 tx=0).stop 2024-04-25T14:39:20.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:17.176+0000 7f5fb8400700 1 -- 172.21.15.53:0/3991338222 shutdown_connections 2024-04-25T14:39:20.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:17.176+0000 7f5fb8400700 1 --2- 172.21.15.53:0/3991338222 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5f980a3cd0 0x7f5f980a6190 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:20.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:17.176+0000 7f5fb8400700 1 --2- 172.21.15.53:0/3991338222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fb0108a50 0x7f5fb019fce0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:20.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:17.176+0000 7f5fb8400700 1 --2- 172.21.15.53:0/3991338222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5fb006b110 0x7f5fb019b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:20.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:17.176+0000 7f5fb8400700 1 --2- 172.21.15.53:0/3991338222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fb006bab0 0x7f5fb019b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:20.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:17.176+0000 7f5fb8400700 1 -- 172.21.15.53:0/3991338222 >> 172.21.15.53:0/3991338222 conn(0x7f5fb00788f0 msgr2=0x7f5fb010deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:20.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:17.177+0000 7f5fb8400700 1 -- 172.21.15.53:0/3991338222 shutdown_connections 2024-04-25T14:39:20.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:17.177+0000 7f5fb8400700 1 -- 172.21.15.53:0/3991338222 wait complete. 2024-04-25T14:39:21.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.000+0000 7ff6a3e00700 1 -- 172.21.15.53:0/3407772049 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7ff69c02df80 con 0x7ff69c109fc0 2024-04-25T14:39:21.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.006+0000 7ff69a000700 1 -- 172.21.15.53:0/3407772049 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff69409d220 con 0x7ff69c109fc0 2024-04-25T14:39:21.006 INFO:teuthology.orchestra.run.smithi053.stdout:55834574930 2024-04-25T14:39:21.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.009+0000 7ff6a3e00700 1 -- 172.21.15.53:0/3407772049 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff6840a3d50 msgr2=0x7ff6840a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:21.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.009+0000 7ff6a3e00700 1 --2- 172.21.15.53:0/3407772049 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff6840a3d50 0x7ff6840a6210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff690009c60 tx=0x7ff690009260 comp rx=0 tx=0).stop 2024-04-25T14:39:21.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.009+0000 7ff6a3e00700 1 -- 172.21.15.53:0/3407772049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff69c109fc0 msgr2=0x7ff69c19fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:21.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.009+0000 7ff6a3e00700 1 --2- 172.21.15.53:0/3407772049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff69c109fc0 0x7ff69c19fc90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff69400c110 tx=0x7ff69400c420 comp rx=0 tx=0).stop 2024-04-25T14:39:21.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.009+0000 7ff6a3e00700 1 -- 172.21.15.53:0/3407772049 shutdown_connections 2024-04-25T14:39:21.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.009+0000 7ff6a3e00700 1 --2- 172.21.15.53:0/3407772049 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff6840a3d50 0x7ff6840a6210 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:21.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.009+0000 7ff6a3e00700 1 --2- 172.21.15.53:0/3407772049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff69c109fc0 0x7ff69c19fc90 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:21.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.009+0000 7ff6a3e00700 1 --2- 172.21.15.53:0/3407772049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff69c104ee0 0x7ff69c19b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:21.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.009+0000 7ff6a3e00700 1 --2- 172.21.15.53:0/3407772049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff69c105890 0x7ff69c19b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:21.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.009+0000 7ff6a3e00700 1 -- 172.21.15.53:0/3407772049 >> 172.21.15.53:0/3407772049 conn(0x7ff69c100770 msgr2=0x7ff69c101b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:21.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.010+0000 7ff6a3e00700 1 -- 172.21.15.53:0/3407772049 shutdown_connections 2024-04-25T14:39:21.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.010+0000 7ff6a3e00700 1 -- 172.21.15.53:0/3407772049 wait complete. 2024-04-25T14:39:21.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:21 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/248225752' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T14:39:21.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:21 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/248225752' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T14:39:21.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:21 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/248225752' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T14:39:21.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.711+0000 7f99f5400700 1 -- 172.21.15.53:0/442190095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99f0077890 msgr2=0x7f99f0113380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:21.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.711+0000 7f99f5400700 1 --2- 172.21.15.53:0/442190095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99f0077890 0x7f99f0113380 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f99dc009990 tx=0x7f99dc009ca0 comp rx=0 tx=0).stop 2024-04-25T14:39:21.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.711+0000 7f99f5400700 1 -- 172.21.15.53:0/442190095 shutdown_connections 2024-04-25T14:39:21.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.711+0000 7f99f5400700 1 --2- 172.21.15.53:0/442190095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99f01138c0 0x7f99f0115cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:21.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.711+0000 7f99f5400700 1 --2- 172.21.15.53:0/442190095 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99f0076f70 0x7f99f0077350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:21.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.711+0000 7f99f5400700 1 --2- 172.21.15.53:0/442190095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99f0077890 0x7f99f0113380 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:21.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.711+0000 7f99f5400700 1 -- 172.21.15.53:0/442190095 >> 172.21.15.53:0/442190095 conn(0x7f99f01006f0 msgr2=0x7f99f0102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:21.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.711+0000 7f99f5400700 1 -- 172.21.15.53:0/442190095 shutdown_connections 2024-04-25T14:39:21.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.712+0000 7f99f5400700 1 -- 172.21.15.53:0/442190095 wait complete. 2024-04-25T14:39:21.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.712+0000 7f99f5400700 1 Processor -- start 2024-04-25T14:39:21.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.713+0000 7f99f5400700 1 -- start start 2024-04-25T14:39:21.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.713+0000 7f99f5400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99f0076f70 0x7f99f01a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:21.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.713+0000 7f99f5400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99f0077890 0x7f99f01a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:21.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.713+0000 7f99f5400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99f01138c0 0x7f99f019f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:21.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.713+0000 7f99f5400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f99f01192b0 con 0x7f99f01138c0 2024-04-25T14:39:21.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.713+0000 7f99f5400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f99f0119130 con 0x7f99f0077890 2024-04-25T14:39:21.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.713+0000 7f99f5400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f99f0119430 con 0x7f99f0076f70 2024-04-25T14:39:21.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.713+0000 7f99ef200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99f01138c0 0x7f99f019f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:21.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.713+0000 7f99ef200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99f01138c0 0x7f99f019f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57540/0 (socket says 172.21.15.53:57540) 2024-04-25T14:39:21.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.713+0000 7f99ef200700 1 -- 172.21.15.53:0/3110247887 learned_addr learned my addr 172.21.15.53:0/3110247887 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:21.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.713+0000 7f99ee800700 1 --2- 172.21.15.53:0/3110247887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99f0076f70 0x7f99f01a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:21.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.713+0000 7f99ede00700 1 --2- 172.21.15.53:0/3110247887 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99f0077890 0x7f99f01a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:21.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.714+0000 7f99ef200700 1 -- 172.21.15.53:0/3110247887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99f0076f70 msgr2=0x7f99f01a5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:21.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.714+0000 7f99ef200700 1 --2- 172.21.15.53:0/3110247887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99f0076f70 0x7f99f01a5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:21.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.714+0000 7f99ef200700 1 -- 172.21.15.53:0/3110247887 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99f0077890 msgr2=0x7f99f01a5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:21.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.714+0000 7f99ef200700 1 --2- 172.21.15.53:0/3110247887 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99f0077890 0x7f99f01a5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:21.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.714+0000 7f99ef200700 1 -- 172.21.15.53:0/3110247887 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f019ff10 con 0x7f99f01138c0 2024-04-25T14:39:21.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.714+0000 7f99ee800700 1 --2- 172.21.15.53:0/3110247887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99f0076f70 0x7f99f01a5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:39:21.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.714+0000 7f99ef200700 1 --2- 172.21.15.53:0/3110247887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99f01138c0 0x7f99f019f630 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f99e8009fd0 tx=0x7f99e800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:21.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.715+0000 7f99e7400700 1 -- 172.21.15.53:0/3110247887 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e80092e0 con 0x7f99f01138c0 2024-04-25T14:39:21.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.715+0000 7f99e7400700 1 -- 172.21.15.53:0/3110247887 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99e8004bb0 con 0x7f99f01138c0 2024-04-25T14:39:21.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.715+0000 7f99f5400700 1 -- 172.21.15.53:0/3110247887 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99f01a01a0 con 0x7f99f01138c0 2024-04-25T14:39:21.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.715+0000 7f99e7400700 1 -- 172.21.15.53:0/3110247887 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e8004020 con 0x7f99f01138c0 2024-04-25T14:39:21.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.715+0000 7f99f5400700 1 -- 172.21.15.53:0/3110247887 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99f01ac310 con 0x7f99f01138c0 2024-04-25T14:39:21.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.716+0000 7f99f5400700 1 -- 172.21.15.53:0/3110247887 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99f00786b0 con 0x7f99f01138c0 2024-04-25T14:39:21.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.719+0000 7f99e7400700 1 -- 172.21.15.53:0/3110247887 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f99e801b030 con 0x7f99f01138c0 2024-04-25T14:39:21.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.721+0000 7f99e7400700 1 --2- 172.21.15.53:0/3110247887 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f99d40a3cd0 0x7f99d40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:21.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.721+0000 7f99e7400700 1 -- 172.21.15.53:0/3110247887 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f99e80d03c0 con 0x7f99f01138c0 2024-04-25T14:39:21.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.721+0000 7f99ee800700 1 --2- 172.21.15.53:0/3110247887 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f99d40a3cd0 0x7f99d40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:21.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.722+0000 7f99ee800700 1 --2- 172.21.15.53:0/3110247887 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f99d40a3cd0 0x7f99d40a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f99e0011490 tx=0x7f99e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:21.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.724+0000 7f99e7400700 1 -- 172.21.15.53:0/3110247887 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f99e8010040 con 0x7f99f01138c0 2024-04-25T14:39:21.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.978+0000 7f99f5400700 1 -- 172.21.15.53:0/3110247887 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f99f002df80 con 0x7f99f01138c0 2024-04-25T14:39:21.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.979+0000 7f99e7400700 1 -- 172.21.15.53:0/3110247887 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f99e801b320 con 0x7f99f01138c0 2024-04-25T14:39:21.979 INFO:teuthology.orchestra.run.smithi053.stdout:146028888133 2024-04-25T14:39:21.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.982+0000 7f99f5400700 1 -- 172.21.15.53:0/3110247887 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f99d40a3cd0 msgr2=0x7f99d40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:21.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.982+0000 7f99f5400700 1 --2- 172.21.15.53:0/3110247887 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f99d40a3cd0 0x7f99d40a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f99e0011490 tx=0x7f99e0009260 comp rx=0 tx=0).stop 2024-04-25T14:39:21.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.982+0000 7f99f5400700 1 -- 172.21.15.53:0/3110247887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99f01138c0 msgr2=0x7f99f019f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:21.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.982+0000 7f99f5400700 1 --2- 172.21.15.53:0/3110247887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99f01138c0 0x7f99f019f630 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f99e8009fd0 tx=0x7f99e800ef90 comp rx=0 tx=0).stop 2024-04-25T14:39:21.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.982+0000 7f99f5400700 1 -- 172.21.15.53:0/3110247887 shutdown_connections 2024-04-25T14:39:21.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.982+0000 7f99f5400700 1 --2- 172.21.15.53:0/3110247887 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f99d40a3cd0 0x7f99d40a6190 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:21.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.982+0000 7f99f5400700 1 --2- 172.21.15.53:0/3110247887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99f01138c0 0x7f99f019f630 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:21.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.982+0000 7f99f5400700 1 --2- 172.21.15.53:0/3110247887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99f0076f70 0x7f99f01a5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:21.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.982+0000 7f99f5400700 1 --2- 172.21.15.53:0/3110247887 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99f0077890 0x7f99f01a5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:21.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.982+0000 7f99f5400700 1 -- 172.21.15.53:0/3110247887 >> 172.21.15.53:0/3110247887 conn(0x7f99f01006f0 msgr2=0x7f99f0102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:21.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.982+0000 7f99f5400700 1 -- 172.21.15.53:0/3110247887 shutdown_connections 2024-04-25T14:39:21.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:21.982+0000 7f99f5400700 1 -- 172.21.15.53:0/3110247887 wait complete. 2024-04-25T14:39:22.277 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:22 smithi053 ceph-mon[27773]: pgmap v353: 97 pgs: 1 peering, 96 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s; 129 B/s, 0 objects/s recovering 2024-04-25T14:39:22.277 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:22 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/3407772049' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T14:39:22.278 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:22 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/3110247887' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T14:39:22.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.302+0000 7f67dfa00700 1 -- 172.21.15.53:0/2121449627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67d81138d0 msgr2=0x7f67d8115cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:22.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.302+0000 7f67dfa00700 1 --2- 172.21.15.53:0/2121449627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67d81138d0 0x7f67d8115cc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f67d000b0a0 tx=0x7f67d000b3b0 comp rx=0 tx=0).stop 2024-04-25T14:39:22.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.303+0000 7f67dfa00700 1 -- 172.21.15.53:0/2121449627 shutdown_connections 2024-04-25T14:39:22.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.303+0000 7f67dfa00700 1 --2- 172.21.15.53:0/2121449627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67d81138d0 0x7f67d8115cc0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:22.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.303+0000 7f67dfa00700 1 --2- 172.21.15.53:0/2121449627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f67d8076f70 0x7f67d8077350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:22.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.303+0000 7f67dfa00700 1 --2- 172.21.15.53:0/2121449627 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67d8077890 0x7f67d8113390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:22.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.303+0000 7f67dfa00700 1 -- 172.21.15.53:0/2121449627 >> 172.21.15.53:0/2121449627 conn(0x7f67d8100720 msgr2=0x7f67d8102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:22.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.303+0000 7f67dfa00700 1 -- 172.21.15.53:0/2121449627 shutdown_connections 2024-04-25T14:39:22.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.303+0000 7f67dfa00700 1 -- 172.21.15.53:0/2121449627 wait complete. 2024-04-25T14:39:22.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.304+0000 7f67dfa00700 1 Processor -- start 2024-04-25T14:39:22.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.304+0000 7f67dfa00700 1 -- start start 2024-04-25T14:39:22.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.305+0000 7f67dfa00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67d8076f70 0x7f67d819f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:22.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.305+0000 7f67dfa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67d8077890 0x7f67d819fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:22.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.305+0000 7f67dfa00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f67d81138d0 0x7f67d81a3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:22.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.305+0000 7f67dfa00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f67d8119250 con 0x7f67d8077890 2024-04-25T14:39:22.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.305+0000 7f67dfa00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f67d81190d0 con 0x7f67d8076f70 2024-04-25T14:39:22.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.305+0000 7f67dfa00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f67d81193d0 con 0x7f67d81138d0 2024-04-25T14:39:22.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.305+0000 7f67dd000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67d8076f70 0x7f67d819f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:22.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.305+0000 7f67d7e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67d8077890 0x7f67d819fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:22.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.305+0000 7f67dd000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67d8076f70 0x7f67d819f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.53:57740/0 (socket says 172.21.15.53:57740) 2024-04-25T14:39:22.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.305+0000 7f67dd000700 1 -- 172.21.15.53:0/1346741801 learned_addr learned my addr 172.21.15.53:0/1346741801 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:22.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.305+0000 7f67dda00700 1 --2- 172.21.15.53:0/1346741801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f67d81138d0 0x7f67d81a3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:22.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.305+0000 7f67d7e00700 1 -- 172.21.15.53:0/1346741801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f67d81138d0 msgr2=0x7f67d81a3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:22.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.305+0000 7f67d7e00700 1 --2- 172.21.15.53:0/1346741801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f67d81138d0 0x7f67d81a3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:22.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.305+0000 7f67d7e00700 1 -- 172.21.15.53:0/1346741801 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67d8076f70 msgr2=0x7f67d819f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:22.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.305+0000 7f67d7e00700 1 --2- 172.21.15.53:0/1346741801 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67d8076f70 0x7f67d819f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:22.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.305+0000 7f67d7e00700 1 -- 172.21.15.53:0/1346741801 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67d81a4700 con 0x7f67d8077890 2024-04-25T14:39:22.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.306+0000 7f67d7e00700 1 --2- 172.21.15.53:0/1346741801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67d8077890 0x7f67d819fc80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f67c400e9f0 tx=0x7f67c400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:22.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.306+0000 7f67d5600700 1 -- 172.21.15.53:0/1346741801 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c400c9c0 con 0x7f67d8077890 2024-04-25T14:39:22.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.306+0000 7f67d5600700 1 -- 172.21.15.53:0/1346741801 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67c400cb60 con 0x7f67d8077890 2024-04-25T14:39:22.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.306+0000 7f67d5600700 1 -- 172.21.15.53:0/1346741801 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c4018680 con 0x7f67d8077890 2024-04-25T14:39:22.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.306+0000 7f67dfa00700 1 -- 172.21.15.53:0/1346741801 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67d81a49f0 con 0x7f67d8077890 2024-04-25T14:39:22.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.306+0000 7f67dfa00700 1 -- 172.21.15.53:0/1346741801 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67d81ac270 con 0x7f67d8077890 2024-04-25T14:39:22.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.307+0000 7f67dfa00700 1 -- 172.21.15.53:0/1346741801 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67d8069730 con 0x7f67d8077890 2024-04-25T14:39:22.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.309+0000 7f67d5600700 1 -- 172.21.15.53:0/1346741801 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f67c4010430 con 0x7f67d8077890 2024-04-25T14:39:22.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.311+0000 7f67d5600700 1 --2- 172.21.15.53:0/1346741801 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f67c00a3d50 0x7f67c00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:22.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.311+0000 7f67d5600700 1 -- 172.21.15.53:0/1346741801 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f67c40d0630 con 0x7f67d8077890 2024-04-25T14:39:22.430 INFO:teuthology.orchestra.run.smithi053.stdout:206158430262 2024-04-25T14:39:22.430 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.311+0000 7f67dd000700 1 --2- 172.21.15.53:0/1346741801 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f67c00a3d50 0x7f67c00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:22.430 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.312+0000 7f67dd000700 1 --2- 172.21.15.53:0/1346741801 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f67c00a3d50 0x7f67c00a6210 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f67cc009d20 tx=0x7f67cc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:22.430 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.315+0000 7f67d5600700 1 -- 172.21.15.53:0/1346741801 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f67c409ce40 con 0x7f67d8077890 2024-04-25T14:39:22.430 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.557+0000 7f67dfa00700 1 -- 172.21.15.53:0/1346741801 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f67d802df80 con 0x7f67d8077890 2024-04-25T14:39:22.430 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.558+0000 7f67d5600700 1 -- 172.21.15.53:0/1346741801 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f67c409c740 con 0x7f67d8077890 2024-04-25T14:39:22.431 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.561+0000 7f67dfa00700 1 -- 172.21.15.53:0/1346741801 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f67c00a3d50 msgr2=0x7f67c00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:22.431 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.561+0000 7f67dfa00700 1 --2- 172.21.15.53:0/1346741801 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f67c00a3d50 0x7f67c00a6210 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f67cc009d20 tx=0x7f67cc009260 comp rx=0 tx=0).stop 2024-04-25T14:39:22.431 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.561+0000 7f67dfa00700 1 -- 172.21.15.53:0/1346741801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67d8077890 msgr2=0x7f67d819fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:22.431 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.561+0000 7f67dfa00700 1 --2- 172.21.15.53:0/1346741801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67d8077890 0x7f67d819fc80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f67c400e9f0 tx=0x7f67c400ed00 comp rx=0 tx=0).stop 2024-04-25T14:39:22.431 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.561+0000 7f67dfa00700 1 -- 172.21.15.53:0/1346741801 shutdown_connections 2024-04-25T14:39:22.431 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.561+0000 7f67dfa00700 1 --2- 172.21.15.53:0/1346741801 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f67c00a3d50 0x7f67c00a6210 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:22.431 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.561+0000 7f67dfa00700 1 --2- 172.21.15.53:0/1346741801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f67d81138d0 0x7f67d81a3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:22.431 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.561+0000 7f67dfa00700 1 --2- 172.21.15.53:0/1346741801 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67d8076f70 0x7f67d819f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:22.431 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.561+0000 7f67dfa00700 1 --2- 172.21.15.53:0/1346741801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67d8077890 0x7f67d819fc80 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:22.431 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.561+0000 7f67dfa00700 1 -- 172.21.15.53:0/1346741801 >> 172.21.15.53:0/1346741801 conn(0x7f67d8100720 msgr2=0x7f67d8102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:22.431 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.561+0000 7f67dfa00700 1 -- 172.21.15.53:0/1346741801 shutdown_connections 2024-04-25T14:39:22.431 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:19.561+0000 7f67dfa00700 1 -- 172.21.15.53:0/1346741801 wait complete. 2024-04-25T14:39:22.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:22 smithi183 ceph-mon[32584]: pgmap v353: 97 pgs: 1 peering, 96 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s; 129 B/s, 0 objects/s recovering 2024-04-25T14:39:22.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:22 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/3407772049' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T14:39:22.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:22 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/3110247887' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T14:39:22.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:22 smithi150 ceph-mon[32504]: pgmap v353: 97 pgs: 1 peering, 96 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s; 129 B/s, 0 objects/s recovering 2024-04-25T14:39:22.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:22 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/3407772049' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T14:39:22.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:22 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/3110247887' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T14:39:23.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:23 smithi053 ceph-mon[27773]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-25T14:39:23.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:23 smithi053 ceph-mon[27773]: Cluster is now healthy 2024-04-25T14:39:23.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:23 smithi183 ceph-mon[32584]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-25T14:39:23.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:23 smithi183 ceph-mon[32584]: Cluster is now healthy 2024-04-25T14:39:23.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:23 smithi150 ceph-mon[32504]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-25T14:39:23.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:23 smithi150 ceph-mon[32504]: Cluster is now healthy 2024-04-25T14:39:24.074 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430256 got 206158430262 for osd.6 2024-04-25T14:39:24.074 DEBUG:teuthology.parallel:result is None 2024-04-25T14:39:24.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:24 smithi053 ceph-mon[27773]: pgmap v354: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s; 129 B/s, 0 objects/s recovering 2024-04-25T14:39:24.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:24 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/2255376639' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T14:39:24.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:24 smithi183 ceph-mon[32584]: pgmap v354: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s; 129 B/s, 0 objects/s recovering 2024-04-25T14:39:24.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:24 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/2255376639' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T14:39:24.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:24 smithi150 ceph-mon[32504]: pgmap v354: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s; 129 B/s, 0 objects/s recovering 2024-04-25T14:39:24.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:24 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/2255376639' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T14:39:24.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.678+0000 7f25b2800700 1 -- 172.21.15.53:0/594486275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25ac075590 msgr2=0x7f25ac10ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:24.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.678+0000 7f25b2800700 1 --2- 172.21.15.53:0/594486275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25ac075590 0x7f25ac10ef00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2598009990 tx=0x7f2598009ca0 comp rx=0 tx=0).stop 2024-04-25T14:39:24.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.678+0000 7f25b2800700 1 -- 172.21.15.53:0/594486275 shutdown_connections 2024-04-25T14:39:24.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.678+0000 7f25b2800700 1 --2- 172.21.15.53:0/594486275 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25ac10f440 0x7f25ac111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:24.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.678+0000 7f25b2800700 1 --2- 172.21.15.53:0/594486275 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f25ac074c70 0x7f25ac075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:24.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.678+0000 7f25b2800700 1 --2- 172.21.15.53:0/594486275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25ac075590 0x7f25ac10ef00 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:24.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.678+0000 7f25b2800700 1 -- 172.21.15.53:0/594486275 >> 172.21.15.53:0/594486275 conn(0x7f25ac079ba0 msgr2=0x7f25ac079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:24.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.678+0000 7f25b2800700 1 -- 172.21.15.53:0/594486275 shutdown_connections 2024-04-25T14:39:24.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.678+0000 7f25b2800700 1 -- 172.21.15.53:0/594486275 wait complete. 2024-04-25T14:39:24.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.679+0000 7f25b2800700 1 Processor -- start 2024-04-25T14:39:24.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.679+0000 7f25b2800700 1 -- start start 2024-04-25T14:39:24.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.679+0000 7f25b2800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25ac074c70 0x7f25ac19b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:24.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.679+0000 7f25b2800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25ac075590 0x7f25ac19b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:24.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.680+0000 7f25b2800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f25ac10f440 0x7f25ac19fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:24.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.680+0000 7f25b2800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f25ac114e50 con 0x7f25ac10f440 2024-04-25T14:39:24.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.680+0000 7f25b2800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f25ac114cd0 con 0x7f25ac075590 2024-04-25T14:39:24.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.680+0000 7f25b2800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f25ac114fd0 con 0x7f25ac074c70 2024-04-25T14:39:24.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.680+0000 7f25ab400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25ac074c70 0x7f25ac19b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:24.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.680+0000 7f25ab400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25ac074c70 0x7f25ac19b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:40750/0 (socket says 172.21.15.53:40750) 2024-04-25T14:39:24.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.680+0000 7f25ab400700 1 -- 172.21.15.53:0/2255376639 learned_addr learned my addr 172.21.15.53:0/2255376639 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:24.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.680+0000 7f25abe00700 1 --2- 172.21.15.53:0/2255376639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f25ac10f440 0x7f25ac19fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:24.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.680+0000 7f25aaa00700 1 --2- 172.21.15.53:0/2255376639 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25ac075590 0x7f25ac19b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:24.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.680+0000 7f25ab400700 1 -- 172.21.15.53:0/2255376639 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25ac075590 msgr2=0x7f25ac19b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:24.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.680+0000 7f25ab400700 1 --2- 172.21.15.53:0/2255376639 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25ac075590 0x7f25ac19b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:24.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.680+0000 7f25ab400700 1 -- 172.21.15.53:0/2255376639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f25ac10f440 msgr2=0x7f25ac19fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:24.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.680+0000 7f25ab400700 1 --2- 172.21.15.53:0/2255376639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f25ac10f440 0x7f25ac19fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:24.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.680+0000 7f25ab400700 1 -- 172.21.15.53:0/2255376639 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25ac1a0340 con 0x7f25ac074c70 2024-04-25T14:39:24.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.681+0000 7f25ab400700 1 --2- 172.21.15.53:0/2255376639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25ac074c70 0x7f25ac19b380 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f259c00eb30 tx=0x7f259c00ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:24.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.681+0000 7f25a3e00700 1 -- 172.21.15.53:0/2255376639 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f259c00cbd0 con 0x7f25ac074c70 2024-04-25T14:39:24.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.681+0000 7f25a3e00700 1 -- 172.21.15.53:0/2255376639 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f259c0044d0 con 0x7f25ac074c70 2024-04-25T14:39:24.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.681+0000 7f25b2800700 1 -- 172.21.15.53:0/2255376639 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25ac1a0630 con 0x7f25ac074c70 2024-04-25T14:39:24.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.681+0000 7f25a3e00700 1 -- 172.21.15.53:0/2255376639 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f259c003df0 con 0x7f25ac074c70 2024-04-25T14:39:24.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.681+0000 7f25b2800700 1 -- 172.21.15.53:0/2255376639 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25ac1a7eb0 con 0x7f25ac074c70 2024-04-25T14:39:24.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.682+0000 7f25b2800700 1 -- 172.21.15.53:0/2255376639 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25ac069730 con 0x7f25ac074c70 2024-04-25T14:39:24.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.684+0000 7f25a3e00700 1 -- 172.21.15.53:0/2255376639 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f259c004000 con 0x7f25ac074c70 2024-04-25T14:39:24.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.686+0000 7f25a3e00700 1 --2- 172.21.15.53:0/2255376639 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f25900a3cd0 0x7f25900a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:24.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.686+0000 7f25a3e00700 1 -- 172.21.15.53:0/2255376639 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f259c0d0850 con 0x7f25ac074c70 2024-04-25T14:39:24.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.686+0000 7f25aaa00700 1 --2- 172.21.15.53:0/2255376639 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f25900a3cd0 0x7f25900a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:24.761 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.687+0000 7f25aaa00700 1 --2- 172.21.15.53:0/2255376639 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f25900a3cd0 0x7f25900a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f25ac1025d0 tx=0x7f259800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:24.761 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.690+0000 7f25a3e00700 1 -- 172.21.15.53:0/2255376639 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f259c0042f0 con 0x7f25ac074c70 2024-04-25T14:39:24.761 INFO:teuthology.orchestra.run.smithi053.stdout:111669149769 2024-04-25T14:39:24.761 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.947+0000 7f25b2800700 1 -- 172.21.15.53:0/2255376639 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f25ac02df80 con 0x7f25ac074c70 2024-04-25T14:39:24.761 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.948+0000 7f25a3e00700 1 -- 172.21.15.53:0/2255376639 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f259c09d060 con 0x7f25ac074c70 2024-04-25T14:39:24.761 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.951+0000 7f25b2800700 1 -- 172.21.15.53:0/2255376639 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f25900a3cd0 msgr2=0x7f25900a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:24.761 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.951+0000 7f25b2800700 1 --2- 172.21.15.53:0/2255376639 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f25900a3cd0 0x7f25900a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f25ac1025d0 tx=0x7f259800b260 comp rx=0 tx=0).stop 2024-04-25T14:39:24.761 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.951+0000 7f25b2800700 1 -- 172.21.15.53:0/2255376639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25ac074c70 msgr2=0x7f25ac19b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:24.761 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.951+0000 7f25b2800700 1 --2- 172.21.15.53:0/2255376639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25ac074c70 0x7f25ac19b380 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f259c00eb30 tx=0x7f259c00ee40 comp rx=0 tx=0).stop 2024-04-25T14:39:24.762 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.951+0000 7f25b2800700 1 -- 172.21.15.53:0/2255376639 shutdown_connections 2024-04-25T14:39:24.762 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.951+0000 7f25b2800700 1 --2- 172.21.15.53:0/2255376639 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f25900a3cd0 0x7f25900a6190 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:24.762 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.951+0000 7f25b2800700 1 --2- 172.21.15.53:0/2255376639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f25ac10f440 0x7f25ac19fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:24.762 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.951+0000 7f25b2800700 1 --2- 172.21.15.53:0/2255376639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25ac074c70 0x7f25ac19b380 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:24.762 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.951+0000 7f25b2800700 1 --2- 172.21.15.53:0/2255376639 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25ac075590 0x7f25ac19b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:24.762 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.951+0000 7f25b2800700 1 -- 172.21.15.53:0/2255376639 >> 172.21.15.53:0/2255376639 conn(0x7f25ac079ba0 msgr2=0x7f25ac0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:24.762 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.951+0000 7f25b2800700 1 -- 172.21.15.53:0/2255376639 shutdown_connections 2024-04-25T14:39:24.762 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:23.951+0000 7f25b2800700 1 -- 172.21.15.53:0/2255376639 wait complete. 2024-04-25T14:39:26.112 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:26 smithi053 ceph-mon[27773]: pgmap v355: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 110 B/s, 0 objects/s recovering 2024-04-25T14:39:26.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:26 smithi183 ceph-mon[32584]: pgmap v355: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 110 B/s, 0 objects/s recovering 2024-04-25T14:39:26.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:26 smithi150 ceph-mon[32504]: pgmap v355: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 110 B/s, 0 objects/s recovering 2024-04-25T14:39:26.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.614+0000 7fa530800700 1 -- 172.21.15.53:0/3683058260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa528105870 msgr2=0x7fa5281098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:26.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.614+0000 7fa530800700 1 --2- 172.21.15.53:0/3683058260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa528105870 0x7fa5281098c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa5180099c0 tx=0x7fa518009cd0 comp rx=0 tx=0).stop 2024-04-25T14:39:26.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.614+0000 7fa530800700 1 -- 172.21.15.53:0/3683058260 shutdown_connections 2024-04-25T14:39:26.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.614+0000 7fa530800700 1 --2- 172.21.15.53:0/3683058260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa528109fa0 0x7fa528111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:26.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.614+0000 7fa530800700 1 --2- 172.21.15.53:0/3683058260 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa528104ec0 0x7fa5281052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:26.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.614+0000 7fa530800700 1 --2- 172.21.15.53:0/3683058260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa528105870 0x7fa5281098c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:26.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.614+0000 7fa530800700 1 -- 172.21.15.53:0/3683058260 >> 172.21.15.53:0/3683058260 conn(0x7fa528100750 msgr2=0x7fa528102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:26.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.614+0000 7fa530800700 1 -- 172.21.15.53:0/3683058260 shutdown_connections 2024-04-25T14:39:26.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.614+0000 7fa530800700 1 -- 172.21.15.53:0/3683058260 wait complete. 2024-04-25T14:39:26.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.616+0000 7fa530800700 1 Processor -- start 2024-04-25T14:39:26.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.616+0000 7fa530800700 1 -- start start 2024-04-25T14:39:26.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.616+0000 7fa530800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa528104ec0 0x7fa52819b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:26.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.616+0000 7fa530800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa528105870 0x7fa52819b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:26.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.616+0000 7fa530800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa528109fa0 0x7fa52819fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:26.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.616+0000 7fa530800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa528114e30 con 0x7fa528104ec0 2024-04-25T14:39:26.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.616+0000 7fa530800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa528114cb0 con 0x7fa528109fa0 2024-04-25T14:39:26.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.616+0000 7fa530800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa528114fb0 con 0x7fa528105870 2024-04-25T14:39:26.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.617+0000 7fa52de00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa528104ec0 0x7fa52819b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:26.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.617+0000 7fa52de00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa528104ec0 0x7fa52819b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38212/0 (socket says 172.21.15.53:38212) 2024-04-25T14:39:26.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.617+0000 7fa52d400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa528105870 0x7fa52819b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:26.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.617+0000 7fa52de00700 1 -- 172.21.15.53:0/1978135677 learned_addr learned my addr 172.21.15.53:0/1978135677 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:26.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.617+0000 7fa52e800700 1 --2- 172.21.15.53:0/1978135677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa528109fa0 0x7fa52819fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:26.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.617+0000 7fa52de00700 1 -- 172.21.15.53:0/1978135677 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa528105870 msgr2=0x7fa52819b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:26.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.617+0000 7fa52de00700 1 --2- 172.21.15.53:0/1978135677 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa528105870 0x7fa52819b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:26.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.617+0000 7fa52de00700 1 -- 172.21.15.53:0/1978135677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa528109fa0 msgr2=0x7fa52819fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:26.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.617+0000 7fa52de00700 1 --2- 172.21.15.53:0/1978135677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa528109fa0 0x7fa52819fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:26.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.617+0000 7fa52de00700 1 -- 172.21.15.53:0/1978135677 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5281a0310 con 0x7fa528104ec0 2024-04-25T14:39:26.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.617+0000 7fa52e800700 1 --2- 172.21.15.53:0/1978135677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa528109fa0 0x7fa52819fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:39:26.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.618+0000 7fa52de00700 1 --2- 172.21.15.53:0/1978135677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa528104ec0 0x7fa52819b300 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa51c00e9a0 tx=0x7fa51c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:26.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.618+0000 7fa522a00700 1 -- 172.21.15.53:0/1978135677 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c00ca10 con 0x7fa528104ec0 2024-04-25T14:39:26.619 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.618+0000 7fa522a00700 1 -- 172.21.15.53:0/1978135677 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa51c00cbb0 con 0x7fa528104ec0 2024-04-25T14:39:26.619 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.618+0000 7fa530800700 1 -- 172.21.15.53:0/1978135677 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5281a0600 con 0x7fa528104ec0 2024-04-25T14:39:26.619 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.618+0000 7fa522a00700 1 -- 172.21.15.53:0/1978135677 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c018680 con 0x7fa528104ec0 2024-04-25T14:39:26.619 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.618+0000 7fa530800700 1 -- 172.21.15.53:0/1978135677 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5281a7e80 con 0x7fa528104ec0 2024-04-25T14:39:26.619 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.619+0000 7fa530800700 1 -- 172.21.15.53:0/1978135677 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa528069730 con 0x7fa528104ec0 2024-04-25T14:39:26.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.621+0000 7fa522a00700 1 -- 172.21.15.53:0/1978135677 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa51c010430 con 0x7fa528104ec0 2024-04-25T14:39:26.623 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.624+0000 7fa522a00700 1 --2- 172.21.15.53:0/1978135677 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fa5100a3cd0 0x7fa5100a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:26.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.624+0000 7fa522a00700 1 -- 172.21.15.53:0/1978135677 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fa51c0d0780 con 0x7fa528104ec0 2024-04-25T14:39:26.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.624+0000 7fa52d400700 1 --2- 172.21.15.53:0/1978135677 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fa5100a3cd0 0x7fa5100a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:26.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.625+0000 7fa52d400700 1 --2- 172.21.15.53:0/1978135677 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fa5100a3cd0 0x7fa5100a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa518000c00 tx=0x7fa518005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:26.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.628+0000 7fa522a00700 1 -- 172.21.15.53:0/1978135677 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa51c09cf90 con 0x7fa528104ec0 2024-04-25T14:39:26.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.878+0000 7fa530800700 1 -- 172.21.15.53:0/1978135677 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fa52802df80 con 0x7fa528104ec0 2024-04-25T14:39:26.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.878+0000 7fa522a00700 1 -- 172.21.15.53:0/1978135677 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fa51c085e20 con 0x7fa528104ec0 2024-04-25T14:39:26.878 INFO:teuthology.orchestra.run.smithi053.stdout:356482285598 2024-04-25T14:39:26.882 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.882+0000 7fa530800700 1 -- 172.21.15.53:0/1978135677 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fa5100a3cd0 msgr2=0x7fa5100a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:26.882 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.882+0000 7fa530800700 1 --2- 172.21.15.53:0/1978135677 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fa5100a3cd0 0x7fa5100a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa518000c00 tx=0x7fa518005fb0 comp rx=0 tx=0).stop 2024-04-25T14:39:26.882 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.882+0000 7fa530800700 1 -- 172.21.15.53:0/1978135677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa528104ec0 msgr2=0x7fa52819b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:26.882 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.882+0000 7fa530800700 1 --2- 172.21.15.53:0/1978135677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa528104ec0 0x7fa52819b300 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa51c00e9a0 tx=0x7fa51c00ecb0 comp rx=0 tx=0).stop 2024-04-25T14:39:26.882 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.882+0000 7fa530800700 1 -- 172.21.15.53:0/1978135677 shutdown_connections 2024-04-25T14:39:26.882 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.882+0000 7fa530800700 1 --2- 172.21.15.53:0/1978135677 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fa5100a3cd0 0x7fa5100a6190 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:26.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.882+0000 7fa530800700 1 --2- 172.21.15.53:0/1978135677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa528109fa0 0x7fa52819fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:26.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.882+0000 7fa530800700 1 --2- 172.21.15.53:0/1978135677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa528104ec0 0x7fa52819b300 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:26.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.882+0000 7fa530800700 1 --2- 172.21.15.53:0/1978135677 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa528105870 0x7fa52819b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:26.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.882+0000 7fa530800700 1 -- 172.21.15.53:0/1978135677 >> 172.21.15.53:0/1978135677 conn(0x7fa528100750 msgr2=0x7fa528102a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:26.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.882+0000 7fa530800700 1 -- 172.21.15.53:0/1978135677 shutdown_connections 2024-04-25T14:39:26.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:26.882+0000 7fa530800700 1 -- 172.21.15.53:0/1978135677 wait complete. 2024-04-25T14:39:27.201 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738451 got 34359738455 for osd.0 2024-04-25T14:39:27.201 DEBUG:teuthology.parallel:result is None 2024-04-25T14:39:27.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:27 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1978135677' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T14:39:27.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:27 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/1978135677' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T14:39:27.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:27 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/1978135677' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T14:39:27.603 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888127 got 146028888133 for osd.4 2024-04-25T14:39:27.603 DEBUG:teuthology.parallel:result is None 2024-04-25T14:39:28.214 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:28 smithi053 ceph-mon[27773]: pgmap v356: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s; 99 B/s, 0 objects/s recovering 2024-04-25T14:39:28.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.213+0000 7f785fa00700 1 -- 172.21.15.53:0/370197915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7858113c10 msgr2=0x7f7858116000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:28.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.213+0000 7f785fa00700 1 --2- 172.21.15.53:0/370197915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7858113c10 0x7f7858116000 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f785000b0a0 tx=0x7f785000b3b0 comp rx=0 tx=0).stop 2024-04-25T14:39:28.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.214+0000 7f785fa00700 1 -- 172.21.15.53:0/370197915 shutdown_connections 2024-04-25T14:39:28.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.214+0000 7f785fa00700 1 --2- 172.21.15.53:0/370197915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7858113c10 0x7f7858116000 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:28.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.214+0000 7f785fa00700 1 --2- 172.21.15.53:0/370197915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7858074c70 0x7f7858075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:28.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.214+0000 7f785fa00700 1 --2- 172.21.15.53:0/370197915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7858075590 0x7f78581136d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:28.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.214+0000 7f785fa00700 1 -- 172.21.15.53:0/370197915 >> 172.21.15.53:0/370197915 conn(0x7f7858079ba0 msgr2=0x7f7858079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:28.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.214+0000 7f785fa00700 1 -- 172.21.15.53:0/370197915 shutdown_connections 2024-04-25T14:39:28.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.214+0000 7f785fa00700 1 -- 172.21.15.53:0/370197915 wait complete. 2024-04-25T14:39:28.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.215+0000 7f785fa00700 1 Processor -- start 2024-04-25T14:39:28.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.215+0000 7f785fa00700 1 -- start start 2024-04-25T14:39:28.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.215+0000 7f785fa00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7858074c70 0x7f785819f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:28.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.215+0000 7f785fa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7858075590 0x7f785819fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:28.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.216+0000 7f785fa00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7858113c10 0x7f78581a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:28.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.216+0000 7f785fa00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7858119240 con 0x7f7858075590 2024-04-25T14:39:28.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.216+0000 7f785fa00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f78581190c0 con 0x7f7858074c70 2024-04-25T14:39:28.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.216+0000 7f785fa00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f78581193c0 con 0x7f7858113c10 2024-04-25T14:39:28.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.216+0000 7f7857e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7858075590 0x7f785819fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:28.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.216+0000 7f785d000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7858074c70 0x7f785819f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:28.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.216+0000 7f7857e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7858075590 0x7f785819fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38240/0 (socket says 172.21.15.53:38240) 2024-04-25T14:39:28.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.216+0000 7f7857e00700 1 -- 172.21.15.53:0/560557691 learned_addr learned my addr 172.21.15.53:0/560557691 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:28.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.216+0000 7f785da00700 1 --2- 172.21.15.53:0/560557691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7858113c10 0x7f78581a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:28.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.216+0000 7f785d000700 1 -- 172.21.15.53:0/560557691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7858113c10 msgr2=0x7f78581a4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:28.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.216+0000 7f785d000700 1 --2- 172.21.15.53:0/560557691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7858113c10 0x7f78581a4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:28.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.216+0000 7f785d000700 1 -- 172.21.15.53:0/560557691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7858075590 msgr2=0x7f785819fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:28.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.216+0000 7f785d000700 1 --2- 172.21.15.53:0/560557691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7858075590 0x7f785819fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:28.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.216+0000 7f785d000700 1 -- 172.21.15.53:0/560557691 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78581a4740 con 0x7f7858074c70 2024-04-25T14:39:28.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.216+0000 7f7857e00700 1 --2- 172.21.15.53:0/560557691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7858075590 0x7f785819fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:39:28.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.217+0000 7f785d000700 1 --2- 172.21.15.53:0/560557691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7858074c70 0x7f785819f730 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f784c00e9f0 tx=0x7f784c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:28.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.217+0000 7f7855600700 1 -- 172.21.15.53:0/560557691 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c004220 con 0x7f7858074c70 2024-04-25T14:39:28.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.217+0000 7f785fa00700 1 -- 172.21.15.53:0/560557691 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78581a4a30 con 0x7f7858074c70 2024-04-25T14:39:28.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.217+0000 7f7855600700 1 -- 172.21.15.53:0/560557691 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f784c0043c0 con 0x7f7858074c70 2024-04-25T14:39:28.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.217+0000 7f7855600700 1 -- 172.21.15.53:0/560557691 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c005020 con 0x7f7858074c70 2024-04-25T14:39:28.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.217+0000 7f785fa00700 1 -- 172.21.15.53:0/560557691 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78581ac2b0 con 0x7f7858074c70 2024-04-25T14:39:28.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.218+0000 7f785fa00700 1 -- 172.21.15.53:0/560557691 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7858069730 con 0x7f7858074c70 2024-04-25T14:39:28.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.220+0000 7f7855600700 1 -- 172.21.15.53:0/560557691 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f784c004750 con 0x7f7858074c70 2024-04-25T14:39:28.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.222+0000 7f7855600700 1 --2- 172.21.15.53:0/560557691 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f78400a3cd0 0x7f78400a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:28.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.222+0000 7f7855600700 1 -- 172.21.15.53:0/560557691 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f784c0d0580 con 0x7f7858074c70 2024-04-25T14:39:28.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.222+0000 7f7857e00700 1 --2- 172.21.15.53:0/560557691 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f78400a3cd0 0x7f78400a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:28.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.223+0000 7f7857e00700 1 --2- 172.21.15.53:0/560557691 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f78400a3cd0 0x7f78400a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7858101b60 tx=0x7f7844009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:28.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.226+0000 7f7855600700 1 -- 172.21.15.53:0/560557691 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f784c09cd90 con 0x7f7858074c70 2024-04-25T14:39:28.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:28 smithi183 ceph-mon[32584]: pgmap v356: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s; 99 B/s, 0 objects/s recovering 2024-04-25T14:39:28.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:28 smithi150 ceph-mon[32504]: pgmap v356: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s; 99 B/s, 0 objects/s recovering 2024-04-25T14:39:28.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.465+0000 7f785fa00700 1 -- 172.21.15.53:0/560557691 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f78581139d0 con 0x7f7858074c70 2024-04-25T14:39:28.466 INFO:teuthology.orchestra.run.smithi053.stdout:266287972399 2024-04-25T14:39:28.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.466+0000 7f7855600700 1 -- 172.21.15.53:0/560557691 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f78581139d0 con 0x7f7858074c70 2024-04-25T14:39:28.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.468+0000 7f785fa00700 1 -- 172.21.15.53:0/560557691 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f78400a3cd0 msgr2=0x7f78400a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:28.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.468+0000 7f785fa00700 1 --2- 172.21.15.53:0/560557691 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f78400a3cd0 0x7f78400a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7858101b60 tx=0x7f7844009260 comp rx=0 tx=0).stop 2024-04-25T14:39:28.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.468+0000 7f785fa00700 1 -- 172.21.15.53:0/560557691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7858074c70 msgr2=0x7f785819f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:28.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.469+0000 7f785fa00700 1 --2- 172.21.15.53:0/560557691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7858074c70 0x7f785819f730 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f784c00e9f0 tx=0x7f784c00ed00 comp rx=0 tx=0).stop 2024-04-25T14:39:28.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.469+0000 7f785fa00700 1 -- 172.21.15.53:0/560557691 shutdown_connections 2024-04-25T14:39:28.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.469+0000 7f785fa00700 1 --2- 172.21.15.53:0/560557691 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f78400a3cd0 0x7f78400a6190 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:28.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.469+0000 7f785fa00700 1 --2- 172.21.15.53:0/560557691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7858113c10 0x7f78581a4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:28.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.469+0000 7f785fa00700 1 --2- 172.21.15.53:0/560557691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7858074c70 0x7f785819f730 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:28.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.469+0000 7f785fa00700 1 --2- 172.21.15.53:0/560557691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7858075590 0x7f785819fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:28.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.469+0000 7f785fa00700 1 -- 172.21.15.53:0/560557691 >> 172.21.15.53:0/560557691 conn(0x7f7858079ba0 msgr2=0x7f7858110b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:28.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.469+0000 7f785fa00700 1 -- 172.21.15.53:0/560557691 shutdown_connections 2024-04-25T14:39:28.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.469+0000 7f785fa00700 1 -- 172.21.15.53:0/560557691 wait complete. 2024-04-25T14:39:28.925 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574925 got 55834574930 for osd.1 2024-04-25T14:39:28.926 DEBUG:teuthology.parallel:result is None 2024-04-25T14:39:29.296 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149762 got 111669149769 for osd.3 2024-04-25T14:39:29.296 DEBUG:teuthology.parallel:result is None 2024-04-25T14:39:29.399 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:29 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/560557691' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T14:39:29.399 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:29 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1479638019' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T14:39:29.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:29 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/560557691' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T14:39:29.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:29 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/1479638019' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T14:39:29.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:29 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/560557691' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T14:39:29.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:29 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/1479638019' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T14:39:29.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.668+0000 7fcf30e00700 1 -- 172.21.15.53:0/1684629466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf2c105840 msgr2=0x7fcf2c109890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:29.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.668+0000 7fcf30e00700 1 --2- 172.21.15.53:0/1684629466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf2c105840 0x7fcf2c109890 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcf140099c0 tx=0x7fcf14009cd0 comp rx=0 tx=0).stop 2024-04-25T14:39:29.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.668+0000 7fcf30e00700 1 -- 172.21.15.53:0/1684629466 shutdown_connections 2024-04-25T14:39:29.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.668+0000 7fcf30e00700 1 --2- 172.21.15.53:0/1684629466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf2c109f70 0x7fcf2c111b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:29.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.668+0000 7fcf30e00700 1 --2- 172.21.15.53:0/1684629466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf2c104e90 0x7fcf2c105270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:29.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.668+0000 7fcf30e00700 1 --2- 172.21.15.53:0/1684629466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf2c105840 0x7fcf2c109890 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:29.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.668+0000 7fcf30e00700 1 -- 172.21.15.53:0/1684629466 >> 172.21.15.53:0/1684629466 conn(0x7fcf2c100720 msgr2=0x7fcf2c102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:29.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.668+0000 7fcf30e00700 1 -- 172.21.15.53:0/1684629466 shutdown_connections 2024-04-25T14:39:29.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.668+0000 7fcf30e00700 1 -- 172.21.15.53:0/1684629466 wait complete. 2024-04-25T14:39:29.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.669+0000 7fcf30e00700 1 Processor -- start 2024-04-25T14:39:29.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.669+0000 7fcf30e00700 1 -- start start 2024-04-25T14:39:29.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.670+0000 7fcf30e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf2c104e90 0x7fcf2c19b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:29.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.670+0000 7fcf30e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf2c105840 0x7fcf2c19b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:29.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.670+0000 7fcf30e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf2c109f70 0x7fcf2c19fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:29.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.670+0000 7fcf30e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c114e30 con 0x7fcf2c104e90 2024-04-25T14:39:29.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.670+0000 7fcf30e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c114cb0 con 0x7fcf2c105840 2024-04-25T14:39:29.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.670+0000 7fcf30e00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c114fb0 con 0x7fcf2c109f70 2024-04-25T14:39:29.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.670+0000 7fcf29e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf2c104e90 0x7fcf2c19b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:29.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.670+0000 7fcf23e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf2c105840 0x7fcf2c19b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:29.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.670+0000 7fcf23e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf2c105840 0x7fcf2c19b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.53:38572/0 (socket says 172.21.15.53:38572) 2024-04-25T14:39:29.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.670+0000 7fcf29e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf2c104e90 0x7fcf2c19b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38278/0 (socket says 172.21.15.53:38278) 2024-04-25T14:39:29.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.670+0000 7fcf23e00700 1 -- 172.21.15.53:0/1938200299 learned_addr learned my addr 172.21.15.53:0/1938200299 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:29.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.670+0000 7fcf2a800700 1 --2- 172.21.15.53:0/1938200299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf2c109f70 0x7fcf2c19fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:29.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.671+0000 7fcf23e00700 1 -- 172.21.15.53:0/1938200299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf2c109f70 msgr2=0x7fcf2c19fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:29.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.671+0000 7fcf23e00700 1 --2- 172.21.15.53:0/1938200299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf2c109f70 0x7fcf2c19fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:29.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.671+0000 7fcf23e00700 1 -- 172.21.15.53:0/1938200299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf2c104e90 msgr2=0x7fcf2c19b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:29.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.671+0000 7fcf23e00700 1 --2- 172.21.15.53:0/1938200299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf2c104e90 0x7fcf2c19b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:29.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.671+0000 7fcf23e00700 1 -- 172.21.15.53:0/1938200299 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf2c1a0360 con 0x7fcf2c105840 2024-04-25T14:39:29.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.671+0000 7fcf2a800700 1 --2- 172.21.15.53:0/1938200299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf2c109f70 0x7fcf2c19fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:39:29.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.671+0000 7fcf23e00700 1 --2- 172.21.15.53:0/1938200299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf2c105840 0x7fcf2c19b8e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcf14005f50 tx=0x7fcf14005280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:29.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.672+0000 7fcf22a00700 1 -- 172.21.15.53:0/1938200299 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf14026070 con 0x7fcf2c105840 2024-04-25T14:39:29.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.672+0000 7fcf22a00700 1 -- 172.21.15.53:0/1938200299 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf14003c70 con 0x7fcf2c105840 2024-04-25T14:39:29.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.672+0000 7fcf30e00700 1 -- 172.21.15.53:0/1938200299 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf2c1a05f0 con 0x7fcf2c105840 2024-04-25T14:39:29.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.672+0000 7fcf22a00700 1 -- 172.21.15.53:0/1938200299 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1400f960 con 0x7fcf2c105840 2024-04-25T14:39:29.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.672+0000 7fcf30e00700 1 -- 172.21.15.53:0/1938200299 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf2c1a7ed0 con 0x7fcf2c105840 2024-04-25T14:39:29.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.673+0000 7fcf30e00700 1 -- 172.21.15.53:0/1938200299 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf2c069730 con 0x7fcf2c105840 2024-04-25T14:39:29.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.675+0000 7fcf22a00700 1 -- 172.21.15.53:0/1938200299 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcf14029070 con 0x7fcf2c105840 2024-04-25T14:39:29.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.677+0000 7fcf22a00700 1 --2- 172.21.15.53:0/1938200299 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fcf100a3d50 0x7fcf100a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:29.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.677+0000 7fcf22a00700 1 -- 172.21.15.53:0/1938200299 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fcf140d1090 con 0x7fcf2c105840 2024-04-25T14:39:29.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.677+0000 7fcf29e00700 1 --2- 172.21.15.53:0/1938200299 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fcf100a3d50 0x7fcf100a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:29.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.678+0000 7fcf29e00700 1 --2- 172.21.15.53:0/1938200299 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fcf100a3d50 0x7fcf100a6210 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c009c60 tx=0x7fcf1c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:29.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.679+0000 7fcf22a00700 1 -- 172.21.15.53:0/1938200299 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcf14030020 con 0x7fcf2c105840 2024-04-25T14:39:29.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.914+0000 7fcf30e00700 1 -- 172.21.15.53:0/1938200299 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fcf2c02df80 con 0x7fcf2c105840 2024-04-25T14:39:29.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.915+0000 7fcf22a00700 1 -- 172.21.15.53:0/1938200299 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fcf14013070 con 0x7fcf2c105840 2024-04-25T14:39:29.915 INFO:teuthology.orchestra.run.smithi053.stdout:322122547235 2024-04-25T14:39:29.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.918+0000 7fcf30e00700 1 -- 172.21.15.53:0/1938200299 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fcf100a3d50 msgr2=0x7fcf100a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:29.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.918+0000 7fcf30e00700 1 --2- 172.21.15.53:0/1938200299 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fcf100a3d50 0x7fcf100a6210 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c009c60 tx=0x7fcf1c009260 comp rx=0 tx=0).stop 2024-04-25T14:39:29.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.918+0000 7fcf30e00700 1 -- 172.21.15.53:0/1938200299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf2c105840 msgr2=0x7fcf2c19b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:29.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.918+0000 7fcf30e00700 1 --2- 172.21.15.53:0/1938200299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf2c105840 0x7fcf2c19b8e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcf14005f50 tx=0x7fcf14005280 comp rx=0 tx=0).stop 2024-04-25T14:39:29.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.918+0000 7fcf30e00700 1 -- 172.21.15.53:0/1938200299 shutdown_connections 2024-04-25T14:39:29.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.918+0000 7fcf30e00700 1 --2- 172.21.15.53:0/1938200299 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fcf100a3d50 0x7fcf100a6210 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:29.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.918+0000 7fcf30e00700 1 --2- 172.21.15.53:0/1938200299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf2c109f70 0x7fcf2c19fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:29.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.918+0000 7fcf30e00700 1 --2- 172.21.15.53:0/1938200299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf2c104e90 0x7fcf2c19b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:29.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.918+0000 7fcf30e00700 1 --2- 172.21.15.53:0/1938200299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf2c105840 0x7fcf2c19b8e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:29.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.918+0000 7fcf30e00700 1 -- 172.21.15.53:0/1938200299 >> 172.21.15.53:0/1938200299 conn(0x7fcf2c100720 msgr2=0x7fcf2c102a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:29.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.918+0000 7fcf30e00700 1 -- 172.21.15.53:0/1938200299 shutdown_connections 2024-04-25T14:39:29.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:29.918+0000 7fcf30e00700 1 -- 172.21.15.53:0/1938200299 wait complete. 2024-04-25T14:39:30.022 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411401 got 77309411407 for osd.2 2024-04-25T14:39:30.022 DEBUG:teuthology.parallel:result is None 2024-04-25T14:39:30.220 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:30 smithi053 ceph-mon[27773]: pgmap v357: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s; 86 B/s, 0 objects/s recovering 2024-04-25T14:39:30.220 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:30 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1938200299' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T14:39:30.355 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:30 smithi183 ceph-mon[32584]: pgmap v357: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s; 86 B/s, 0 objects/s recovering 2024-04-25T14:39:30.355 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:30 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/1938200299' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T14:39:30.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:30 smithi150 ceph-mon[32504]: pgmap v357: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s; 86 B/s, 0 objects/s recovering 2024-04-25T14:39:30.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:30 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/1938200299' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T14:39:31.112 INFO:tasks.cephadm.ceph_manager.ceph:need seq 322122547229 got 322122547235 for osd.10 2024-04-25T14:39:31.112 DEBUG:teuthology.parallel:result is None 2024-04-25T14:39:31.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.561+0000 7f61ab400700 1 -- 172.21.15.53:0/4249824609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61a410f230 msgr2=0x7f61a41116d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:31.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.561+0000 7f61ab400700 1 --2- 172.21.15.53:0/4249824609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61a410f230 0x7f61a41116d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f619c00b0a0 tx=0x7f619c00b3b0 comp rx=0 tx=0).stop 2024-04-25T14:39:31.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.561+0000 7f61ab400700 1 -- 172.21.15.53:0/4249824609 shutdown_connections 2024-04-25T14:39:31.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.561+0000 7f61ab400700 1 --2- 172.21.15.53:0/4249824609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61a410f230 0x7f61a41116d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:31.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.561+0000 7f61ab400700 1 --2- 172.21.15.53:0/4249824609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61a4100e20 0x7f61a4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:31.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.561+0000 7f61ab400700 1 --2- 172.21.15.53:0/4249824609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61a4101740 0x7f61a410ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:31.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.561+0000 7f61ab400700 1 -- 172.21.15.53:0/4249824609 >> 172.21.15.53:0/4249824609 conn(0x7f61a4077b40 msgr2=0x7f61a40feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:31.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.561+0000 7f61ab400700 1 -- 172.21.15.53:0/4249824609 shutdown_connections 2024-04-25T14:39:31.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.561+0000 7f61ab400700 1 -- 172.21.15.53:0/4249824609 wait complete. 2024-04-25T14:39:31.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.562+0000 7f61ab400700 1 Processor -- start 2024-04-25T14:39:31.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.562+0000 7f61ab400700 1 -- start start 2024-04-25T14:39:31.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.562+0000 7f61ab400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61a4100e20 0x7f61a419b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:31.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.562+0000 7f61ab400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61a4101740 0x7f61a419b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:31.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.562+0000 7f61ab400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61a410f230 0x7f61a419fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:31.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.562+0000 7f61ab400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f61a4114e70 con 0x7f61a410f230 2024-04-25T14:39:31.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.562+0000 7f61ab400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f61a4114cf0 con 0x7f61a4101740 2024-04-25T14:39:31.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.562+0000 7f61ab400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f61a4114ff0 con 0x7f61a4100e20 2024-04-25T14:39:31.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.563+0000 7f61a9400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61a410f230 0x7f61a419fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:31.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.563+0000 7f61a9400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61a410f230 0x7f61a419fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38252/0 (socket says 172.21.15.53:38252) 2024-04-25T14:39:31.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.563+0000 7f61a8a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61a4100e20 0x7f61a419b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:31.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.563+0000 7f61a9400700 1 -- 172.21.15.53:0/1479638019 learned_addr learned my addr 172.21.15.53:0/1479638019 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:31.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.563+0000 7f61a3e00700 1 --2- 172.21.15.53:0/1479638019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61a4101740 0x7f61a419b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:31.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.563+0000 7f61a9400700 1 -- 172.21.15.53:0/1479638019 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61a4100e20 msgr2=0x7f61a419b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:31.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.563+0000 7f61a9400700 1 --2- 172.21.15.53:0/1479638019 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61a4100e20 0x7f61a419b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:31.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.563+0000 7f61a9400700 1 -- 172.21.15.53:0/1479638019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61a4101740 msgr2=0x7f61a419b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:31.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.563+0000 7f61a9400700 1 --2- 172.21.15.53:0/1479638019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61a4101740 0x7f61a419b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:31.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.563+0000 7f61a9400700 1 -- 172.21.15.53:0/1479638019 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61a41a0290 con 0x7f61a410f230 2024-04-25T14:39:31.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.563+0000 7f61a8a00700 1 --2- 172.21.15.53:0/1479638019 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61a4100e20 0x7f61a419b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:39:31.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.564+0000 7f61a9400700 1 --2- 172.21.15.53:0/1479638019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61a410f230 0x7f61a419fb50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f619c00b070 tx=0x7f619c009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:31.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.564+0000 7f61a1600700 1 -- 172.21.15.53:0/1479638019 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619c021070 con 0x7f61a410f230 2024-04-25T14:39:31.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.564+0000 7f61a1600700 1 -- 172.21.15.53:0/1479638019 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f619c004e40 con 0x7f61a410f230 2024-04-25T14:39:31.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.564+0000 7f61a1600700 1 -- 172.21.15.53:0/1479638019 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619c004630 con 0x7f61a410f230 2024-04-25T14:39:31.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.564+0000 7f61ab400700 1 -- 172.21.15.53:0/1479638019 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61a41a0520 con 0x7f61a410f230 2024-04-25T14:39:31.930 INFO:teuthology.orchestra.run.smithi053.stdout:300647710760 2024-04-25T14:39:31.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.564+0000 7f61ab400700 1 -- 172.21.15.53:0/1479638019 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61a41a7e00 con 0x7f61a410f230 2024-04-25T14:39:31.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.565+0000 7f61ab400700 1 -- 172.21.15.53:0/1479638019 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61a4101f50 con 0x7f61a410f230 2024-04-25T14:39:31.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.568+0000 7f61a1600700 1 -- 172.21.15.53:0/1479638019 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f619c007980 con 0x7f61a410f230 2024-04-25T14:39:31.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.569+0000 7f61a1600700 1 --2- 172.21.15.53:0/1479638019 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f618c0a3d50 0x7f618c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:31.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.570+0000 7f61a1600700 1 -- 172.21.15.53:0/1479638019 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f619c00f020 con 0x7f61a410f230 2024-04-25T14:39:31.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.570+0000 7f61a8a00700 1 --2- 172.21.15.53:0/1479638019 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f618c0a3d50 0x7f618c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:31.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.570+0000 7f61a8a00700 1 --2- 172.21.15.53:0/1479638019 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f618c0a3d50 0x7f618c0a6210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6198011490 tx=0x7f6198009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:31.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.574+0000 7f61a1600700 1 -- 172.21.15.53:0/1479638019 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f619c02a020 con 0x7f61a410f230 2024-04-25T14:39:31.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.829+0000 7f61ab400700 1 -- 172.21.15.53:0/1479638019 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f61a402df80 con 0x7f61a410f230 2024-04-25T14:39:31.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.829+0000 7f61a1600700 1 -- 172.21.15.53:0/1479638019 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f619c09e120 con 0x7f61a410f230 2024-04-25T14:39:31.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.832+0000 7f61ab400700 1 -- 172.21.15.53:0/1479638019 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f618c0a3d50 msgr2=0x7f618c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:31.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.832+0000 7f61ab400700 1 --2- 172.21.15.53:0/1479638019 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f618c0a3d50 0x7f618c0a6210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6198011490 tx=0x7f6198009260 comp rx=0 tx=0).stop 2024-04-25T14:39:31.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.832+0000 7f61ab400700 1 -- 172.21.15.53:0/1479638019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61a410f230 msgr2=0x7f61a419fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:31.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.832+0000 7f61ab400700 1 --2- 172.21.15.53:0/1479638019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61a410f230 0x7f61a419fb50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f619c00b070 tx=0x7f619c009c90 comp rx=0 tx=0).stop 2024-04-25T14:39:31.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.832+0000 7f61ab400700 1 -- 172.21.15.53:0/1479638019 shutdown_connections 2024-04-25T14:39:31.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.832+0000 7f61ab400700 1 --2- 172.21.15.53:0/1479638019 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f618c0a3d50 0x7f618c0a6210 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:31.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.832+0000 7f61ab400700 1 --2- 172.21.15.53:0/1479638019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61a410f230 0x7f61a419fb50 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:31.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.832+0000 7f61ab400700 1 --2- 172.21.15.53:0/1479638019 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61a4100e20 0x7f61a419b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:31.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.832+0000 7f61ab400700 1 --2- 172.21.15.53:0/1479638019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61a4101740 0x7f61a419b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:31.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.832+0000 7f61ab400700 1 -- 172.21.15.53:0/1479638019 >> 172.21.15.53:0/1479638019 conn(0x7f61a4077b40 msgr2=0x7f61a4110460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:31.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.833+0000 7f61ab400700 1 -- 172.21.15.53:0/1479638019 shutdown_connections 2024-04-25T14:39:31.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:28.833+0000 7f61ab400700 1 -- 172.21.15.53:0/1479638019 wait complete. 2024-04-25T14:39:32.094 INFO:tasks.cephadm.ceph_manager.ceph:need seq 356482285593 got 356482285598 for osd.11 2024-04-25T14:39:32.094 DEBUG:teuthology.parallel:result is None 2024-04-25T14:39:32.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:32 smithi053 ceph-mon[27773]: pgmap v358: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T14:39:32.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:32 smithi183 ceph-mon[32584]: pgmap v358: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T14:39:32.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:32 smithi150 ceph-mon[32504]: pgmap v358: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T14:39:32.625 INFO:tasks.cephadm.ceph_manager.ceph:need seq 266287972393 got 266287972399 for osd.8 2024-04-25T14:39:32.625 DEBUG:teuthology.parallel:result is None 2024-04-25T14:39:33.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.078+0000 7f22dcc00700 1 -- 172.21.15.53:0/1665847636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22d8108a50 msgr2=0x7f22d8108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:33.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.078+0000 7f22dcc00700 1 --2- 172.21.15.53:0/1665847636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22d8108a50 0x7f22d8108e30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f22c80098f0 tx=0x7f22c8009c00 comp rx=0 tx=0).stop 2024-04-25T14:39:33.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.078+0000 7f22dcc00700 1 -- 172.21.15.53:0/1665847636 shutdown_connections 2024-04-25T14:39:33.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.078+0000 7f22dcc00700 1 --2- 172.21.15.53:0/1665847636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22d8103450 0x7f22d810f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:33.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.078+0000 7f22dcc00700 1 --2- 172.21.15.53:0/1665847636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22d8108a50 0x7f22d8108e30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:33.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.078+0000 7f22dcc00700 1 --2- 172.21.15.53:0/1665847636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22d8102a90 0x7f22d8102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:33.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.078+0000 7f22dcc00700 1 -- 172.21.15.53:0/1665847636 >> 172.21.15.53:0/1665847636 conn(0x7f22d8079ba0 msgr2=0x7f22d8079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:33.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.078+0000 7f22dcc00700 1 -- 172.21.15.53:0/1665847636 shutdown_connections 2024-04-25T14:39:33.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.078+0000 7f22dcc00700 1 -- 172.21.15.53:0/1665847636 wait complete. 2024-04-25T14:39:33.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.079+0000 7f22dcc00700 1 Processor -- start 2024-04-25T14:39:33.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.079+0000 7f22dcc00700 1 -- start start 2024-04-25T14:39:33.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.080+0000 7f22dcc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22d8102a90 0x7f22d81991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:33.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.080+0000 7f22dcc00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22d8103450 0x7f22d8199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:33.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.080+0000 7f22dcc00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22d8108a50 0x7f22d819da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:33.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.080+0000 7f22dcc00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f22d8112c30 con 0x7f22d8102a90 2024-04-25T14:39:33.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.080+0000 7f22dcc00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f22d8112ab0 con 0x7f22d8108a50 2024-04-25T14:39:33.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.080+0000 7f22dcc00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f22d8112db0 con 0x7f22d8103450 2024-04-25T14:39:33.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.080+0000 7f22d5e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22d8102a90 0x7f22d81991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:33.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.080+0000 7f22d5e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22d8102a90 0x7f22d81991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48868/0 (socket says 172.21.15.53:48868) 2024-04-25T14:39:33.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.080+0000 7f22d5e00700 1 -- 172.21.15.53:0/464723610 learned_addr learned my addr 172.21.15.53:0/464723610 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:33.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.080+0000 7f22cfe00700 1 --2- 172.21.15.53:0/464723610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22d8103450 0x7f22d8199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:33.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.080+0000 7f22d6800700 1 --2- 172.21.15.53:0/464723610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22d8108a50 0x7f22d819da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:33.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.081+0000 7f22d5e00700 1 -- 172.21.15.53:0/464723610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22d8103450 msgr2=0x7f22d8199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:33.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.081+0000 7f22d5e00700 1 --2- 172.21.15.53:0/464723610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22d8103450 0x7f22d8199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:33.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.081+0000 7f22d5e00700 1 -- 172.21.15.53:0/464723610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22d8108a50 msgr2=0x7f22d819da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:33.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.081+0000 7f22d5e00700 1 --2- 172.21.15.53:0/464723610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22d8108a50 0x7f22d819da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:33.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.081+0000 7f22d5e00700 1 -- 172.21.15.53:0/464723610 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22d819e1d0 con 0x7f22d8102a90 2024-04-25T14:39:33.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.081+0000 7f22cfe00700 1 --2- 172.21.15.53:0/464723610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22d8103450 0x7f22d8199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:39:33.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.081+0000 7f22d5e00700 1 --2- 172.21.15.53:0/464723610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22d8102a90 0x7f22d81991c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f22c8009230 tx=0x7f22c800fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:33.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.081+0000 7f22cea00700 1 -- 172.21.15.53:0/464723610 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c80050c0 con 0x7f22d8102a90 2024-04-25T14:39:33.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.081+0000 7f22cea00700 1 -- 172.21.15.53:0/464723610 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22c800b560 con 0x7f22d8102a90 2024-04-25T14:39:33.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.081+0000 7f22dcc00700 1 -- 172.21.15.53:0/464723610 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22d819e460 con 0x7f22d8102a90 2024-04-25T14:39:33.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.081+0000 7f22cea00700 1 -- 172.21.15.53:0/464723610 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c8003a00 con 0x7f22d8102a90 2024-04-25T14:39:33.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.082+0000 7f22dcc00700 1 -- 172.21.15.53:0/464723610 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22d81a5d40 con 0x7f22d8102a90 2024-04-25T14:39:33.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.083+0000 7f22dcc00700 1 -- 172.21.15.53:0/464723610 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22d8069730 con 0x7f22d8102a90 2024-04-25T14:39:33.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.085+0000 7f22cea00700 1 -- 172.21.15.53:0/464723610 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f22c800b710 con 0x7f22d8102a90 2024-04-25T14:39:33.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.086+0000 7f22cea00700 1 --2- 172.21.15.53:0/464723610 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f22bc0a3cd0 0x7f22bc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:33.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.086+0000 7f22cea00700 1 -- 172.21.15.53:0/464723610 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f22c80d14c0 con 0x7f22d8102a90 2024-04-25T14:39:33.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.087+0000 7f22cfe00700 1 --2- 172.21.15.53:0/464723610 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f22bc0a3cd0 0x7f22bc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:33.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.087+0000 7f22cfe00700 1 --2- 172.21.15.53:0/464723610 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f22bc0a3cd0 0x7f22bc0a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f22c0009750 tx=0x7f22c0009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:33.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.091+0000 7f22cea00700 1 -- 172.21.15.53:0/464723610 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f22c8013030 con 0x7f22d8102a90 2024-04-25T14:39:33.242 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:39:33.252 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710754 got 300647710760 for osd.9 2024-04-25T14:39:33.252 DEBUG:teuthology.parallel:result is None 2024-04-25T14:39:33.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.353+0000 7f22dcc00700 1 -- 172.21.15.53:0/464723610 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f22d802df80 con 0x7f22d8102a90 2024-04-25T14:39:33.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.354+0000 7f22cea00700 1 -- 172.21.15.53:0/464723610 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f22c8021380 con 0x7f22d8102a90 2024-04-25T14:39:33.354 INFO:teuthology.orchestra.run.smithi053.stdout:171798691905 2024-04-25T14:39:33.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.357+0000 7f22dcc00700 1 -- 172.21.15.53:0/464723610 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f22bc0a3cd0 msgr2=0x7f22bc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:33.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.357+0000 7f22dcc00700 1 --2- 172.21.15.53:0/464723610 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f22bc0a3cd0 0x7f22bc0a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f22c0009750 tx=0x7f22c0009310 comp rx=0 tx=0).stop 2024-04-25T14:39:33.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.357+0000 7f22dcc00700 1 -- 172.21.15.53:0/464723610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22d8102a90 msgr2=0x7f22d81991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:33.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.357+0000 7f22dcc00700 1 --2- 172.21.15.53:0/464723610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22d8102a90 0x7f22d81991c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f22c8009230 tx=0x7f22c800fa70 comp rx=0 tx=0).stop 2024-04-25T14:39:33.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.357+0000 7f22dcc00700 1 -- 172.21.15.53:0/464723610 shutdown_connections 2024-04-25T14:39:33.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.357+0000 7f22dcc00700 1 --2- 172.21.15.53:0/464723610 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f22bc0a3cd0 0x7f22bc0a6190 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:33.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.357+0000 7f22dcc00700 1 --2- 172.21.15.53:0/464723610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22d8108a50 0x7f22d819da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:33.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.357+0000 7f22dcc00700 1 --2- 172.21.15.53:0/464723610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22d8102a90 0x7f22d81991c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:33.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.357+0000 7f22dcc00700 1 --2- 172.21.15.53:0/464723610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22d8103450 0x7f22d8199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:33.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.357+0000 7f22dcc00700 1 -- 172.21.15.53:0/464723610 >> 172.21.15.53:0/464723610 conn(0x7f22d8079ba0 msgr2=0x7f22d8101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:33.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.357+0000 7f22dcc00700 1 -- 172.21.15.53:0/464723610 shutdown_connections 2024-04-25T14:39:33.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:33.357+0000 7f22dcc00700 1 -- 172.21.15.53:0/464723610 wait complete. 2024-04-25T14:39:34.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:34 smithi053 ceph-mon[27773]: pgmap v359: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T14:39:34.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:34 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/464723610' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T14:39:34.419 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691896 got 171798691905 for osd.5 2024-04-25T14:39:34.419 DEBUG:teuthology.parallel:result is None 2024-04-25T14:39:34.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:34 smithi183 ceph-mon[32584]: pgmap v359: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T14:39:34.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:34 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/464723610' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T14:39:34.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:34 smithi150 ceph-mon[32504]: pgmap v359: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T14:39:34.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:34 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/464723610' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T14:39:36.429 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:36 smithi053 ceph-mon[27773]: pgmap v360: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T14:39:36.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:36 smithi183 ceph-mon[32584]: pgmap v360: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T14:39:36.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:36 smithi150 ceph-mon[32504]: pgmap v360: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T14:39:36.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:36.768+0000 7f3518c00700 1 -- 172.21.15.53:0/4182708228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3514108a80 msgr2=0x7f3514108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:36.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:36.768+0000 7f3518c00700 1 --2- 172.21.15.53:0/4182708228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3514108a80 0x7f3514108e60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3504009990 tx=0x7f3504009ca0 comp rx=0 tx=0).stop 2024-04-25T14:39:36.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:36.768+0000 7f3518c00700 1 -- 172.21.15.53:0/4182708228 shutdown_connections 2024-04-25T14:39:36.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:36.768+0000 7f3518c00700 1 --2- 172.21.15.53:0/4182708228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3514103450 0x7f351410f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:36.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:36.768+0000 7f3518c00700 1 --2- 172.21.15.53:0/4182708228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3514108a80 0x7f3514108e60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:36.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:36.768+0000 7f3518c00700 1 --2- 172.21.15.53:0/4182708228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3514102a90 0x7f3514102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:36.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:36.768+0000 7f3518c00700 1 -- 172.21.15.53:0/4182708228 >> 172.21.15.53:0/4182708228 conn(0x7f3514079bf0 msgr2=0x7f351407a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:36.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:36.769+0000 7f3518c00700 1 -- 172.21.15.53:0/4182708228 shutdown_connections 2024-04-25T14:39:36.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:36.769+0000 7f3518c00700 1 -- 172.21.15.53:0/4182708228 wait complete. 2024-04-25T14:39:36.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:36.770+0000 7f3518c00700 1 Processor -- start 2024-04-25T14:39:36.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:36.771+0000 7f3518c00700 1 -- start start 2024-04-25T14:39:36.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:36.771+0000 7f3518c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3514102a90 0x7f3514199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:36.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:36.771+0000 7f3518c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3514103450 0x7f3514199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:36.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:36.772+0000 7f3518c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3514108a80 0x7f351419dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:36.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:36.772+0000 7f3518c00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3514112c90 con 0x7f3514103450 2024-04-25T14:39:36.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:36.772+0000 7f3518c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3514112b10 con 0x7f3514102a90 2024-04-25T14:39:36.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:36.772+0000 7f3518c00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3514112e10 con 0x7f3514108a80 2024-04-25T14:39:36.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:36.772+0000 7f3511e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3514102a90 0x7f3514199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:36.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:36.772+0000 7f3511400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3514103450 0x7f3514199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:36.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:36.772+0000 7f3511e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3514102a90 0x7f3514199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.53:57928/0 (socket says 172.21.15.53:57928) 2024-04-25T14:39:36.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:36.772+0000 7f3511e00700 1 -- 172.21.15.53:0/4196393412 learned_addr learned my addr 172.21.15.53:0/4196393412 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:36.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:36.772+0000 7f3512800700 1 --2- 172.21.15.53:0/4196393412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3514108a80 0x7f351419dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:36.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:36.772+0000 7f3511e00700 1 -- 172.21.15.53:0/4196393412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3514108a80 msgr2=0x7f351419dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:36.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:36.772+0000 7f3511e00700 1 --2- 172.21.15.53:0/4196393412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3514108a80 0x7f351419dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:36.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:36.772+0000 7f3511e00700 1 -- 172.21.15.53:0/4196393412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3514103450 msgr2=0x7f3514199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:36.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:36.772+0000 7f3511e00700 1 --2- 172.21.15.53:0/4196393412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3514103450 0x7f3514199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:36.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:36.772+0000 7f3511e00700 1 -- 172.21.15.53:0/4196393412 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f351419e320 con 0x7f3514102a90 2024-04-25T14:39:36.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:36.773+0000 7f3511e00700 1 --2- 172.21.15.53:0/4196393412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3514102a90 0x7f3514199310 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3504005910 tx=0x7f350400b670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:36.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:36.773+0000 7f350aa00700 1 -- 172.21.15.53:0/4196393412 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3504027070 con 0x7f3514102a90 2024-04-25T14:39:36.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:36.773+0000 7f3518c00700 1 -- 172.21.15.53:0/4196393412 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f351419e5b0 con 0x7f3514102a90 2024-04-25T14:39:36.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:36.773+0000 7f350aa00700 1 -- 172.21.15.53:0/4196393412 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3504022470 con 0x7f3514102a90 2024-04-25T14:39:36.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:36.773+0000 7f350aa00700 1 -- 172.21.15.53:0/4196393412 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3504017650 con 0x7f3514102a90 2024-04-25T14:39:36.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:36.773+0000 7f3518c00700 1 -- 172.21.15.53:0/4196393412 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35141a5e90 con 0x7f3514102a90 2024-04-25T14:39:36.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:36.775+0000 7f3518c00700 1 -- 172.21.15.53:0/4196393412 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3514069730 con 0x7f3514102a90 2024-04-25T14:39:36.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:36.777+0000 7f350aa00700 1 -- 172.21.15.53:0/4196393412 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f350402a070 con 0x7f3514102a90 2024-04-25T14:39:36.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:36.778+0000 7f350aa00700 1 --2- 172.21.15.53:0/4196393412 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f34f80a3d50 0x7f34f80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:36.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:36.778+0000 7f350aa00700 1 -- 172.21.15.53:0/4196393412 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f35040d1240 con 0x7f3514102a90 2024-04-25T14:39:36.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:36.779+0000 7f3511400700 1 --2- 172.21.15.53:0/4196393412 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f34f80a3d50 0x7f34f80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:36.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:36.779+0000 7f3511400700 1 --2- 172.21.15.53:0/4196393412 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f34f80a3d50 0x7f34f80a6210 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f351407a1c0 tx=0x7f3500009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:36.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:36.783+0000 7f350aa00700 1 -- 172.21.15.53:0/4196393412 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3504013070 con 0x7f3514102a90 2024-04-25T14:39:37.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:37.029+0000 7f3518c00700 1 -- 172.21.15.53:0/4196393412 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f351402df80 con 0x7f3514102a90 2024-04-25T14:39:37.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:37.030+0000 7f350aa00700 1 -- 172.21.15.53:0/4196393412 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f350409dae0 con 0x7f3514102a90 2024-04-25T14:39:37.030 INFO:teuthology.orchestra.run.smithi053.stdout:227633266740 2024-04-25T14:39:37.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:37.033+0000 7f3518c00700 1 -- 172.21.15.53:0/4196393412 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f34f80a3d50 msgr2=0x7f34f80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:37.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:37.033+0000 7f3518c00700 1 --2- 172.21.15.53:0/4196393412 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f34f80a3d50 0x7f34f80a6210 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f351407a1c0 tx=0x7f3500009260 comp rx=0 tx=0).stop 2024-04-25T14:39:37.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:37.033+0000 7f3518c00700 1 -- 172.21.15.53:0/4196393412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3514102a90 msgr2=0x7f3514199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:37.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:37.033+0000 7f3518c00700 1 --2- 172.21.15.53:0/4196393412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3514102a90 0x7f3514199310 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3504005910 tx=0x7f350400b670 comp rx=0 tx=0).stop 2024-04-25T14:39:37.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:37.033+0000 7f3518c00700 1 -- 172.21.15.53:0/4196393412 shutdown_connections 2024-04-25T14:39:37.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:37.033+0000 7f3518c00700 1 --2- 172.21.15.53:0/4196393412 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f34f80a3d50 0x7f34f80a6210 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:37.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:37.033+0000 7f3518c00700 1 --2- 172.21.15.53:0/4196393412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3514108a80 0x7f351419dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:37.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:37.033+0000 7f3518c00700 1 --2- 172.21.15.53:0/4196393412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3514102a90 0x7f3514199310 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:37.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:37.033+0000 7f3518c00700 1 --2- 172.21.15.53:0/4196393412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3514103450 0x7f3514199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:37.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:37.033+0000 7f3518c00700 1 -- 172.21.15.53:0/4196393412 >> 172.21.15.53:0/4196393412 conn(0x7f3514079bf0 msgr2=0x7f35141018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:37.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:37.034+0000 7f3518c00700 1 -- 172.21.15.53:0/4196393412 shutdown_connections 2024-04-25T14:39:37.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:37.034+0000 7f3518c00700 1 -- 172.21.15.53:0/4196393412 wait complete. 2024-04-25T14:39:37.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:37 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/4196393412' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T14:39:37.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:37 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/4196393412' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T14:39:37.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:37 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/4196393412' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T14:39:37.955 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266735 got 227633266740 for osd.7 2024-04-25T14:39:37.955 DEBUG:teuthology.parallel:result is None 2024-04-25T14:39:37.955 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T14:39:37.955 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T14:39:38.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:38 smithi053 ceph-mon[27773]: pgmap v361: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T14:39:38.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:38 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:39:38.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:38 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:39:38.411 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:39:38.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:38 smithi183 ceph-mon[32584]: pgmap v361: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T14:39:38.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:38 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:39:38.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:38 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:39:38.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:38 smithi150 ceph-mon[32504]: pgmap v361: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T14:39:38.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:38 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:39:38.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:38 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:39:40.390 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:40 smithi053 ceph-mon[27773]: pgmap v362: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T14:39:40.390 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:40 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:39:40.390 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:40 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:39:40.390 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:40 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:39:40.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:40 smithi183 ceph-mon[32584]: pgmap v362: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T14:39:40.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:40 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:39:40.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:40 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:39:40.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:40 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:39:40.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:40 smithi150 ceph-mon[32504]: pgmap v362: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T14:39:40.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:40 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:39:40.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:40 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:39:40.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:40 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' 2024-04-25T14:39:40.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:40.749+0000 7f15c9400700 1 -- 172.21.15.53:0/2291614532 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15c4075590 msgr2=0x7f15c410ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:40.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:40.749+0000 7f15c9400700 1 --2- 172.21.15.53:0/2291614532 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15c4075590 0x7f15c410ef00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f15b0009990 tx=0x7f15b0009ca0 comp rx=0 tx=0).stop 2024-04-25T14:39:40.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:40.749+0000 7f15c9400700 1 -- 172.21.15.53:0/2291614532 shutdown_connections 2024-04-25T14:39:40.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:40.749+0000 7f15c9400700 1 --2- 172.21.15.53:0/2291614532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15c410f440 0x7f15c4111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:40.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:40.749+0000 7f15c9400700 1 --2- 172.21.15.53:0/2291614532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15c4074c70 0x7f15c4075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:40.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:40.749+0000 7f15c9400700 1 --2- 172.21.15.53:0/2291614532 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15c4075590 0x7f15c410ef00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:40.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:40.749+0000 7f15c9400700 1 -- 172.21.15.53:0/2291614532 >> 172.21.15.53:0/2291614532 conn(0x7f15c4079ba0 msgr2=0x7f15c4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:40.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:40.749+0000 7f15c9400700 1 -- 172.21.15.53:0/2291614532 shutdown_connections 2024-04-25T14:39:40.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:40.749+0000 7f15c9400700 1 -- 172.21.15.53:0/2291614532 wait complete. 2024-04-25T14:39:40.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:40.751+0000 7f15c9400700 1 Processor -- start 2024-04-25T14:39:40.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:40.751+0000 7f15c9400700 1 -- start start 2024-04-25T14:39:40.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:40.752+0000 7f15c9400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15c4074c70 0x7f15c419b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:40.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:40.752+0000 7f15c9400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15c4075590 0x7f15c419b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:40.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:40.752+0000 7f15c9400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15c410f440 0x7f15c419fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:40.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:40.752+0000 7f15c9400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f15c4114e50 con 0x7f15c4074c70 2024-04-25T14:39:40.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:40.752+0000 7f15c9400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f15c4114cd0 con 0x7f15c410f440 2024-04-25T14:39:40.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:40.752+0000 7f15c9400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f15c4114fd0 con 0x7f15c4075590 2024-04-25T14:39:40.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:40.752+0000 7f15c2800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15c4074c70 0x7f15c419b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:40.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:40.752+0000 7f15c1e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15c4075590 0x7f15c419b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:40.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:40.752+0000 7f15c2800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15c4074c70 0x7f15c419b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48890/0 (socket says 172.21.15.53:48890) 2024-04-25T14:39:40.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:40.752+0000 7f15c2800700 1 -- 172.21.15.53:0/3927574405 learned_addr learned my addr 172.21.15.53:0/3927574405 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:40.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:40.752+0000 7f15c3200700 1 --2- 172.21.15.53:0/3927574405 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15c410f440 0x7f15c419fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:40.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:40.753+0000 7f15c1e00700 1 -- 172.21.15.53:0/3927574405 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15c410f440 msgr2=0x7f15c419fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:40.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:40.753+0000 7f15c1e00700 1 --2- 172.21.15.53:0/3927574405 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15c410f440 0x7f15c419fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:40.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:40.753+0000 7f15c1e00700 1 -- 172.21.15.53:0/3927574405 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15c4074c70 msgr2=0x7f15c419b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:40.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:40.753+0000 7f15c1e00700 1 --2- 172.21.15.53:0/3927574405 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15c4074c70 0x7f15c419b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:40.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:40.753+0000 7f15c1e00700 1 -- 172.21.15.53:0/3927574405 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15c41a0310 con 0x7f15c4075590 2024-04-25T14:39:40.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:40.753+0000 7f15c2800700 1 --2- 172.21.15.53:0/3927574405 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15c4074c70 0x7f15c419b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:39:40.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:40.753+0000 7f15c1e00700 1 --2- 172.21.15.53:0/3927574405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15c4075590 0x7f15c419b840 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f15b00098e0 tx=0x7f15b00057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:40.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:40.753+0000 7f15bb400700 1 -- 172.21.15.53:0/3927574405 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b0027070 con 0x7f15c4075590 2024-04-25T14:39:40.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:40.753+0000 7f15bb400700 1 -- 172.21.15.53:0/3927574405 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15b0022470 con 0x7f15c4075590 2024-04-25T14:39:40.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:40.753+0000 7f15bb400700 1 -- 172.21.15.53:0/3927574405 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b0017650 con 0x7f15c4075590 2024-04-25T14:39:40.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:40.753+0000 7f15c9400700 1 -- 172.21.15.53:0/3927574405 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15c41a05a0 con 0x7f15c4075590 2024-04-25T14:39:40.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:40.754+0000 7f15c9400700 1 -- 172.21.15.53:0/3927574405 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15c41a7e80 con 0x7f15c4075590 2024-04-25T14:39:40.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:40.755+0000 7f15c9400700 1 -- 172.21.15.53:0/3927574405 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15c4069730 con 0x7f15c4075590 2024-04-25T14:39:40.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:40.759+0000 7f15bb400700 1 -- 172.21.15.53:0/3927574405 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f15b0009d10 con 0x7f15c4075590 2024-04-25T14:39:40.761 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:40.760+0000 7f15bb400700 1 --2- 172.21.15.53:0/3927574405 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f15a80a3d50 0x7f15a80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:40.761 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:40.761+0000 7f15bb400700 1 -- 172.21.15.53:0/3927574405 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f15b00d1300 con 0x7f15c4075590 2024-04-25T14:39:40.761 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:40.761+0000 7f15c2800700 1 --2- 172.21.15.53:0/3927574405 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f15a80a3d50 0x7f15a80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:40.761 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:40.761+0000 7f15bb400700 1 -- 172.21.15.53:0/3927574405 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f15b0013070 con 0x7f15c4075590 2024-04-25T14:39:40.761 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:40.762+0000 7f15c2800700 1 --2- 172.21.15.53:0/3927574405 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f15a80a3d50 0x7f15a80a6210 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f15b4011630 tx=0x7f15b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:40.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:40.992+0000 7f15c9400700 1 -- 172.21.15.53:0/3927574405 --> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f15c419c660 con 0x7f15a80a3d50 2024-04-25T14:39:41.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:41.025+0000 7f15bb400700 1 -- 172.21.15.53:0/3927574405 <== mgr.14152 v2:172.21.15.53:6800/4154933108 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258717 (secure 0 0 0) 0x7f15c419c660 con 0x7f15a80a3d50 2024-04-25T14:39:41.035 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:39:41.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:41.032+0000 7f15c9400700 1 -- 172.21.15.53:0/3927574405 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f15a80a3d50 msgr2=0x7f15a80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:41.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:41.032+0000 7f15c9400700 1 --2- 172.21.15.53:0/3927574405 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f15a80a3d50 0x7f15a80a6210 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f15b4011630 tx=0x7f15b4009260 comp rx=0 tx=0).stop 2024-04-25T14:39:41.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:41.032+0000 7f15c9400700 1 -- 172.21.15.53:0/3927574405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15c4075590 msgr2=0x7f15c419b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:41.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:41.032+0000 7f15c9400700 1 --2- 172.21.15.53:0/3927574405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15c4075590 0x7f15c419b840 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f15b00098e0 tx=0x7f15b00057c0 comp rx=0 tx=0).stop 2024-04-25T14:39:41.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:41.032+0000 7f15c9400700 1 -- 172.21.15.53:0/3927574405 shutdown_connections 2024-04-25T14:39:41.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:41.032+0000 7f15c9400700 1 --2- 172.21.15.53:0/3927574405 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f15a80a3d50 0x7f15a80a6210 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:41.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:41.032+0000 7f15c9400700 1 --2- 172.21.15.53:0/3927574405 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15c410f440 0x7f15c419fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:41.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:41.032+0000 7f15c9400700 1 --2- 172.21.15.53:0/3927574405 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15c4074c70 0x7f15c419b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:41.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:41.032+0000 7f15c9400700 1 --2- 172.21.15.53:0/3927574405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15c4075590 0x7f15c419b840 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:41.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:41.032+0000 7f15c9400700 1 -- 172.21.15.53:0/3927574405 >> 172.21.15.53:0/3927574405 conn(0x7f15c4079ba0 msgr2=0x7f15c40ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:41.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:41.032+0000 7f15c9400700 1 -- 172.21.15.53:0/3927574405 shutdown_connections 2024-04-25T14:39:41.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:41.033+0000 7f15c9400700 1 -- 172.21.15.53:0/3927574405 wait complete. 2024-04-25T14:39:41.038 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-04-25T14:39:41.903 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-04-25T14:39:40.341382+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656890,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":662,"num_read_kb":453,"num_write":344,"num_write_kb":3051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1181771,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":277,"ondisk_log_size":277,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375356,"kb_used_data":13644,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566468,"statfs":{"total":1151940427776,"available":1151556063232,"internally_reserved":0,"allocated":13971456,"data_stored":7599498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19137,"internal_metadata":370193727},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005133"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.978785+0000","last_change":"2024-04-25T14:38:10.435535+0000","last_active":"2024-04-25T14:39:15.978785+0000","last_peered":"2024-04-25T14:39:15.978785+0000","last_clean":"2024-04-25T14:39:15.978785+0000","last_became_active":"2024-04-25T14:38:10.435205+0000","last_became_peered":"2024-04-25T14:38:10.435205+0000","last_unstale":"2024-04-25T14:39:15.978785+0000","last_undegraded":"2024-04-25T14:39:15.978785+0000","last_fullsized":"2024-04-25T14:39:15.978785+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:12.301771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.991499+0000","last_change":"2024-04-25T14:38:10.428216+0000","last_active":"2024-04-25T14:39:15.991499+0000","last_peered":"2024-04-25T14:39:15.991499+0000","last_clean":"2024-04-25T14:39:15.991499+0000","last_became_active":"2024-04-25T14:38:10.427419+0000","last_became_peered":"2024-04-25T14:38:10.427419+0000","last_unstale":"2024-04-25T14:39:15.991499+0000","last_undegraded":"2024-04-25T14:39:15.991499+0000","last_fullsized":"2024-04-25T14:39:15.991499+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:42.580424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.991243+0000","last_change":"2024-04-25T14:38:10.427827+0000","last_active":"2024-04-25T14:39:15.991243+0000","last_peered":"2024-04-25T14:39:15.991243+0000","last_clean":"2024-04-25T14:39:15.991243+0000","last_became_active":"2024-04-25T14:38:10.427435+0000","last_became_peered":"2024-04-25T14:38:10.427435+0000","last_unstale":"2024-04-25T14:39:15.991243+0000","last_undegraded":"2024-04-25T14:39:15.991243+0000","last_fullsized":"2024-04-25T14:39:15.991243+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:15.539399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.970735+0000","last_change":"2024-04-25T14:38:10.423751+0000","last_active":"2024-04-25T14:39:15.970735+0000","last_peered":"2024-04-25T14:39:15.970735+0000","last_clean":"2024-04-25T14:39:15.970735+0000","last_became_active":"2024-04-25T14:38:10.423409+0000","last_became_peered":"2024-04-25T14:38:10.423409+0000","last_unstale":"2024-04-25T14:39:15.970735+0000","last_undegraded":"2024-04-25T14:39:15.970735+0000","last_fullsized":"2024-04-25T14:39:15.970735+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:45.403788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.976066+0000","last_change":"2024-04-25T14:38:10.430000+0000","last_active":"2024-04-25T14:39:15.976066+0000","last_peered":"2024-04-25T14:39:15.976066+0000","last_clean":"2024-04-25T14:39:15.976066+0000","last_became_active":"2024-04-25T14:38:10.429684+0000","last_became_peered":"2024-04-25T14:38:10.429684+0000","last_unstale":"2024-04-25T14:39:15.976066+0000","last_undegraded":"2024-04-25T14:39:15.976066+0000","last_fullsized":"2024-04-25T14:39:15.976066+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:18.421970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.984180+0000","last_change":"2024-04-25T14:39:15.984180+0000","last_active":"2024-04-25T14:39:15.984180+0000","last_peered":"2024-04-25T14:39:15.984180+0000","last_clean":"2024-04-25T14:39:15.984180+0000","last_became_active":"2024-04-25T14:39:15.983821+0000","last_became_peered":"2024-04-25T14:39:15.983821+0000","last_unstale":"2024-04-25T14:39:15.984180+0000","last_undegraded":"2024-04-25T14:39:15.984180+0000","last_fullsized":"2024-04-25T14:39:15.984180+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:54:47.617713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.972805+0000","last_change":"2024-04-25T14:38:10.420022+0000","last_active":"2024-04-25T14:39:15.972805+0000","last_peered":"2024-04-25T14:39:15.972805+0000","last_clean":"2024-04-25T14:39:15.972805+0000","last_became_active":"2024-04-25T14:38:10.419673+0000","last_became_peered":"2024-04-25T14:38:10.419673+0000","last_unstale":"2024-04-25T14:39:15.972805+0000","last_undegraded":"2024-04-25T14:39:15.972805+0000","last_fullsized":"2024-04-25T14:39:15.972805+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:09.177814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.988374+0000","last_change":"2024-04-25T14:39:15.988374+0000","last_active":"2024-04-25T14:39:15.988374+0000","last_peered":"2024-04-25T14:39:15.988374+0000","last_clean":"2024-04-25T14:39:15.988374+0000","last_became_active":"2024-04-25T14:39:15.983741+0000","last_became_peered":"2024-04-25T14:39:15.983741+0000","last_unstale":"2024-04-25T14:39:15.988374+0000","last_undegraded":"2024-04-25T14:39:15.988374+0000","last_fullsized":"2024-04-25T14:39:15.988374+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:15.523804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,10],"acting":[3,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:16.535006+0000","last_change":"2024-04-25T14:38:10.424086+0000","last_active":"2024-04-25T14:39:16.535006+0000","last_peered":"2024-04-25T14:39:16.535006+0000","last_clean":"2024-04-25T14:39:16.535006+0000","last_became_active":"2024-04-25T14:38:10.423769+0000","last_became_peered":"2024-04-25T14:38:10.423769+0000","last_unstale":"2024-04-25T14:39:16.535006+0000","last_undegraded":"2024-04-25T14:39:16.535006+0000","last_fullsized":"2024-04-25T14:39:16.535006+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:19:22.736053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.973107+0000","last_change":"2024-04-25T14:38:10.427049+0000","last_active":"2024-04-25T14:39:15.973107+0000","last_peered":"2024-04-25T14:39:15.973107+0000","last_clean":"2024-04-25T14:39:15.973107+0000","last_became_active":"2024-04-25T14:38:10.426729+0000","last_became_peered":"2024-04-25T14:38:10.426729+0000","last_unstale":"2024-04-25T14:39:15.973107+0000","last_undegraded":"2024-04-25T14:39:15.973107+0000","last_fullsized":"2024-04-25T14:39:15.973107+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:00.849650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.973395+0000","last_change":"2024-04-25T14:38:10.424976+0000","last_active":"2024-04-25T14:39:15.973395+0000","last_peered":"2024-04-25T14:39:15.973395+0000","last_clean":"2024-04-25T14:39:15.973395+0000","last_became_active":"2024-04-25T14:38:10.424585+0000","last_became_peered":"2024-04-25T14:38:10.424585+0000","last_unstale":"2024-04-25T14:39:15.973395+0000","last_undegraded":"2024-04-25T14:39:15.973395+0000","last_fullsized":"2024-04-25T14:39:15.973395+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:54.639471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.970561+0000","last_change":"2024-04-25T14:38:10.418022+0000","last_active":"2024-04-25T14:39:15.970561+0000","last_peered":"2024-04-25T14:39:15.970561+0000","last_clean":"2024-04-25T14:39:15.970561+0000","last_became_active":"2024-04-25T14:38:10.417735+0000","last_became_peered":"2024-04-25T14:38:10.417735+0000","last_unstale":"2024-04-25T14:39:15.970561+0000","last_undegraded":"2024-04-25T14:39:15.970561+0000","last_fullsized":"2024-04-25T14:39:15.970561+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:48.785521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.974740+0000","last_change":"2024-04-25T14:38:10.423464+0000","last_active":"2024-04-25T14:39:15.974740+0000","last_peered":"2024-04-25T14:39:15.974740+0000","last_clean":"2024-04-25T14:39:15.974740+0000","last_became_active":"2024-04-25T14:38:10.423105+0000","last_became_peered":"2024-04-25T14:38:10.423105+0000","last_unstale":"2024-04-25T14:39:15.974740+0000","last_undegraded":"2024-04-25T14:39:15.974740+0000","last_fullsized":"2024-04-25T14:39:15.974740+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:26.568198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.989428+0000","last_change":"2024-04-25T14:38:10.426870+0000","last_active":"2024-04-25T14:39:15.989428+0000","last_peered":"2024-04-25T14:39:15.989428+0000","last_clean":"2024-04-25T14:39:15.989428+0000","last_became_active":"2024-04-25T14:38:10.426506+0000","last_became_peered":"2024-04-25T14:38:10.426506+0000","last_unstale":"2024-04-25T14:39:15.989428+0000","last_undegraded":"2024-04-25T14:39:15.989428+0000","last_fullsized":"2024-04-25T14:39:15.989428+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:16:49.539221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.988922+0000","last_change":"2024-04-25T14:38:10.434618+0000","last_active":"2024-04-25T14:39:15.988922+0000","last_peered":"2024-04-25T14:39:15.988922+0000","last_clean":"2024-04-25T14:39:15.988922+0000","last_became_active":"2024-04-25T14:38:10.430661+0000","last_became_peered":"2024-04-25T14:38:10.430661+0000","last_unstale":"2024-04-25T14:39:15.988922+0000","last_undegraded":"2024-04-25T14:39:15.988922+0000","last_fullsized":"2024-04-25T14:39:15.988922+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:44.548788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:16.534468+0000","last_change":"2024-04-25T14:38:10.429898+0000","last_active":"2024-04-25T14:39:16.534468+0000","last_peered":"2024-04-25T14:39:16.534468+0000","last_clean":"2024-04-25T14:39:16.534468+0000","last_became_active":"2024-04-25T14:38:10.429557+0000","last_became_peered":"2024-04-25T14:38:10.429557+0000","last_unstale":"2024-04-25T14:39:16.534468+0000","last_undegraded":"2024-04-25T14:39:16.534468+0000","last_fullsized":"2024-04-25T14:39:16.534468+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:54.258526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.978841+0000","last_change":"2024-04-25T14:38:10.435134+0000","last_active":"2024-04-25T14:39:15.978841+0000","last_peered":"2024-04-25T14:39:15.978841+0000","last_clean":"2024-04-25T14:39:15.978841+0000","last_became_active":"2024-04-25T14:38:10.434775+0000","last_became_peered":"2024-04-25T14:38:10.434775+0000","last_unstale":"2024-04-25T14:39:15.978841+0000","last_undegraded":"2024-04-25T14:39:15.978841+0000","last_fullsized":"2024-04-25T14:39:15.978841+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:49.827881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.979305+0000","last_change":"2024-04-25T14:38:10.424929+0000","last_active":"2024-04-25T14:39:15.979305+0000","last_peered":"2024-04-25T14:39:15.979305+0000","last_clean":"2024-04-25T14:39:15.979305+0000","last_became_active":"2024-04-25T14:38:10.424399+0000","last_became_peered":"2024-04-25T14:38:10.424399+0000","last_unstale":"2024-04-25T14:39:15.979305+0000","last_undegraded":"2024-04-25T14:39:15.979305+0000","last_fullsized":"2024-04-25T14:39:15.979305+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:08.834176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.972902+0000","last_change":"2024-04-25T14:38:10.444004+0000","last_active":"2024-04-25T14:39:15.972902+0000","last_peered":"2024-04-25T14:39:15.972902+0000","last_clean":"2024-04-25T14:39:15.972902+0000","last_became_active":"2024-04-25T14:38:10.443669+0000","last_became_peered":"2024-04-25T14:38:10.443669+0000","last_unstale":"2024-04-25T14:39:15.972902+0000","last_undegraded":"2024-04-25T14:39:15.972902+0000","last_fullsized":"2024-04-25T14:39:15.972902+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:02.302057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.980018+0000","last_change":"2024-04-25T14:38:09.545010+0000","last_active":"2024-04-25T14:39:15.980018+0000","last_peered":"2024-04-25T14:39:15.980018+0000","last_clean":"2024-04-25T14:39:15.980018+0000","last_became_active":"2024-04-25T14:38:09.544622+0000","last_became_peered":"2024-04-25T14:38:09.544622+0000","last_unstale":"2024-04-25T14:39:15.980018+0000","last_undegraded":"2024-04-25T14:39:15.980018+0000","last_fullsized":"2024-04-25T14:39:15.980018+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:51.200242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.996660+0000","last_change":"2024-04-25T14:39:15.993912+0000","last_active":"2024-04-25T14:39:15.996660+0000","last_peered":"2024-04-25T14:39:15.996660+0000","last_clean":"2024-04-25T14:39:15.996660+0000","last_became_active":"2024-04-25T14:39:15.992511+0000","last_became_peered":"2024-04-25T14:39:15.992511+0000","last_unstale":"2024-04-25T14:39:15.996660+0000","last_undegraded":"2024-04-25T14:39:15.996660+0000","last_fullsized":"2024-04-25T14:39:15.996660+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:12:57.639396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,6],"acting":[9,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.988836+0000","last_change":"2024-04-25T14:38:09.555825+0000","last_active":"2024-04-25T14:39:15.988836+0000","last_peered":"2024-04-25T14:39:15.988836+0000","last_clean":"2024-04-25T14:39:15.988836+0000","last_became_active":"2024-04-25T14:38:09.555436+0000","last_became_peered":"2024-04-25T14:38:09.555436+0000","last_unstale":"2024-04-25T14:39:15.988836+0000","last_undegraded":"2024-04-25T14:39:15.988836+0000","last_fullsized":"2024-04-25T14:39:15.988836+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:45.097583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.992495+0000","last_change":"2024-04-25T14:38:10.432254+0000","last_active":"2024-04-25T14:39:15.992495+0000","last_peered":"2024-04-25T14:39:15.992495+0000","last_clean":"2024-04-25T14:39:15.992495+0000","last_became_active":"2024-04-25T14:38:10.431835+0000","last_became_peered":"2024-04-25T14:38:10.431835+0000","last_unstale":"2024-04-25T14:39:15.992495+0000","last_undegraded":"2024-04-25T14:39:15.992495+0000","last_fullsized":"2024-04-25T14:39:15.992495+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:23.089447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.976577+0000","last_change":"2024-04-25T14:38:09.550254+0000","last_active":"2024-04-25T14:39:15.976577+0000","last_peered":"2024-04-25T14:39:15.976577+0000","last_clean":"2024-04-25T14:39:15.976577+0000","last_became_active":"2024-04-25T14:38:09.549958+0000","last_became_peered":"2024-04-25T14:38:09.549958+0000","last_unstale":"2024-04-25T14:39:15.976577+0000","last_undegraded":"2024-04-25T14:39:15.976577+0000","last_fullsized":"2024-04-25T14:39:15.976577+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:56.161427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:16.534628+0000","last_change":"2024-04-25T14:38:10.425648+0000","last_active":"2024-04-25T14:39:16.534628+0000","last_peered":"2024-04-25T14:39:16.534628+0000","last_clean":"2024-04-25T14:39:16.534628+0000","last_became_active":"2024-04-25T14:38:10.425009+0000","last_became_peered":"2024-04-25T14:38:10.425009+0000","last_unstale":"2024-04-25T14:39:16.534628+0000","last_undegraded":"2024-04-25T14:39:16.534628+0000","last_fullsized":"2024-04-25T14:39:16.534628+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:46.696610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.976289+0000","last_change":"2024-04-25T14:38:09.544481+0000","last_active":"2024-04-25T14:39:15.976289+0000","last_peered":"2024-04-25T14:39:15.976289+0000","last_clean":"2024-04-25T14:39:15.976289+0000","last_became_active":"2024-04-25T14:38:09.544147+0000","last_became_peered":"2024-04-25T14:38:09.544147+0000","last_unstale":"2024-04-25T14:39:15.976289+0000","last_undegraded":"2024-04-25T14:39:15.976289+0000","last_fullsized":"2024-04-25T14:39:15.976289+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:35:22.971028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.970180+0000","last_change":"2024-04-25T14:38:10.426851+0000","last_active":"2024-04-25T14:39:15.970180+0000","last_peered":"2024-04-25T14:39:15.970180+0000","last_clean":"2024-04-25T14:39:15.970180+0000","last_became_active":"2024-04-25T14:38:10.426475+0000","last_became_peered":"2024-04-25T14:38:10.426475+0000","last_unstale":"2024-04-25T14:39:15.970180+0000","last_undegraded":"2024-04-25T14:39:15.970180+0000","last_fullsized":"2024-04-25T14:39:15.970180+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:28.213318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":121,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:35.003970+0000","last_change":"2024-04-25T14:39:15.979586+0000","last_active":"2024-04-25T14:39:35.003970+0000","last_peered":"2024-04-25T14:39:35.003970+0000","last_clean":"2024-04-25T14:39:35.003970+0000","last_became_active":"2024-04-25T14:39:15.974397+0000","last_became_peered":"2024-04-25T14:39:15.974397+0000","last_unstale":"2024-04-25T14:39:35.003970+0000","last_undegraded":"2024-04-25T14:39:35.003970+0000","last_fullsized":"2024-04-25T14:39:35.003970+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:02.152202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.975974+0000","last_change":"2024-04-25T14:38:10.425424+0000","last_active":"2024-04-25T14:39:15.975974+0000","last_peered":"2024-04-25T14:39:15.975974+0000","last_clean":"2024-04-25T14:39:15.975974+0000","last_became_active":"2024-04-25T14:38:10.425101+0000","last_became_peered":"2024-04-25T14:38:10.425101+0000","last_unstale":"2024-04-25T14:39:15.975974+0000","last_undegraded":"2024-04-25T14:39:15.975974+0000","last_fullsized":"2024-04-25T14:39:15.975974+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:52:29.871113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.973226+0000","last_change":"2024-04-25T14:38:09.552287+0000","last_active":"2024-04-25T14:39:15.973226+0000","last_peered":"2024-04-25T14:39:15.973226+0000","last_clean":"2024-04-25T14:39:15.973226+0000","last_became_active":"2024-04-25T14:38:09.551957+0000","last_became_peered":"2024-04-25T14:38:09.551957+0000","last_unstale":"2024-04-25T14:39:15.973226+0000","last_undegraded":"2024-04-25T14:39:15.973226+0000","last_fullsized":"2024-04-25T14:39:15.973226+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:22.586868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.990838+0000","last_change":"2024-04-25T14:39:15.990838+0000","last_active":"2024-04-25T14:39:15.990838+0000","last_peered":"2024-04-25T14:39:15.990838+0000","last_clean":"2024-04-25T14:39:15.990838+0000","last_became_active":"2024-04-25T14:39:15.990370+0000","last_became_peered":"2024-04-25T14:39:15.990370+0000","last_unstale":"2024-04-25T14:39:15.990838+0000","last_undegraded":"2024-04-25T14:39:15.990838+0000","last_fullsized":"2024-04-25T14:39:15.990838+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:50.676958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.975073+0000","last_change":"2024-04-25T14:38:09.552376+0000","last_active":"2024-04-25T14:39:15.975073+0000","last_peered":"2024-04-25T14:39:15.975073+0000","last_clean":"2024-04-25T14:39:15.975073+0000","last_became_active":"2024-04-25T14:38:09.548614+0000","last_became_peered":"2024-04-25T14:38:09.548614+0000","last_unstale":"2024-04-25T14:39:15.975073+0000","last_undegraded":"2024-04-25T14:39:15.975073+0000","last_fullsized":"2024-04-25T14:39:15.975073+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:27:27.133106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.982412+0000","last_change":"2024-04-25T14:38:10.434611+0000","last_active":"2024-04-25T14:39:15.982412+0000","last_peered":"2024-04-25T14:39:15.982412+0000","last_clean":"2024-04-25T14:39:15.982412+0000","last_became_active":"2024-04-25T14:38:10.434280+0000","last_became_peered":"2024-04-25T14:38:10.434280+0000","last_unstale":"2024-04-25T14:39:15.982412+0000","last_undegraded":"2024-04-25T14:39:15.982412+0000","last_fullsized":"2024-04-25T14:39:15.982412+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:12:32.275663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.969820+0000","last_change":"2024-04-25T14:38:09.539743+0000","last_active":"2024-04-25T14:39:15.969820+0000","last_peered":"2024-04-25T14:39:15.969820+0000","last_clean":"2024-04-25T14:39:15.969820+0000","last_became_active":"2024-04-25T14:38:09.539272+0000","last_became_peered":"2024-04-25T14:38:09.539272+0000","last_unstale":"2024-04-25T14:39:15.969820+0000","last_undegraded":"2024-04-25T14:39:15.969820+0000","last_fullsized":"2024-04-25T14:39:15.969820+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:29:23.159901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.977082+0000","last_change":"2024-04-25T14:38:10.444899+0000","last_active":"2024-04-25T14:39:15.977082+0000","last_peered":"2024-04-25T14:39:15.977082+0000","last_clean":"2024-04-25T14:39:15.977082+0000","last_became_active":"2024-04-25T14:38:10.444383+0000","last_became_peered":"2024-04-25T14:38:10.444383+0000","last_unstale":"2024-04-25T14:39:15.977082+0000","last_undegraded":"2024-04-25T14:39:15.977082+0000","last_fullsized":"2024-04-25T14:39:15.977082+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:21.351436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":107,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:39.008909+0000","last_change":"2024-04-25T14:38:09.550181+0000","last_active":"2024-04-25T14:39:39.008909+0000","last_peered":"2024-04-25T14:39:39.008909+0000","last_clean":"2024-04-25T14:39:39.008909+0000","last_became_active":"2024-04-25T14:38:09.549824+0000","last_became_peered":"2024-04-25T14:38:09.549824+0000","last_unstale":"2024-04-25T14:39:39.008909+0000","last_undegraded":"2024-04-25T14:39:39.008909+0000","last_fullsized":"2024-04-25T14:39:39.008909+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:59.220917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":85,"num_read_kb":85,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:16.534903+0000","last_change":"2024-04-25T14:38:10.421339+0000","last_active":"2024-04-25T14:39:16.534903+0000","last_peered":"2024-04-25T14:39:16.534903+0000","last_clean":"2024-04-25T14:39:16.534903+0000","last_became_active":"2024-04-25T14:38:10.421005+0000","last_became_peered":"2024-04-25T14:38:10.421005+0000","last_unstale":"2024-04-25T14:39:16.534903+0000","last_undegraded":"2024-04-25T14:39:16.534903+0000","last_fullsized":"2024-04-25T14:39:16.534903+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:51.949054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.973211+0000","last_change":"2024-04-25T14:38:09.548245+0000","last_active":"2024-04-25T14:39:15.973211+0000","last_peered":"2024-04-25T14:39:15.973211+0000","last_clean":"2024-04-25T14:39:15.973211+0000","last_became_active":"2024-04-25T14:38:09.547905+0000","last_became_peered":"2024-04-25T14:38:09.547905+0000","last_unstale":"2024-04-25T14:39:15.973211+0000","last_undegraded":"2024-04-25T14:39:15.973211+0000","last_fullsized":"2024-04-25T14:39:15.973211+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:56.798594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.992319+0000","last_change":"2024-04-25T14:38:10.431427+0000","last_active":"2024-04-25T14:39:15.992319+0000","last_peered":"2024-04-25T14:39:15.992319+0000","last_clean":"2024-04-25T14:39:15.992319+0000","last_became_active":"2024-04-25T14:38:10.428759+0000","last_became_peered":"2024-04-25T14:38:10.428759+0000","last_unstale":"2024-04-25T14:39:15.992319+0000","last_undegraded":"2024-04-25T14:39:15.992319+0000","last_fullsized":"2024-04-25T14:39:15.992319+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:15:27.341445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.970306+0000","last_change":"2024-04-25T14:38:09.556839+0000","last_active":"2024-04-25T14:39:15.970306+0000","last_peered":"2024-04-25T14:39:15.970306+0000","last_clean":"2024-04-25T14:39:15.970306+0000","last_became_active":"2024-04-25T14:38:09.556443+0000","last_became_peered":"2024-04-25T14:38:09.556443+0000","last_unstale":"2024-04-25T14:39:15.970306+0000","last_undegraded":"2024-04-25T14:39:15.970306+0000","last_fullsized":"2024-04-25T14:39:15.970306+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:46.621883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:16.534780+0000","last_change":"2024-04-25T14:38:10.414507+0000","last_active":"2024-04-25T14:39:16.534780+0000","last_peered":"2024-04-25T14:39:16.534780+0000","last_clean":"2024-04-25T14:39:16.534780+0000","last_became_active":"2024-04-25T14:38:10.414154+0000","last_became_peered":"2024-04-25T14:38:10.414154+0000","last_unstale":"2024-04-25T14:39:16.534780+0000","last_undegraded":"2024-04-25T14:39:16.534780+0000","last_fullsized":"2024-04-25T14:39:16.534780+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:12.208774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'28","reported_seq":225,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:40.010403+0000","last_change":"2024-04-25T14:38:09.549239+0000","last_active":"2024-04-25T14:39:40.010403+0000","last_peered":"2024-04-25T14:39:40.010403+0000","last_clean":"2024-04-25T14:39:40.010403+0000","last_became_active":"2024-04-25T14:38:09.548888+0000","last_became_peered":"2024-04-25T14:38:09.548888+0000","last_unstale":"2024-04-25T14:39:40.010403+0000","last_undegraded":"2024-04-25T14:39:40.010403+0000","last_fullsized":"2024-04-25T14:39:40.010403+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:36:46.536908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61896,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":178,"num_read_kb":149,"num_write":29,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.976353+0000","last_change":"2024-04-25T14:38:10.417510+0000","last_active":"2024-04-25T14:39:15.976353+0000","last_peered":"2024-04-25T14:39:15.976353+0000","last_clean":"2024-04-25T14:39:15.976353+0000","last_became_active":"2024-04-25T14:38:10.417079+0000","last_became_peered":"2024-04-25T14:38:10.417079+0000","last_unstale":"2024-04-25T14:39:15.976353+0000","last_undegraded":"2024-04-25T14:39:15.976353+0000","last_fullsized":"2024-04-25T14:39:15.976353+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:35.213199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.972736+0000","last_change":"2024-04-25T14:38:09.563650+0000","last_active":"2024-04-25T14:39:15.972736+0000","last_peered":"2024-04-25T14:39:15.972736+0000","last_clean":"2024-04-25T14:39:15.972736+0000","last_became_active":"2024-04-25T14:38:09.563347+0000","last_became_peered":"2024-04-25T14:38:09.563347+0000","last_unstale":"2024-04-25T14:39:15.972736+0000","last_undegraded":"2024-04-25T14:39:15.972736+0000","last_fullsized":"2024-04-25T14:39:15.972736+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:16.793191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.974525+0000","last_change":"2024-04-25T14:38:10.447504+0000","last_active":"2024-04-25T14:39:15.974525+0000","last_peered":"2024-04-25T14:39:15.974525+0000","last_clean":"2024-04-25T14:39:15.974525+0000","last_became_active":"2024-04-25T14:38:10.447157+0000","last_became_peered":"2024-04-25T14:38:10.447157+0000","last_unstale":"2024-04-25T14:39:15.974525+0000","last_undegraded":"2024-04-25T14:39:15.974525+0000","last_fullsized":"2024-04-25T14:39:15.974525+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:38.940320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.969971+0000","last_change":"2024-04-25T14:38:09.545214+0000","last_active":"2024-04-25T14:39:15.969971+0000","last_peered":"2024-04-25T14:39:15.969971+0000","last_clean":"2024-04-25T14:39:15.969971+0000","last_became_active":"2024-04-25T14:38:09.544903+0000","last_became_peered":"2024-04-25T14:38:09.544903+0000","last_unstale":"2024-04-25T14:39:15.969971+0000","last_undegraded":"2024-04-25T14:39:15.969971+0000","last_fullsized":"2024-04-25T14:39:15.969971+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:58:15.817036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.973508+0000","last_change":"2024-04-25T14:38:10.432017+0000","last_active":"2024-04-25T14:39:15.973508+0000","last_peered":"2024-04-25T14:39:15.973508+0000","last_clean":"2024-04-25T14:39:15.973508+0000","last_became_active":"2024-04-25T14:38:10.431596+0000","last_became_peered":"2024-04-25T14:38:10.431596+0000","last_unstale":"2024-04-25T14:39:15.973508+0000","last_undegraded":"2024-04-25T14:39:15.973508+0000","last_fullsized":"2024-04-25T14:39:15.973508+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:30.653642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'4","reported_seq":110,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:37.822135+0000","last_change":"2024-04-25T14:38:09.545477+0000","last_active":"2024-04-25T14:39:37.822135+0000","last_peered":"2024-04-25T14:39:37.822135+0000","last_clean":"2024-04-25T14:39:37.822135+0000","last_became_active":"2024-04-25T14:38:09.545093+0000","last_became_peered":"2024-04-25T14:38:09.545093+0000","last_unstale":"2024-04-25T14:39:37.822135+0000","last_undegraded":"2024-04-25T14:39:37.822135+0000","last_fullsized":"2024-04-25T14:39:37.822135+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:56.589231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":88,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.973286+0000","last_change":"2024-04-25T14:38:10.418914+0000","last_active":"2024-04-25T14:39:15.973286+0000","last_peered":"2024-04-25T14:39:15.973286+0000","last_clean":"2024-04-25T14:39:15.973286+0000","last_became_active":"2024-04-25T14:38:10.418600+0000","last_became_peered":"2024-04-25T14:38:10.418600+0000","last_unstale":"2024-04-25T14:39:15.973286+0000","last_undegraded":"2024-04-25T14:39:15.973286+0000","last_fullsized":"2024-04-25T14:39:15.973286+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:31.324035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.975733+0000","last_change":"2024-04-25T14:38:09.556574+0000","last_active":"2024-04-25T14:39:15.975733+0000","last_peered":"2024-04-25T14:39:15.975733+0000","last_clean":"2024-04-25T14:39:15.975733+0000","last_became_active":"2024-04-25T14:38:09.556071+0000","last_became_peered":"2024-04-25T14:38:09.556071+0000","last_unstale":"2024-04-25T14:39:15.975733+0000","last_undegraded":"2024-04-25T14:39:15.975733+0000","last_fullsized":"2024-04-25T14:39:15.975733+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:05.424593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.969805+0000","last_change":"2024-04-25T14:38:10.424591+0000","last_active":"2024-04-25T14:39:15.969805+0000","last_peered":"2024-04-25T14:39:15.969805+0000","last_clean":"2024-04-25T14:39:15.969805+0000","last_became_active":"2024-04-25T14:38:10.424258+0000","last_became_peered":"2024-04-25T14:38:10.424258+0000","last_unstale":"2024-04-25T14:39:15.969805+0000","last_undegraded":"2024-04-25T14:39:15.969805+0000","last_fullsized":"2024-04-25T14:39:15.969805+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:22.753595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.976682+0000","last_change":"2024-04-25T14:38:09.546073+0000","last_active":"2024-04-25T14:39:15.976682+0000","last_peered":"2024-04-25T14:39:15.976682+0000","last_clean":"2024-04-25T14:39:15.976682+0000","last_became_active":"2024-04-25T14:38:09.545783+0000","last_became_peered":"2024-04-25T14:38:09.545783+0000","last_unstale":"2024-04-25T14:39:15.976682+0000","last_undegraded":"2024-04-25T14:39:15.976682+0000","last_fullsized":"2024-04-25T14:39:15.976682+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:42.537431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.978465+0000","last_change":"2024-04-25T14:38:10.417230+0000","last_active":"2024-04-25T14:39:15.978465+0000","last_peered":"2024-04-25T14:39:15.978465+0000","last_clean":"2024-04-25T14:39:15.978465+0000","last_became_active":"2024-04-25T14:38:10.416891+0000","last_became_peered":"2024-04-25T14:38:10.416891+0000","last_unstale":"2024-04-25T14:39:15.978465+0000","last_undegraded":"2024-04-25T14:39:15.978465+0000","last_fullsized":"2024-04-25T14:39:15.978465+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:10:13.434518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.994906+0000","last_change":"2024-04-25T14:39:15.994906+0000","last_active":"2024-04-25T14:39:15.994906+0000","last_peered":"2024-04-25T14:39:15.994906+0000","last_clean":"2024-04-25T14:39:15.994906+0000","last_became_active":"2024-04-25T14:39:15.994365+0000","last_became_peered":"2024-04-25T14:39:15.994365+0000","last_unstale":"2024-04-25T14:39:15.994906+0000","last_undegraded":"2024-04-25T14:39:15.994906+0000","last_fullsized":"2024-04-25T14:39:15.994906+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:19.026241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.972917+0000","last_change":"2024-04-25T14:38:10.446846+0000","last_active":"2024-04-25T14:39:15.972917+0000","last_peered":"2024-04-25T14:39:15.972917+0000","last_clean":"2024-04-25T14:39:15.972917+0000","last_became_active":"2024-04-25T14:38:10.446498+0000","last_became_peered":"2024-04-25T14:38:10.446498+0000","last_unstale":"2024-04-25T14:39:15.972917+0000","last_undegraded":"2024-04-25T14:39:15.972917+0000","last_fullsized":"2024-04-25T14:39:15.972917+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:14:37.179198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.972775+0000","last_change":"2024-04-25T14:38:09.544283+0000","last_active":"2024-04-25T14:39:15.972775+0000","last_peered":"2024-04-25T14:39:15.972775+0000","last_clean":"2024-04-25T14:39:15.972775+0000","last_became_active":"2024-04-25T14:38:09.543135+0000","last_became_peered":"2024-04-25T14:38:09.543135+0000","last_unstale":"2024-04-25T14:39:15.972775+0000","last_undegraded":"2024-04-25T14:39:15.972775+0000","last_fullsized":"2024-04-25T14:39:15.972775+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:05.977062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.972870+0000","last_change":"2024-04-25T14:38:10.432863+0000","last_active":"2024-04-25T14:39:15.972870+0000","last_peered":"2024-04-25T14:39:15.972870+0000","last_clean":"2024-04-25T14:39:15.972870+0000","last_became_active":"2024-04-25T14:38:10.432493+0000","last_became_peered":"2024-04-25T14:38:10.432493+0000","last_unstale":"2024-04-25T14:39:15.972870+0000","last_undegraded":"2024-04-25T14:39:15.972870+0000","last_fullsized":"2024-04-25T14:39:15.972870+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:25:53.774764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.981979+0000","last_change":"2024-04-25T14:39:15.981979+0000","last_active":"2024-04-25T14:39:15.981979+0000","last_peered":"2024-04-25T14:39:15.981979+0000","last_clean":"2024-04-25T14:39:15.981979+0000","last_became_active":"2024-04-25T14:39:15.981539+0000","last_became_peered":"2024-04-25T14:39:15.981539+0000","last_unstale":"2024-04-25T14:39:15.981979+0000","last_undegraded":"2024-04-25T14:39:15.981979+0000","last_fullsized":"2024-04-25T14:39:15.981979+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:22.906826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":104,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:34.813737+0000","last_change":"2024-04-25T14:38:09.554305+0000","last_active":"2024-04-25T14:39:34.813737+0000","last_peered":"2024-04-25T14:39:34.813737+0000","last_clean":"2024-04-25T14:39:34.813737+0000","last_became_active":"2024-04-25T14:38:09.553984+0000","last_became_peered":"2024-04-25T14:38:09.553984+0000","last_unstale":"2024-04-25T14:39:34.813737+0000","last_undegraded":"2024-04-25T14:39:34.813737+0000","last_fullsized":"2024-04-25T14:39:34.813737+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:41.580867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":81,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.985001+0000","last_change":"2024-04-25T14:38:10.448290+0000","last_active":"2024-04-25T14:39:15.985001+0000","last_peered":"2024-04-25T14:39:15.985001+0000","last_clean":"2024-04-25T14:39:15.985001+0000","last_became_active":"2024-04-25T14:38:10.446147+0000","last_became_peered":"2024-04-25T14:38:10.446147+0000","last_unstale":"2024-04-25T14:39:15.985001+0000","last_undegraded":"2024-04-25T14:39:15.985001+0000","last_fullsized":"2024-04-25T14:39:15.985001+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:12.775551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.971273+0000","last_change":"2024-04-25T14:38:09.545209+0000","last_active":"2024-04-25T14:39:15.971273+0000","last_peered":"2024-04-25T14:39:15.971273+0000","last_clean":"2024-04-25T14:39:15.971273+0000","last_became_active":"2024-04-25T14:38:09.544861+0000","last_became_peered":"2024-04-25T14:38:09.544861+0000","last_unstale":"2024-04-25T14:39:15.971273+0000","last_undegraded":"2024-04-25T14:39:15.971273+0000","last_fullsized":"2024-04-25T14:39:15.971273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:25.829162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":354,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.973537+0000","last_change":"2024-04-25T14:36:48.250081+0000","last_active":"2024-04-25T14:39:15.973537+0000","last_peered":"2024-04-25T14:39:15.973537+0000","last_clean":"2024-04-25T14:39:15.973537+0000","last_became_active":"2024-04-25T14:36:48.249520+0000","last_became_peered":"2024-04-25T14:36:48.249520+0000","last_unstale":"2024-04-25T14:39:15.973537+0000","last_undegraded":"2024-04-25T14:39:15.973537+0000","last_fullsized":"2024-04-25T14:39:15.973537+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:33:03.061016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:33:03.061016+0000","last_clean_scrub_stamp":"2024-04-25T14:33:03.061016+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:04:06.653326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.973820+0000","last_change":"2024-04-25T14:38:10.422966+0000","last_active":"2024-04-25T14:39:15.973820+0000","last_peered":"2024-04-25T14:39:15.973820+0000","last_clean":"2024-04-25T14:39:15.973820+0000","last_became_active":"2024-04-25T14:38:10.422568+0000","last_became_peered":"2024-04-25T14:38:10.422568+0000","last_unstale":"2024-04-25T14:39:15.973820+0000","last_undegraded":"2024-04-25T14:39:15.973820+0000","last_fullsized":"2024-04-25T14:39:15.973820+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:17.405074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.982120+0000","last_change":"2024-04-25T14:39:15.982120+0000","last_active":"2024-04-25T14:39:15.982120+0000","last_peered":"2024-04-25T14:39:15.982120+0000","last_clean":"2024-04-25T14:39:15.982120+0000","last_became_active":"2024-04-25T14:39:15.975343+0000","last_became_peered":"2024-04-25T14:39:15.975343+0000","last_unstale":"2024-04-25T14:39:15.982120+0000","last_undegraded":"2024-04-25T14:39:15.982120+0000","last_fullsized":"2024-04-25T14:39:15.982120+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:38.632052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.999140+0000","last_change":"2024-04-25T14:39:15.998744+0000","last_active":"2024-04-25T14:39:15.999140+0000","last_peered":"2024-04-25T14:39:15.999140+0000","last_clean":"2024-04-25T14:39:15.999140+0000","last_became_active":"2024-04-25T14:39:15.993416+0000","last_became_peered":"2024-04-25T14:39:15.993416+0000","last_unstale":"2024-04-25T14:39:15.999140+0000","last_undegraded":"2024-04-25T14:39:15.999140+0000","last_fullsized":"2024-04-25T14:39:15.999140+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:24.019707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.976059+0000","last_change":"2024-04-25T14:38:10.429554+0000","last_active":"2024-04-25T14:39:15.976059+0000","last_peered":"2024-04-25T14:39:15.976059+0000","last_clean":"2024-04-25T14:39:15.976059+0000","last_became_active":"2024-04-25T14:38:10.428944+0000","last_became_peered":"2024-04-25T14:38:10.428944+0000","last_unstale":"2024-04-25T14:39:15.976059+0000","last_undegraded":"2024-04-25T14:39:15.976059+0000","last_fullsized":"2024-04-25T14:39:15.976059+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:38.013028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.978926+0000","last_change":"2024-04-25T14:38:09.564849+0000","last_active":"2024-04-25T14:39:15.978926+0000","last_peered":"2024-04-25T14:39:15.978926+0000","last_clean":"2024-04-25T14:39:15.978926+0000","last_became_active":"2024-04-25T14:38:09.564354+0000","last_became_peered":"2024-04-25T14:38:09.564354+0000","last_unstale":"2024-04-25T14:39:15.978926+0000","last_undegraded":"2024-04-25T14:39:15.978926+0000","last_fullsized":"2024-04-25T14:39:15.978926+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:44.308366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.989359+0000","last_change":"2024-04-25T14:38:10.438178+0000","last_active":"2024-04-25T14:39:15.989359+0000","last_peered":"2024-04-25T14:39:15.989359+0000","last_clean":"2024-04-25T14:39:15.989359+0000","last_became_active":"2024-04-25T14:38:10.429936+0000","last_became_peered":"2024-04-25T14:38:10.429936+0000","last_unstale":"2024-04-25T14:39:15.989359+0000","last_undegraded":"2024-04-25T14:39:15.989359+0000","last_fullsized":"2024-04-25T14:39:15.989359+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:33.598747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.970600+0000","last_change":"2024-04-25T14:38:09.547680+0000","last_active":"2024-04-25T14:39:15.970600+0000","last_peered":"2024-04-25T14:39:15.970600+0000","last_clean":"2024-04-25T14:39:15.970600+0000","last_became_active":"2024-04-25T14:38:09.547337+0000","last_became_peered":"2024-04-25T14:38:09.547337+0000","last_unstale":"2024-04-25T14:39:15.970600+0000","last_undegraded":"2024-04-25T14:39:15.970600+0000","last_fullsized":"2024-04-25T14:39:15.970600+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:32.754439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.991740+0000","last_change":"2024-04-25T14:38:10.447115+0000","last_active":"2024-04-25T14:39:15.991740+0000","last_peered":"2024-04-25T14:39:15.991740+0000","last_clean":"2024-04-25T14:39:15.991740+0000","last_became_active":"2024-04-25T14:38:10.446681+0000","last_became_peered":"2024-04-25T14:38:10.446681+0000","last_unstale":"2024-04-25T14:39:15.991740+0000","last_undegraded":"2024-04-25T14:39:15.991740+0000","last_fullsized":"2024-04-25T14:39:15.991740+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:02:58.942053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:16.534479+0000","last_change":"2024-04-25T14:38:09.551523+0000","last_active":"2024-04-25T14:39:16.534479+0000","last_peered":"2024-04-25T14:39:16.534479+0000","last_clean":"2024-04-25T14:39:16.534479+0000","last_became_active":"2024-04-25T14:38:09.551213+0000","last_became_peered":"2024-04-25T14:38:09.551213+0000","last_unstale":"2024-04-25T14:39:16.534479+0000","last_undegraded":"2024-04-25T14:39:16.534479+0000","last_fullsized":"2024-04-25T14:39:16.534479+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:48.948571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.975775+0000","last_change":"2024-04-25T14:38:10.420032+0000","last_active":"2024-04-25T14:39:15.975775+0000","last_peered":"2024-04-25T14:39:15.975775+0000","last_clean":"2024-04-25T14:39:15.975775+0000","last_became_active":"2024-04-25T14:38:10.418349+0000","last_became_peered":"2024-04-25T14:38:10.418349+0000","last_unstale":"2024-04-25T14:39:15.975775+0000","last_undegraded":"2024-04-25T14:39:15.975775+0000","last_fullsized":"2024-04-25T14:39:15.975775+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:38:55.669414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.984235+0000","last_change":"2024-04-25T14:38:09.567097+0000","last_active":"2024-04-25T14:39:15.984235+0000","last_peered":"2024-04-25T14:39:15.984235+0000","last_clean":"2024-04-25T14:39:15.984235+0000","last_became_active":"2024-04-25T14:38:09.566852+0000","last_became_peered":"2024-04-25T14:38:09.566852+0000","last_unstale":"2024-04-25T14:39:15.984235+0000","last_undegraded":"2024-04-25T14:39:15.984235+0000","last_fullsized":"2024-04-25T14:39:15.984235+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:41:55.773005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.978187+0000","last_change":"2024-04-25T14:38:10.432691+0000","last_active":"2024-04-25T14:39:15.978187+0000","last_peered":"2024-04-25T14:39:15.978187+0000","last_clean":"2024-04-25T14:39:15.978187+0000","last_became_active":"2024-04-25T14:38:10.432321+0000","last_became_peered":"2024-04-25T14:38:10.432321+0000","last_unstale":"2024-04-25T14:39:15.978187+0000","last_undegraded":"2024-04-25T14:39:15.978187+0000","last_fullsized":"2024-04-25T14:39:15.978187+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:47:58.026331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.972673+0000","last_change":"2024-04-25T14:38:09.563692+0000","last_active":"2024-04-25T14:39:15.972673+0000","last_peered":"2024-04-25T14:39:15.972673+0000","last_clean":"2024-04-25T14:39:15.972673+0000","last_became_active":"2024-04-25T14:38:09.563345+0000","last_became_peered":"2024-04-25T14:38:09.563345+0000","last_unstale":"2024-04-25T14:39:15.972673+0000","last_undegraded":"2024-04-25T14:39:15.972673+0000","last_fullsized":"2024-04-25T14:39:15.972673+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:43.221429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.984407+0000","last_change":"2024-04-25T14:38:10.442519+0000","last_active":"2024-04-25T14:39:15.984407+0000","last_peered":"2024-04-25T14:39:15.984407+0000","last_clean":"2024-04-25T14:39:15.984407+0000","last_became_active":"2024-04-25T14:38:10.439045+0000","last_became_peered":"2024-04-25T14:38:10.439045+0000","last_unstale":"2024-04-25T14:39:15.984407+0000","last_undegraded":"2024-04-25T14:39:15.984407+0000","last_fullsized":"2024-04-25T14:39:15.984407+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:42.425106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.976059+0000","last_change":"2024-04-25T14:38:09.550363+0000","last_active":"2024-04-25T14:39:15.976059+0000","last_peered":"2024-04-25T14:39:15.976059+0000","last_clean":"2024-04-25T14:39:15.976059+0000","last_became_active":"2024-04-25T14:38:09.550044+0000","last_became_peered":"2024-04-25T14:38:09.550044+0000","last_unstale":"2024-04-25T14:39:15.976059+0000","last_undegraded":"2024-04-25T14:39:15.976059+0000","last_fullsized":"2024-04-25T14:39:15.976059+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:25.573353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.972986+0000","last_change":"2024-04-25T14:38:10.428736+0000","last_active":"2024-04-25T14:39:15.972986+0000","last_peered":"2024-04-25T14:39:15.972986+0000","last_clean":"2024-04-25T14:39:15.972986+0000","last_became_active":"2024-04-25T14:38:10.428410+0000","last_became_peered":"2024-04-25T14:38:10.428410+0000","last_unstale":"2024-04-25T14:39:15.972986+0000","last_undegraded":"2024-04-25T14:39:15.972986+0000","last_fullsized":"2024-04-25T14:39:15.972986+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:02:51.474220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.972844+0000","last_change":"2024-04-25T14:38:09.552800+0000","last_active":"2024-04-25T14:39:15.972844+0000","last_peered":"2024-04-25T14:39:15.972844+0000","last_clean":"2024-04-25T14:39:15.972844+0000","last_became_active":"2024-04-25T14:38:09.552460+0000","last_became_peered":"2024-04-25T14:38:09.552460+0000","last_unstale":"2024-04-25T14:39:15.972844+0000","last_undegraded":"2024-04-25T14:39:15.972844+0000","last_fullsized":"2024-04-25T14:39:15.972844+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:00.800770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.973424+0000","last_change":"2024-04-25T14:38:10.424359+0000","last_active":"2024-04-25T14:39:15.973424+0000","last_peered":"2024-04-25T14:39:15.973424+0000","last_clean":"2024-04-25T14:39:15.973424+0000","last_became_active":"2024-04-25T14:38:10.424020+0000","last_became_peered":"2024-04-25T14:38:10.424020+0000","last_unstale":"2024-04-25T14:39:15.973424+0000","last_undegraded":"2024-04-25T14:39:15.973424+0000","last_fullsized":"2024-04-25T14:39:15.973424+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:14.700772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.970176+0000","last_change":"2024-04-25T14:38:09.545637+0000","last_active":"2024-04-25T14:39:15.970176+0000","last_peered":"2024-04-25T14:39:15.970176+0000","last_clean":"2024-04-25T14:39:15.970176+0000","last_became_active":"2024-04-25T14:38:09.545319+0000","last_became_peered":"2024-04-25T14:38:09.545319+0000","last_unstale":"2024-04-25T14:39:15.970176+0000","last_undegraded":"2024-04-25T14:39:15.970176+0000","last_fullsized":"2024-04-25T14:39:15.970176+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:02.284715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.973079+0000","last_change":"2024-04-25T14:38:10.446928+0000","last_active":"2024-04-25T14:39:15.973079+0000","last_peered":"2024-04-25T14:39:15.973079+0000","last_clean":"2024-04-25T14:39:15.973079+0000","last_became_active":"2024-04-25T14:38:10.446518+0000","last_became_peered":"2024-04-25T14:38:10.446518+0000","last_unstale":"2024-04-25T14:39:15.973079+0000","last_undegraded":"2024-04-25T14:39:15.973079+0000","last_fullsized":"2024-04-25T14:39:15.973079+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:40.769059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.976728+0000","last_change":"2024-04-25T14:38:09.557381+0000","last_active":"2024-04-25T14:39:15.976728+0000","last_peered":"2024-04-25T14:39:15.976728+0000","last_clean":"2024-04-25T14:39:15.976728+0000","last_became_active":"2024-04-25T14:38:09.556504+0000","last_became_peered":"2024-04-25T14:38:09.556504+0000","last_unstale":"2024-04-25T14:39:15.976728+0000","last_undegraded":"2024-04-25T14:39:15.976728+0000","last_fullsized":"2024-04-25T14:39:15.976728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:01:02.761249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.973396+0000","last_change":"2024-04-25T14:38:10.418104+0000","last_active":"2024-04-25T14:39:15.973396+0000","last_peered":"2024-04-25T14:39:15.973396+0000","last_clean":"2024-04-25T14:39:15.973396+0000","last_became_active":"2024-04-25T14:38:10.417774+0000","last_became_peered":"2024-04-25T14:38:10.417774+0000","last_unstale":"2024-04-25T14:39:15.973396+0000","last_undegraded":"2024-04-25T14:39:15.973396+0000","last_fullsized":"2024-04-25T14:39:15.973396+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:32:44.229899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.978293+0000","last_change":"2024-04-25T14:38:10.423546+0000","last_active":"2024-04-25T14:39:15.978293+0000","last_peered":"2024-04-25T14:39:15.978293+0000","last_clean":"2024-04-25T14:39:15.978293+0000","last_became_active":"2024-04-25T14:38:10.423216+0000","last_became_peered":"2024-04-25T14:38:10.423216+0000","last_unstale":"2024-04-25T14:39:15.978293+0000","last_undegraded":"2024-04-25T14:39:15.978293+0000","last_fullsized":"2024-04-25T14:39:15.978293+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:58.640669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.972779+0000","last_change":"2024-04-25T14:38:10.422504+0000","last_active":"2024-04-25T14:39:15.972779+0000","last_peered":"2024-04-25T14:39:15.972779+0000","last_clean":"2024-04-25T14:39:15.972779+0000","last_became_active":"2024-04-25T14:38:10.422095+0000","last_became_peered":"2024-04-25T14:38:10.422095+0000","last_unstale":"2024-04-25T14:39:15.972779+0000","last_undegraded":"2024-04-25T14:39:15.972779+0000","last_fullsized":"2024-04-25T14:39:15.972779+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:25.795962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.972439+0000","last_change":"2024-04-25T14:38:10.447446+0000","last_active":"2024-04-25T14:39:15.972439+0000","last_peered":"2024-04-25T14:39:15.972439+0000","last_clean":"2024-04-25T14:39:15.972439+0000","last_became_active":"2024-04-25T14:38:10.447056+0000","last_became_peered":"2024-04-25T14:38:10.447056+0000","last_unstale":"2024-04-25T14:39:15.972439+0000","last_undegraded":"2024-04-25T14:39:15.972439+0000","last_fullsized":"2024-04-25T14:39:15.972439+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:40.462340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.990523+0000","last_change":"2024-04-25T14:38:10.433444+0000","last_active":"2024-04-25T14:39:15.990523+0000","last_peered":"2024-04-25T14:39:15.990523+0000","last_clean":"2024-04-25T14:39:15.990523+0000","last_became_active":"2024-04-25T14:38:10.433153+0000","last_became_peered":"2024-04-25T14:38:10.433153+0000","last_unstale":"2024-04-25T14:39:15.990523+0000","last_undegraded":"2024-04-25T14:39:15.990523+0000","last_fullsized":"2024-04-25T14:39:15.990523+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:10.722650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.975940+0000","last_change":"2024-04-25T14:38:10.438322+0000","last_active":"2024-04-25T14:39:15.975940+0000","last_peered":"2024-04-25T14:39:15.975940+0000","last_clean":"2024-04-25T14:39:15.975940+0000","last_became_active":"2024-04-25T14:38:10.436103+0000","last_became_peered":"2024-04-25T14:38:10.436103+0000","last_unstale":"2024-04-25T14:39:15.975940+0000","last_undegraded":"2024-04-25T14:39:15.975940+0000","last_fullsized":"2024-04-25T14:39:15.975940+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:16.790202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.978611+0000","last_change":"2024-04-25T14:38:10.419710+0000","last_active":"2024-04-25T14:39:15.978611+0000","last_peered":"2024-04-25T14:39:15.978611+0000","last_clean":"2024-04-25T14:39:15.978611+0000","last_became_active":"2024-04-25T14:38:10.419406+0000","last_became_peered":"2024-04-25T14:38:10.419406+0000","last_unstale":"2024-04-25T14:39:15.978611+0000","last_undegraded":"2024-04-25T14:39:15.978611+0000","last_fullsized":"2024-04-25T14:39:15.978611+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:37.964698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.972816+0000","last_change":"2024-04-25T14:38:10.429510+0000","last_active":"2024-04-25T14:39:15.972816+0000","last_peered":"2024-04-25T14:39:15.972816+0000","last_clean":"2024-04-25T14:39:15.972816+0000","last_became_active":"2024-04-25T14:38:10.429219+0000","last_became_peered":"2024-04-25T14:38:10.429219+0000","last_unstale":"2024-04-25T14:39:15.972816+0000","last_undegraded":"2024-04-25T14:39:15.972816+0000","last_fullsized":"2024-04-25T14:39:15.972816+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:52:46.673126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.989155+0000","last_change":"2024-04-25T14:38:10.441487+0000","last_active":"2024-04-25T14:39:15.989155+0000","last_peered":"2024-04-25T14:39:15.989155+0000","last_clean":"2024-04-25T14:39:15.989155+0000","last_became_active":"2024-04-25T14:38:10.441111+0000","last_became_peered":"2024-04-25T14:38:10.441111+0000","last_unstale":"2024-04-25T14:39:15.989155+0000","last_undegraded":"2024-04-25T14:39:15.989155+0000","last_fullsized":"2024-04-25T14:39:15.989155+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:59.842687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.970065+0000","last_change":"2024-04-25T14:38:10.436511+0000","last_active":"2024-04-25T14:39:15.970065+0000","last_peered":"2024-04-25T14:39:15.970065+0000","last_clean":"2024-04-25T14:39:15.970065+0000","last_became_active":"2024-04-25T14:38:10.436171+0000","last_became_peered":"2024-04-25T14:38:10.436171+0000","last_unstale":"2024-04-25T14:39:15.970065+0000","last_undegraded":"2024-04-25T14:39:15.970065+0000","last_fullsized":"2024-04-25T14:39:15.970065+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:27:19.605907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.983017+0000","last_change":"2024-04-25T14:38:10.433624+0000","last_active":"2024-04-25T14:39:15.983017+0000","last_peered":"2024-04-25T14:39:15.983017+0000","last_clean":"2024-04-25T14:39:15.983017+0000","last_became_active":"2024-04-25T14:38:10.433258+0000","last_became_peered":"2024-04-25T14:38:10.433258+0000","last_unstale":"2024-04-25T14:39:15.983017+0000","last_undegraded":"2024-04-25T14:39:15.983017+0000","last_fullsized":"2024-04-25T14:39:15.983017+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:53.563110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:16.534632+0000","last_change":"2024-04-25T14:38:10.446924+0000","last_active":"2024-04-25T14:39:16.534632+0000","last_peered":"2024-04-25T14:39:16.534632+0000","last_clean":"2024-04-25T14:39:16.534632+0000","last_became_active":"2024-04-25T14:38:10.446553+0000","last_became_peered":"2024-04-25T14:38:10.446553+0000","last_unstale":"2024-04-25T14:39:16.534632+0000","last_undegraded":"2024-04-25T14:39:16.534632+0000","last_fullsized":"2024-04-25T14:39:16.534632+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:00.203796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.974877+0000","last_change":"2024-04-25T14:38:10.429952+0000","last_active":"2024-04-25T14:39:15.974877+0000","last_peered":"2024-04-25T14:39:15.974877+0000","last_clean":"2024-04-25T14:39:15.974877+0000","last_became_active":"2024-04-25T14:38:10.429626+0000","last_became_peered":"2024-04-25T14:38:10.429626+0000","last_unstale":"2024-04-25T14:39:15.974877+0000","last_undegraded":"2024-04-25T14:39:15.974877+0000","last_fullsized":"2024-04-25T14:39:15.974877+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:07.287113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.978351+0000","last_change":"2024-04-25T14:38:10.418292+0000","last_active":"2024-04-25T14:39:15.978351+0000","last_peered":"2024-04-25T14:39:15.978351+0000","last_clean":"2024-04-25T14:39:15.978351+0000","last_became_active":"2024-04-25T14:38:10.418005+0000","last_became_peered":"2024-04-25T14:38:10.418005+0000","last_unstale":"2024-04-25T14:39:15.978351+0000","last_undegraded":"2024-04-25T14:39:15.978351+0000","last_fullsized":"2024-04-25T14:39:15.978351+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:25.486993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66288,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":526,"num_read_kb":334,"num_write":161,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1035,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":198864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285601,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":987136,"data_stored":469673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547238,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28664,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716488,"statfs":{"total":95995035648,"available":95965683712,"internally_reserved":0,"allocated":1679360,"data_stored":1123419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710763,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":469887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972402,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":995328,"data_stored":469785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266741,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28576,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716576,"statfs":{"total":95995035648,"available":95965773824,"internally_reserved":0,"allocated":1593344,"data_stored":1062128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430266,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":469276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":530965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1007616,"data_stored":470547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1064960,"data_stored":531693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149772,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":469587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888137,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":999424,"data_stored":471117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691906,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32748,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712404,"statfs":{"total":95995035648,"available":95961501696,"internally_reserved":0,"allocated":1605632,"data_stored":1061421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":62624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:39:41.907 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T14:39:42.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:42 smithi053 ceph-mon[27773]: pgmap v363: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T14:39:42.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:42 smithi053 ceph-mon[27773]: from='client.24913 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:39:42.381 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:39:42.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:42 smithi183 ceph-mon[32584]: pgmap v363: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T14:39:42.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:42 smithi183 ceph-mon[32584]: from='client.24913 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:39:42.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:42 smithi150 ceph-mon[32504]: pgmap v363: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T14:39:42.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:42 smithi150 ceph-mon[32504]: from='client.24913 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:39:44.370 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:44 smithi053 ceph-mon[27773]: pgmap v364: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T14:39:44.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:44 smithi183 ceph-mon[32584]: pgmap v364: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T14:39:44.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:44 smithi150 ceph-mon[32504]: pgmap v364: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T14:39:44.649 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.648+0000 7fddde000700 1 -- 172.21.15.53:0/977100512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddd8108a30 msgr2=0x7fddd8108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:44.649 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.648+0000 7fddde000700 1 --2- 172.21.15.53:0/977100512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddd8108a30 0x7fddd8108e10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fddc8009990 tx=0x7fddc8009ca0 comp rx=0 tx=0).stop 2024-04-25T14:39:44.649 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.648+0000 7fddde000700 1 -- 172.21.15.53:0/977100512 shutdown_connections 2024-04-25T14:39:44.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.648+0000 7fddde000700 1 --2- 172.21.15.53:0/977100512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fddd806b0f0 0x7fddd810f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:44.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.648+0000 7fddde000700 1 --2- 172.21.15.53:0/977100512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddd8108a30 0x7fddd8108e10 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:44.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.648+0000 7fddde000700 1 --2- 172.21.15.53:0/977100512 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fddd806a750 0x7fddd806abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:44.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.648+0000 7fddde000700 1 -- 172.21.15.53:0/977100512 >> 172.21.15.53:0/977100512 conn(0x7fddd80788f0 msgr2=0x7fddd8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:44.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.648+0000 7fddde000700 1 -- 172.21.15.53:0/977100512 shutdown_connections 2024-04-25T14:39:44.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.648+0000 7fddde000700 1 -- 172.21.15.53:0/977100512 wait complete. 2024-04-25T14:39:44.651 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.649+0000 7fddde000700 1 Processor -- start 2024-04-25T14:39:44.651 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.650+0000 7fddde000700 1 -- start start 2024-04-25T14:39:44.651 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.650+0000 7fddde000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fddd806a750 0x7fddd819b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:44.651 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.650+0000 7fddde000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddd806b0f0 0x7fddd819b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:44.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.650+0000 7fddde000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fddd8108a30 0x7fddd819fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:44.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.650+0000 7fddde000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fddd8112bb0 con 0x7fddd806b0f0 2024-04-25T14:39:44.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.650+0000 7fddde000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fddd8112a30 con 0x7fddd806a750 2024-04-25T14:39:44.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.650+0000 7fddde000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fddd8112d30 con 0x7fddd8108a30 2024-04-25T14:39:44.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.650+0000 7fddd7400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fddd806a750 0x7fddd819b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:44.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.650+0000 7fddd7400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fddd806a750 0x7fddd819b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.53:57932/0 (socket says 172.21.15.53:57932) 2024-04-25T14:39:44.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.650+0000 7fddd7400700 1 -- 172.21.15.53:0/3453620620 learned_addr learned my addr 172.21.15.53:0/3453620620 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:44.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.650+0000 7fddcea00700 1 --2- 172.21.15.53:0/3453620620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddd806b0f0 0x7fddd819b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:44.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.651+0000 7fddd7e00700 1 --2- 172.21.15.53:0/3453620620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fddd8108a30 0x7fddd819fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:44.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.651+0000 7fddd7400700 1 -- 172.21.15.53:0/3453620620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fddd8108a30 msgr2=0x7fddd819fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:44.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.651+0000 7fddd7400700 1 --2- 172.21.15.53:0/3453620620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fddd8108a30 0x7fddd819fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:44.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.651+0000 7fddd7400700 1 -- 172.21.15.53:0/3453620620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddd806b0f0 msgr2=0x7fddd819b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:44.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.651+0000 7fddd7400700 1 --2- 172.21.15.53:0/3453620620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddd806b0f0 0x7fddd819b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:44.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.651+0000 7fddd7400700 1 -- 172.21.15.53:0/3453620620 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd81a0250 con 0x7fddd806a750 2024-04-25T14:39:44.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.651+0000 7fddd7400700 1 --2- 172.21.15.53:0/3453620620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fddd806a750 0x7fddd819b240 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fddc8000c00 tx=0x7fddc80047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:44.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.651+0000 7fddd4c00700 1 -- 172.21.15.53:0/3453620620 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc8027070 con 0x7fddd806a750 2024-04-25T14:39:44.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.652+0000 7fddd4c00700 1 -- 172.21.15.53:0/3453620620 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddc8003f20 con 0x7fddd806a750 2024-04-25T14:39:44.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.652+0000 7fddde000700 1 -- 172.21.15.53:0/3453620620 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddd81a04e0 con 0x7fddd806a750 2024-04-25T14:39:44.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.652+0000 7fddd4c00700 1 -- 172.21.15.53:0/3453620620 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc8022ca0 con 0x7fddd806a750 2024-04-25T14:39:44.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.652+0000 7fddde000700 1 -- 172.21.15.53:0/3453620620 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddd81a7dc0 con 0x7fddd806a750 2024-04-25T14:39:44.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.653+0000 7fddde000700 1 -- 172.21.15.53:0/3453620620 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddd8069730 con 0x7fddd806a750 2024-04-25T14:39:44.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.655+0000 7fddd4c00700 1 -- 172.21.15.53:0/3453620620 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fddc802a070 con 0x7fddd806a750 2024-04-25T14:39:44.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.657+0000 7fddd4c00700 1 --2- 172.21.15.53:0/3453620620 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fddbc0a3d50 0x7fddbc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:44.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.657+0000 7fddd4c00700 1 -- 172.21.15.53:0/3453620620 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fddc80d1ab0 con 0x7fddd806a750 2024-04-25T14:39:44.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.657+0000 7fddcea00700 1 --2- 172.21.15.53:0/3453620620 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fddbc0a3d50 0x7fddbc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:44.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.658+0000 7fddcea00700 1 --2- 172.21.15.53:0/3453620620 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fddbc0a3d50 0x7fddbc0a6210 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fddd8078e60 tx=0x7fddc000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:44.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.659+0000 7fddd4c00700 1 -- 172.21.15.53:0/3453620620 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fddc8031070 con 0x7fddd806a750 2024-04-25T14:39:44.887 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.886+0000 7fddde000700 1 -- 172.21.15.53:0/3453620620 --> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fddd819c5a0 con 0x7fddbc0a3d50 2024-04-25T14:39:44.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.897+0000 7fddd4c00700 1 -- 172.21.15.53:0/3453620620 <== mgr.14152 v2:172.21.15.53:6800/4154933108 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258752 (secure 0 0 0) 0x7fddd819c5a0 con 0x7fddbc0a3d50 2024-04-25T14:39:44.903 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:39:44.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.902+0000 7fddde000700 1 -- 172.21.15.53:0/3453620620 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fddbc0a3d50 msgr2=0x7fddbc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:44.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.902+0000 7fddde000700 1 --2- 172.21.15.53:0/3453620620 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fddbc0a3d50 0x7fddbc0a6210 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fddd8078e60 tx=0x7fddc000b410 comp rx=0 tx=0).stop 2024-04-25T14:39:44.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.903+0000 7fddde000700 1 -- 172.21.15.53:0/3453620620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fddd806a750 msgr2=0x7fddd819b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:44.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.903+0000 7fddde000700 1 --2- 172.21.15.53:0/3453620620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fddd806a750 0x7fddd819b240 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fddc8000c00 tx=0x7fddc80047b0 comp rx=0 tx=0).stop 2024-04-25T14:39:44.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.903+0000 7fddde000700 1 -- 172.21.15.53:0/3453620620 shutdown_connections 2024-04-25T14:39:44.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.903+0000 7fddde000700 1 --2- 172.21.15.53:0/3453620620 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fddbc0a3d50 0x7fddbc0a6210 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:44.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.903+0000 7fddde000700 1 --2- 172.21.15.53:0/3453620620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fddd8108a30 0x7fddd819fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:44.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.903+0000 7fddde000700 1 --2- 172.21.15.53:0/3453620620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fddd806a750 0x7fddd819b240 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:44.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.903+0000 7fddde000700 1 --2- 172.21.15.53:0/3453620620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddd806b0f0 0x7fddd819b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:44.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.903+0000 7fddde000700 1 -- 172.21.15.53:0/3453620620 >> 172.21.15.53:0/3453620620 conn(0x7fddd80788f0 msgr2=0x7fddd810ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:44.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.903+0000 7fddde000700 1 -- 172.21.15.53:0/3453620620 shutdown_connections 2024-04-25T14:39:44.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:44.903+0000 7fddde000700 1 -- 172.21.15.53:0/3453620620 wait complete. 2024-04-25T14:39:44.905 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-04-25T14:39:45.642 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-04-25T14:39:44.342724+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656890,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":677,"num_read_kb":458,"num_write":344,"num_write_kb":3051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1181771,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":303,"ondisk_log_size":303,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375348,"kb_used_data":13644,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566476,"statfs":{"total":1151940427776,"available":1151556071424,"internally_reserved":0,"allocated":13971456,"data_stored":7599498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19137,"internal_metadata":370193727},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":64,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004788"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.978785+0000","last_change":"2024-04-25T14:38:10.435535+0000","last_active":"2024-04-25T14:39:15.978785+0000","last_peered":"2024-04-25T14:39:15.978785+0000","last_clean":"2024-04-25T14:39:15.978785+0000","last_became_active":"2024-04-25T14:38:10.435205+0000","last_became_peered":"2024-04-25T14:38:10.435205+0000","last_unstale":"2024-04-25T14:39:15.978785+0000","last_undegraded":"2024-04-25T14:39:15.978785+0000","last_fullsized":"2024-04-25T14:39:15.978785+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:12.301771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.991499+0000","last_change":"2024-04-25T14:38:10.428216+0000","last_active":"2024-04-25T14:39:15.991499+0000","last_peered":"2024-04-25T14:39:15.991499+0000","last_clean":"2024-04-25T14:39:15.991499+0000","last_became_active":"2024-04-25T14:38:10.427419+0000","last_became_peered":"2024-04-25T14:38:10.427419+0000","last_unstale":"2024-04-25T14:39:15.991499+0000","last_undegraded":"2024-04-25T14:39:15.991499+0000","last_fullsized":"2024-04-25T14:39:15.991499+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:42.580424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.991243+0000","last_change":"2024-04-25T14:38:10.427827+0000","last_active":"2024-04-25T14:39:15.991243+0000","last_peered":"2024-04-25T14:39:15.991243+0000","last_clean":"2024-04-25T14:39:15.991243+0000","last_became_active":"2024-04-25T14:38:10.427435+0000","last_became_peered":"2024-04-25T14:38:10.427435+0000","last_unstale":"2024-04-25T14:39:15.991243+0000","last_undegraded":"2024-04-25T14:39:15.991243+0000","last_fullsized":"2024-04-25T14:39:15.991243+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:15.539399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.970735+0000","last_change":"2024-04-25T14:38:10.423751+0000","last_active":"2024-04-25T14:39:15.970735+0000","last_peered":"2024-04-25T14:39:15.970735+0000","last_clean":"2024-04-25T14:39:15.970735+0000","last_became_active":"2024-04-25T14:38:10.423409+0000","last_became_peered":"2024-04-25T14:38:10.423409+0000","last_unstale":"2024-04-25T14:39:15.970735+0000","last_undegraded":"2024-04-25T14:39:15.970735+0000","last_fullsized":"2024-04-25T14:39:15.970735+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:45.403788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.976066+0000","last_change":"2024-04-25T14:38:10.430000+0000","last_active":"2024-04-25T14:39:15.976066+0000","last_peered":"2024-04-25T14:39:15.976066+0000","last_clean":"2024-04-25T14:39:15.976066+0000","last_became_active":"2024-04-25T14:38:10.429684+0000","last_became_peered":"2024-04-25T14:38:10.429684+0000","last_unstale":"2024-04-25T14:39:15.976066+0000","last_undegraded":"2024-04-25T14:39:15.976066+0000","last_fullsized":"2024-04-25T14:39:15.976066+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:18.421970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.984180+0000","last_change":"2024-04-25T14:39:15.984180+0000","last_active":"2024-04-25T14:39:15.984180+0000","last_peered":"2024-04-25T14:39:15.984180+0000","last_clean":"2024-04-25T14:39:15.984180+0000","last_became_active":"2024-04-25T14:39:15.983821+0000","last_became_peered":"2024-04-25T14:39:15.983821+0000","last_unstale":"2024-04-25T14:39:15.984180+0000","last_undegraded":"2024-04-25T14:39:15.984180+0000","last_fullsized":"2024-04-25T14:39:15.984180+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:54:47.617713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.972805+0000","last_change":"2024-04-25T14:38:10.420022+0000","last_active":"2024-04-25T14:39:15.972805+0000","last_peered":"2024-04-25T14:39:15.972805+0000","last_clean":"2024-04-25T14:39:15.972805+0000","last_became_active":"2024-04-25T14:38:10.419673+0000","last_became_peered":"2024-04-25T14:38:10.419673+0000","last_unstale":"2024-04-25T14:39:15.972805+0000","last_undegraded":"2024-04-25T14:39:15.972805+0000","last_fullsized":"2024-04-25T14:39:15.972805+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:09.177814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.988374+0000","last_change":"2024-04-25T14:39:15.988374+0000","last_active":"2024-04-25T14:39:15.988374+0000","last_peered":"2024-04-25T14:39:15.988374+0000","last_clean":"2024-04-25T14:39:15.988374+0000","last_became_active":"2024-04-25T14:39:15.983741+0000","last_became_peered":"2024-04-25T14:39:15.983741+0000","last_unstale":"2024-04-25T14:39:15.988374+0000","last_undegraded":"2024-04-25T14:39:15.988374+0000","last_fullsized":"2024-04-25T14:39:15.988374+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:15.523804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,10],"acting":[3,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:16.535006+0000","last_change":"2024-04-25T14:38:10.424086+0000","last_active":"2024-04-25T14:39:16.535006+0000","last_peered":"2024-04-25T14:39:16.535006+0000","last_clean":"2024-04-25T14:39:16.535006+0000","last_became_active":"2024-04-25T14:38:10.423769+0000","last_became_peered":"2024-04-25T14:38:10.423769+0000","last_unstale":"2024-04-25T14:39:16.535006+0000","last_undegraded":"2024-04-25T14:39:16.535006+0000","last_fullsized":"2024-04-25T14:39:16.535006+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:19:22.736053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.973107+0000","last_change":"2024-04-25T14:38:10.427049+0000","last_active":"2024-04-25T14:39:15.973107+0000","last_peered":"2024-04-25T14:39:15.973107+0000","last_clean":"2024-04-25T14:39:15.973107+0000","last_became_active":"2024-04-25T14:38:10.426729+0000","last_became_peered":"2024-04-25T14:38:10.426729+0000","last_unstale":"2024-04-25T14:39:15.973107+0000","last_undegraded":"2024-04-25T14:39:15.973107+0000","last_fullsized":"2024-04-25T14:39:15.973107+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:00.849650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.973395+0000","last_change":"2024-04-25T14:38:10.424976+0000","last_active":"2024-04-25T14:39:15.973395+0000","last_peered":"2024-04-25T14:39:15.973395+0000","last_clean":"2024-04-25T14:39:15.973395+0000","last_became_active":"2024-04-25T14:38:10.424585+0000","last_became_peered":"2024-04-25T14:38:10.424585+0000","last_unstale":"2024-04-25T14:39:15.973395+0000","last_undegraded":"2024-04-25T14:39:15.973395+0000","last_fullsized":"2024-04-25T14:39:15.973395+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:54.639471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.970561+0000","last_change":"2024-04-25T14:38:10.418022+0000","last_active":"2024-04-25T14:39:15.970561+0000","last_peered":"2024-04-25T14:39:15.970561+0000","last_clean":"2024-04-25T14:39:15.970561+0000","last_became_active":"2024-04-25T14:38:10.417735+0000","last_became_peered":"2024-04-25T14:38:10.417735+0000","last_unstale":"2024-04-25T14:39:15.970561+0000","last_undegraded":"2024-04-25T14:39:15.970561+0000","last_fullsized":"2024-04-25T14:39:15.970561+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:48.785521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.974740+0000","last_change":"2024-04-25T14:38:10.423464+0000","last_active":"2024-04-25T14:39:15.974740+0000","last_peered":"2024-04-25T14:39:15.974740+0000","last_clean":"2024-04-25T14:39:15.974740+0000","last_became_active":"2024-04-25T14:38:10.423105+0000","last_became_peered":"2024-04-25T14:38:10.423105+0000","last_unstale":"2024-04-25T14:39:15.974740+0000","last_undegraded":"2024-04-25T14:39:15.974740+0000","last_fullsized":"2024-04-25T14:39:15.974740+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:26.568198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.989428+0000","last_change":"2024-04-25T14:38:10.426870+0000","last_active":"2024-04-25T14:39:15.989428+0000","last_peered":"2024-04-25T14:39:15.989428+0000","last_clean":"2024-04-25T14:39:15.989428+0000","last_became_active":"2024-04-25T14:38:10.426506+0000","last_became_peered":"2024-04-25T14:38:10.426506+0000","last_unstale":"2024-04-25T14:39:15.989428+0000","last_undegraded":"2024-04-25T14:39:15.989428+0000","last_fullsized":"2024-04-25T14:39:15.989428+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:16:49.539221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.988922+0000","last_change":"2024-04-25T14:38:10.434618+0000","last_active":"2024-04-25T14:39:15.988922+0000","last_peered":"2024-04-25T14:39:15.988922+0000","last_clean":"2024-04-25T14:39:15.988922+0000","last_became_active":"2024-04-25T14:38:10.430661+0000","last_became_peered":"2024-04-25T14:38:10.430661+0000","last_unstale":"2024-04-25T14:39:15.988922+0000","last_undegraded":"2024-04-25T14:39:15.988922+0000","last_fullsized":"2024-04-25T14:39:15.988922+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:44.548788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:16.534468+0000","last_change":"2024-04-25T14:38:10.429898+0000","last_active":"2024-04-25T14:39:16.534468+0000","last_peered":"2024-04-25T14:39:16.534468+0000","last_clean":"2024-04-25T14:39:16.534468+0000","last_became_active":"2024-04-25T14:38:10.429557+0000","last_became_peered":"2024-04-25T14:38:10.429557+0000","last_unstale":"2024-04-25T14:39:16.534468+0000","last_undegraded":"2024-04-25T14:39:16.534468+0000","last_fullsized":"2024-04-25T14:39:16.534468+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:54.258526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.978841+0000","last_change":"2024-04-25T14:38:10.435134+0000","last_active":"2024-04-25T14:39:15.978841+0000","last_peered":"2024-04-25T14:39:15.978841+0000","last_clean":"2024-04-25T14:39:15.978841+0000","last_became_active":"2024-04-25T14:38:10.434775+0000","last_became_peered":"2024-04-25T14:38:10.434775+0000","last_unstale":"2024-04-25T14:39:15.978841+0000","last_undegraded":"2024-04-25T14:39:15.978841+0000","last_fullsized":"2024-04-25T14:39:15.978841+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:49.827881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:44.314128+0000","last_change":"2024-04-25T14:38:10.424929+0000","last_active":"2024-04-25T14:39:44.314128+0000","last_peered":"2024-04-25T14:39:44.314128+0000","last_clean":"2024-04-25T14:39:44.314128+0000","last_became_active":"2024-04-25T14:38:10.424399+0000","last_became_peered":"2024-04-25T14:38:10.424399+0000","last_unstale":"2024-04-25T14:39:44.314128+0000","last_undegraded":"2024-04-25T14:39:44.314128+0000","last_fullsized":"2024-04-25T14:39:44.314128+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:08.834176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.972902+0000","last_change":"2024-04-25T14:38:10.444004+0000","last_active":"2024-04-25T14:39:15.972902+0000","last_peered":"2024-04-25T14:39:15.972902+0000","last_clean":"2024-04-25T14:39:15.972902+0000","last_became_active":"2024-04-25T14:38:10.443669+0000","last_became_peered":"2024-04-25T14:38:10.443669+0000","last_unstale":"2024-04-25T14:39:15.972902+0000","last_undegraded":"2024-04-25T14:39:15.972902+0000","last_fullsized":"2024-04-25T14:39:15.972902+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:02.302057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.980018+0000","last_change":"2024-04-25T14:38:09.545010+0000","last_active":"2024-04-25T14:39:15.980018+0000","last_peered":"2024-04-25T14:39:15.980018+0000","last_clean":"2024-04-25T14:39:15.980018+0000","last_became_active":"2024-04-25T14:38:09.544622+0000","last_became_peered":"2024-04-25T14:38:09.544622+0000","last_unstale":"2024-04-25T14:39:15.980018+0000","last_undegraded":"2024-04-25T14:39:15.980018+0000","last_fullsized":"2024-04-25T14:39:15.980018+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:51.200242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:44.307838+0000","last_change":"2024-04-25T14:39:15.993912+0000","last_active":"2024-04-25T14:39:44.307838+0000","last_peered":"2024-04-25T14:39:44.307838+0000","last_clean":"2024-04-25T14:39:44.307838+0000","last_became_active":"2024-04-25T14:39:15.992511+0000","last_became_peered":"2024-04-25T14:39:15.992511+0000","last_unstale":"2024-04-25T14:39:44.307838+0000","last_undegraded":"2024-04-25T14:39:44.307838+0000","last_fullsized":"2024-04-25T14:39:44.307838+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:12:57.639396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,6],"acting":[9,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.988836+0000","last_change":"2024-04-25T14:38:09.555825+0000","last_active":"2024-04-25T14:39:15.988836+0000","last_peered":"2024-04-25T14:39:15.988836+0000","last_clean":"2024-04-25T14:39:15.988836+0000","last_became_active":"2024-04-25T14:38:09.555436+0000","last_became_peered":"2024-04-25T14:38:09.555436+0000","last_unstale":"2024-04-25T14:39:15.988836+0000","last_undegraded":"2024-04-25T14:39:15.988836+0000","last_fullsized":"2024-04-25T14:39:15.988836+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:45.097583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.992495+0000","last_change":"2024-04-25T14:38:10.432254+0000","last_active":"2024-04-25T14:39:15.992495+0000","last_peered":"2024-04-25T14:39:15.992495+0000","last_clean":"2024-04-25T14:39:15.992495+0000","last_became_active":"2024-04-25T14:38:10.431835+0000","last_became_peered":"2024-04-25T14:38:10.431835+0000","last_unstale":"2024-04-25T14:39:15.992495+0000","last_undegraded":"2024-04-25T14:39:15.992495+0000","last_fullsized":"2024-04-25T14:39:15.992495+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:23.089447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.976577+0000","last_change":"2024-04-25T14:38:09.550254+0000","last_active":"2024-04-25T14:39:15.976577+0000","last_peered":"2024-04-25T14:39:15.976577+0000","last_clean":"2024-04-25T14:39:15.976577+0000","last_became_active":"2024-04-25T14:38:09.549958+0000","last_became_peered":"2024-04-25T14:38:09.549958+0000","last_unstale":"2024-04-25T14:39:15.976577+0000","last_undegraded":"2024-04-25T14:39:15.976577+0000","last_fullsized":"2024-04-25T14:39:15.976577+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:56.161427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:16.534628+0000","last_change":"2024-04-25T14:38:10.425648+0000","last_active":"2024-04-25T14:39:16.534628+0000","last_peered":"2024-04-25T14:39:16.534628+0000","last_clean":"2024-04-25T14:39:16.534628+0000","last_became_active":"2024-04-25T14:38:10.425009+0000","last_became_peered":"2024-04-25T14:38:10.425009+0000","last_unstale":"2024-04-25T14:39:16.534628+0000","last_undegraded":"2024-04-25T14:39:16.534628+0000","last_fullsized":"2024-04-25T14:39:16.534628+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:46.696610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.976289+0000","last_change":"2024-04-25T14:38:09.544481+0000","last_active":"2024-04-25T14:39:15.976289+0000","last_peered":"2024-04-25T14:39:15.976289+0000","last_clean":"2024-04-25T14:39:15.976289+0000","last_became_active":"2024-04-25T14:38:09.544147+0000","last_became_peered":"2024-04-25T14:38:09.544147+0000","last_unstale":"2024-04-25T14:39:15.976289+0000","last_undegraded":"2024-04-25T14:39:15.976289+0000","last_fullsized":"2024-04-25T14:39:15.976289+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:35:22.971028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.970180+0000","last_change":"2024-04-25T14:38:10.426851+0000","last_active":"2024-04-25T14:39:15.970180+0000","last_peered":"2024-04-25T14:39:15.970180+0000","last_clean":"2024-04-25T14:39:15.970180+0000","last_became_active":"2024-04-25T14:38:10.426475+0000","last_became_peered":"2024-04-25T14:38:10.426475+0000","last_unstale":"2024-04-25T14:39:15.970180+0000","last_undegraded":"2024-04-25T14:39:15.970180+0000","last_fullsized":"2024-04-25T14:39:15.970180+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:28.213318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":126,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:40.010768+0000","last_change":"2024-04-25T14:39:15.979586+0000","last_active":"2024-04-25T14:39:40.010768+0000","last_peered":"2024-04-25T14:39:40.010768+0000","last_clean":"2024-04-25T14:39:40.010768+0000","last_became_active":"2024-04-25T14:39:15.974397+0000","last_became_peered":"2024-04-25T14:39:15.974397+0000","last_unstale":"2024-04-25T14:39:40.010768+0000","last_undegraded":"2024-04-25T14:39:40.010768+0000","last_fullsized":"2024-04-25T14:39:40.010768+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:02.152202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":87,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.975974+0000","last_change":"2024-04-25T14:38:10.425424+0000","last_active":"2024-04-25T14:39:15.975974+0000","last_peered":"2024-04-25T14:39:15.975974+0000","last_clean":"2024-04-25T14:39:15.975974+0000","last_became_active":"2024-04-25T14:38:10.425101+0000","last_became_peered":"2024-04-25T14:38:10.425101+0000","last_unstale":"2024-04-25T14:39:15.975974+0000","last_undegraded":"2024-04-25T14:39:15.975974+0000","last_fullsized":"2024-04-25T14:39:15.975974+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:52:29.871113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.973226+0000","last_change":"2024-04-25T14:38:09.552287+0000","last_active":"2024-04-25T14:39:15.973226+0000","last_peered":"2024-04-25T14:39:15.973226+0000","last_clean":"2024-04-25T14:39:15.973226+0000","last_became_active":"2024-04-25T14:38:09.551957+0000","last_became_peered":"2024-04-25T14:38:09.551957+0000","last_unstale":"2024-04-25T14:39:15.973226+0000","last_undegraded":"2024-04-25T14:39:15.973226+0000","last_fullsized":"2024-04-25T14:39:15.973226+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:22.586868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.990838+0000","last_change":"2024-04-25T14:39:15.990838+0000","last_active":"2024-04-25T14:39:15.990838+0000","last_peered":"2024-04-25T14:39:15.990838+0000","last_clean":"2024-04-25T14:39:15.990838+0000","last_became_active":"2024-04-25T14:39:15.990370+0000","last_became_peered":"2024-04-25T14:39:15.990370+0000","last_unstale":"2024-04-25T14:39:15.990838+0000","last_undegraded":"2024-04-25T14:39:15.990838+0000","last_fullsized":"2024-04-25T14:39:15.990838+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:50.676958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.975073+0000","last_change":"2024-04-25T14:38:09.552376+0000","last_active":"2024-04-25T14:39:15.975073+0000","last_peered":"2024-04-25T14:39:15.975073+0000","last_clean":"2024-04-25T14:39:15.975073+0000","last_became_active":"2024-04-25T14:38:09.548614+0000","last_became_peered":"2024-04-25T14:38:09.548614+0000","last_unstale":"2024-04-25T14:39:15.975073+0000","last_undegraded":"2024-04-25T14:39:15.975073+0000","last_fullsized":"2024-04-25T14:39:15.975073+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:27:27.133106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.982412+0000","last_change":"2024-04-25T14:38:10.434611+0000","last_active":"2024-04-25T14:39:15.982412+0000","last_peered":"2024-04-25T14:39:15.982412+0000","last_clean":"2024-04-25T14:39:15.982412+0000","last_became_active":"2024-04-25T14:38:10.434280+0000","last_became_peered":"2024-04-25T14:38:10.434280+0000","last_unstale":"2024-04-25T14:39:15.982412+0000","last_undegraded":"2024-04-25T14:39:15.982412+0000","last_fullsized":"2024-04-25T14:39:15.982412+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:12:32.275663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.969820+0000","last_change":"2024-04-25T14:38:09.539743+0000","last_active":"2024-04-25T14:39:15.969820+0000","last_peered":"2024-04-25T14:39:15.969820+0000","last_clean":"2024-04-25T14:39:15.969820+0000","last_became_active":"2024-04-25T14:38:09.539272+0000","last_became_peered":"2024-04-25T14:38:09.539272+0000","last_unstale":"2024-04-25T14:39:15.969820+0000","last_undegraded":"2024-04-25T14:39:15.969820+0000","last_fullsized":"2024-04-25T14:39:15.969820+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:29:23.159901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:44.324693+0000","last_change":"2024-04-25T14:38:10.444899+0000","last_active":"2024-04-25T14:39:44.324693+0000","last_peered":"2024-04-25T14:39:44.324693+0000","last_clean":"2024-04-25T14:39:44.324693+0000","last_became_active":"2024-04-25T14:38:10.444383+0000","last_became_peered":"2024-04-25T14:38:10.444383+0000","last_unstale":"2024-04-25T14:39:44.324693+0000","last_undegraded":"2024-04-25T14:39:44.324693+0000","last_fullsized":"2024-04-25T14:39:44.324693+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:21.351436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":107,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:39.008909+0000","last_change":"2024-04-25T14:38:09.550181+0000","last_active":"2024-04-25T14:39:39.008909+0000","last_peered":"2024-04-25T14:39:39.008909+0000","last_clean":"2024-04-25T14:39:39.008909+0000","last_became_active":"2024-04-25T14:38:09.549824+0000","last_became_peered":"2024-04-25T14:38:09.549824+0000","last_unstale":"2024-04-25T14:39:39.008909+0000","last_undegraded":"2024-04-25T14:39:39.008909+0000","last_fullsized":"2024-04-25T14:39:39.008909+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:59.220917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":85,"num_read_kb":85,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:16.534903+0000","last_change":"2024-04-25T14:38:10.421339+0000","last_active":"2024-04-25T14:39:16.534903+0000","last_peered":"2024-04-25T14:39:16.534903+0000","last_clean":"2024-04-25T14:39:16.534903+0000","last_became_active":"2024-04-25T14:38:10.421005+0000","last_became_peered":"2024-04-25T14:38:10.421005+0000","last_unstale":"2024-04-25T14:39:16.534903+0000","last_undegraded":"2024-04-25T14:39:16.534903+0000","last_fullsized":"2024-04-25T14:39:16.534903+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:51.949054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.973211+0000","last_change":"2024-04-25T14:38:09.548245+0000","last_active":"2024-04-25T14:39:15.973211+0000","last_peered":"2024-04-25T14:39:15.973211+0000","last_clean":"2024-04-25T14:39:15.973211+0000","last_became_active":"2024-04-25T14:38:09.547905+0000","last_became_peered":"2024-04-25T14:38:09.547905+0000","last_unstale":"2024-04-25T14:39:15.973211+0000","last_undegraded":"2024-04-25T14:39:15.973211+0000","last_fullsized":"2024-04-25T14:39:15.973211+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:56.798594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.992319+0000","last_change":"2024-04-25T14:38:10.431427+0000","last_active":"2024-04-25T14:39:15.992319+0000","last_peered":"2024-04-25T14:39:15.992319+0000","last_clean":"2024-04-25T14:39:15.992319+0000","last_became_active":"2024-04-25T14:38:10.428759+0000","last_became_peered":"2024-04-25T14:38:10.428759+0000","last_unstale":"2024-04-25T14:39:15.992319+0000","last_undegraded":"2024-04-25T14:39:15.992319+0000","last_fullsized":"2024-04-25T14:39:15.992319+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:15:27.341445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.970306+0000","last_change":"2024-04-25T14:38:09.556839+0000","last_active":"2024-04-25T14:39:15.970306+0000","last_peered":"2024-04-25T14:39:15.970306+0000","last_clean":"2024-04-25T14:39:15.970306+0000","last_became_active":"2024-04-25T14:38:09.556443+0000","last_became_peered":"2024-04-25T14:38:09.556443+0000","last_unstale":"2024-04-25T14:39:15.970306+0000","last_undegraded":"2024-04-25T14:39:15.970306+0000","last_fullsized":"2024-04-25T14:39:15.970306+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:46.621883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:16.534780+0000","last_change":"2024-04-25T14:38:10.414507+0000","last_active":"2024-04-25T14:39:16.534780+0000","last_peered":"2024-04-25T14:39:16.534780+0000","last_clean":"2024-04-25T14:39:16.534780+0000","last_became_active":"2024-04-25T14:38:10.414154+0000","last_became_peered":"2024-04-25T14:38:10.414154+0000","last_unstale":"2024-04-25T14:39:16.534780+0000","last_undegraded":"2024-04-25T14:39:16.534780+0000","last_fullsized":"2024-04-25T14:39:16.534780+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:12.208774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'28","reported_seq":225,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:40.010403+0000","last_change":"2024-04-25T14:38:09.549239+0000","last_active":"2024-04-25T14:39:40.010403+0000","last_peered":"2024-04-25T14:39:40.010403+0000","last_clean":"2024-04-25T14:39:40.010403+0000","last_became_active":"2024-04-25T14:38:09.548888+0000","last_became_peered":"2024-04-25T14:38:09.548888+0000","last_unstale":"2024-04-25T14:39:40.010403+0000","last_undegraded":"2024-04-25T14:39:40.010403+0000","last_fullsized":"2024-04-25T14:39:40.010403+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:36:46.536908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61896,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":178,"num_read_kb":149,"num_write":29,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.976353+0000","last_change":"2024-04-25T14:38:10.417510+0000","last_active":"2024-04-25T14:39:15.976353+0000","last_peered":"2024-04-25T14:39:15.976353+0000","last_clean":"2024-04-25T14:39:15.976353+0000","last_became_active":"2024-04-25T14:38:10.417079+0000","last_became_peered":"2024-04-25T14:38:10.417079+0000","last_unstale":"2024-04-25T14:39:15.976353+0000","last_undegraded":"2024-04-25T14:39:15.976353+0000","last_fullsized":"2024-04-25T14:39:15.976353+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:35.213199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.972736+0000","last_change":"2024-04-25T14:38:09.563650+0000","last_active":"2024-04-25T14:39:15.972736+0000","last_peered":"2024-04-25T14:39:15.972736+0000","last_clean":"2024-04-25T14:39:15.972736+0000","last_became_active":"2024-04-25T14:38:09.563347+0000","last_became_peered":"2024-04-25T14:38:09.563347+0000","last_unstale":"2024-04-25T14:39:15.972736+0000","last_undegraded":"2024-04-25T14:39:15.972736+0000","last_fullsized":"2024-04-25T14:39:15.972736+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:16.793191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.974525+0000","last_change":"2024-04-25T14:38:10.447504+0000","last_active":"2024-04-25T14:39:15.974525+0000","last_peered":"2024-04-25T14:39:15.974525+0000","last_clean":"2024-04-25T14:39:15.974525+0000","last_became_active":"2024-04-25T14:38:10.447157+0000","last_became_peered":"2024-04-25T14:38:10.447157+0000","last_unstale":"2024-04-25T14:39:15.974525+0000","last_undegraded":"2024-04-25T14:39:15.974525+0000","last_fullsized":"2024-04-25T14:39:15.974525+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:38.940320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.969971+0000","last_change":"2024-04-25T14:38:09.545214+0000","last_active":"2024-04-25T14:39:15.969971+0000","last_peered":"2024-04-25T14:39:15.969971+0000","last_clean":"2024-04-25T14:39:15.969971+0000","last_became_active":"2024-04-25T14:38:09.544903+0000","last_became_peered":"2024-04-25T14:38:09.544903+0000","last_unstale":"2024-04-25T14:39:15.969971+0000","last_undegraded":"2024-04-25T14:39:15.969971+0000","last_fullsized":"2024-04-25T14:39:15.969971+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:58:15.817036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.973508+0000","last_change":"2024-04-25T14:38:10.432017+0000","last_active":"2024-04-25T14:39:15.973508+0000","last_peered":"2024-04-25T14:39:15.973508+0000","last_clean":"2024-04-25T14:39:15.973508+0000","last_became_active":"2024-04-25T14:38:10.431596+0000","last_became_peered":"2024-04-25T14:38:10.431596+0000","last_unstale":"2024-04-25T14:39:15.973508+0000","last_undegraded":"2024-04-25T14:39:15.973508+0000","last_fullsized":"2024-04-25T14:39:15.973508+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:30.653642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'4","reported_seq":115,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:42.834977+0000","last_change":"2024-04-25T14:38:09.545477+0000","last_active":"2024-04-25T14:39:42.834977+0000","last_peered":"2024-04-25T14:39:42.834977+0000","last_clean":"2024-04-25T14:39:42.834977+0000","last_became_active":"2024-04-25T14:38:09.545093+0000","last_became_peered":"2024-04-25T14:38:09.545093+0000","last_unstale":"2024-04-25T14:39:42.834977+0000","last_undegraded":"2024-04-25T14:39:42.834977+0000","last_fullsized":"2024-04-25T14:39:42.834977+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:56.589231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":93,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.973286+0000","last_change":"2024-04-25T14:38:10.418914+0000","last_active":"2024-04-25T14:39:15.973286+0000","last_peered":"2024-04-25T14:39:15.973286+0000","last_clean":"2024-04-25T14:39:15.973286+0000","last_became_active":"2024-04-25T14:38:10.418600+0000","last_became_peered":"2024-04-25T14:38:10.418600+0000","last_unstale":"2024-04-25T14:39:15.973286+0000","last_undegraded":"2024-04-25T14:39:15.973286+0000","last_fullsized":"2024-04-25T14:39:15.973286+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:31.324035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.975733+0000","last_change":"2024-04-25T14:38:09.556574+0000","last_active":"2024-04-25T14:39:15.975733+0000","last_peered":"2024-04-25T14:39:15.975733+0000","last_clean":"2024-04-25T14:39:15.975733+0000","last_became_active":"2024-04-25T14:38:09.556071+0000","last_became_peered":"2024-04-25T14:38:09.556071+0000","last_unstale":"2024-04-25T14:39:15.975733+0000","last_undegraded":"2024-04-25T14:39:15.975733+0000","last_fullsized":"2024-04-25T14:39:15.975733+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:05.424593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.969805+0000","last_change":"2024-04-25T14:38:10.424591+0000","last_active":"2024-04-25T14:39:15.969805+0000","last_peered":"2024-04-25T14:39:15.969805+0000","last_clean":"2024-04-25T14:39:15.969805+0000","last_became_active":"2024-04-25T14:38:10.424258+0000","last_became_peered":"2024-04-25T14:38:10.424258+0000","last_unstale":"2024-04-25T14:39:15.969805+0000","last_undegraded":"2024-04-25T14:39:15.969805+0000","last_fullsized":"2024-04-25T14:39:15.969805+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:22.753595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.976682+0000","last_change":"2024-04-25T14:38:09.546073+0000","last_active":"2024-04-25T14:39:15.976682+0000","last_peered":"2024-04-25T14:39:15.976682+0000","last_clean":"2024-04-25T14:39:15.976682+0000","last_became_active":"2024-04-25T14:38:09.545783+0000","last_became_peered":"2024-04-25T14:38:09.545783+0000","last_unstale":"2024-04-25T14:39:15.976682+0000","last_undegraded":"2024-04-25T14:39:15.976682+0000","last_fullsized":"2024-04-25T14:39:15.976682+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:42.537431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.978465+0000","last_change":"2024-04-25T14:38:10.417230+0000","last_active":"2024-04-25T14:39:15.978465+0000","last_peered":"2024-04-25T14:39:15.978465+0000","last_clean":"2024-04-25T14:39:15.978465+0000","last_became_active":"2024-04-25T14:38:10.416891+0000","last_became_peered":"2024-04-25T14:38:10.416891+0000","last_unstale":"2024-04-25T14:39:15.978465+0000","last_undegraded":"2024-04-25T14:39:15.978465+0000","last_fullsized":"2024-04-25T14:39:15.978465+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:10:13.434518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.994906+0000","last_change":"2024-04-25T14:39:15.994906+0000","last_active":"2024-04-25T14:39:15.994906+0000","last_peered":"2024-04-25T14:39:15.994906+0000","last_clean":"2024-04-25T14:39:15.994906+0000","last_became_active":"2024-04-25T14:39:15.994365+0000","last_became_peered":"2024-04-25T14:39:15.994365+0000","last_unstale":"2024-04-25T14:39:15.994906+0000","last_undegraded":"2024-04-25T14:39:15.994906+0000","last_fullsized":"2024-04-25T14:39:15.994906+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:19.026241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.972917+0000","last_change":"2024-04-25T14:38:10.446846+0000","last_active":"2024-04-25T14:39:15.972917+0000","last_peered":"2024-04-25T14:39:15.972917+0000","last_clean":"2024-04-25T14:39:15.972917+0000","last_became_active":"2024-04-25T14:38:10.446498+0000","last_became_peered":"2024-04-25T14:38:10.446498+0000","last_unstale":"2024-04-25T14:39:15.972917+0000","last_undegraded":"2024-04-25T14:39:15.972917+0000","last_fullsized":"2024-04-25T14:39:15.972917+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:14:37.179198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.972775+0000","last_change":"2024-04-25T14:38:09.544283+0000","last_active":"2024-04-25T14:39:15.972775+0000","last_peered":"2024-04-25T14:39:15.972775+0000","last_clean":"2024-04-25T14:39:15.972775+0000","last_became_active":"2024-04-25T14:38:09.543135+0000","last_became_peered":"2024-04-25T14:38:09.543135+0000","last_unstale":"2024-04-25T14:39:15.972775+0000","last_undegraded":"2024-04-25T14:39:15.972775+0000","last_fullsized":"2024-04-25T14:39:15.972775+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:05.977062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.972870+0000","last_change":"2024-04-25T14:38:10.432863+0000","last_active":"2024-04-25T14:39:15.972870+0000","last_peered":"2024-04-25T14:39:15.972870+0000","last_clean":"2024-04-25T14:39:15.972870+0000","last_became_active":"2024-04-25T14:38:10.432493+0000","last_became_peered":"2024-04-25T14:38:10.432493+0000","last_unstale":"2024-04-25T14:39:15.972870+0000","last_undegraded":"2024-04-25T14:39:15.972870+0000","last_fullsized":"2024-04-25T14:39:15.972870+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:25:53.774764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.981979+0000","last_change":"2024-04-25T14:39:15.981979+0000","last_active":"2024-04-25T14:39:15.981979+0000","last_peered":"2024-04-25T14:39:15.981979+0000","last_clean":"2024-04-25T14:39:15.981979+0000","last_became_active":"2024-04-25T14:39:15.981539+0000","last_became_peered":"2024-04-25T14:39:15.981539+0000","last_unstale":"2024-04-25T14:39:15.981979+0000","last_undegraded":"2024-04-25T14:39:15.981979+0000","last_fullsized":"2024-04-25T14:39:15.981979+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:22.906826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":109,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:39.826472+0000","last_change":"2024-04-25T14:38:09.554305+0000","last_active":"2024-04-25T14:39:39.826472+0000","last_peered":"2024-04-25T14:39:39.826472+0000","last_clean":"2024-04-25T14:39:39.826472+0000","last_became_active":"2024-04-25T14:38:09.553984+0000","last_became_peered":"2024-04-25T14:38:09.553984+0000","last_unstale":"2024-04-25T14:39:39.826472+0000","last_undegraded":"2024-04-25T14:39:39.826472+0000","last_fullsized":"2024-04-25T14:39:39.826472+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:41.580867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":86,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.985001+0000","last_change":"2024-04-25T14:38:10.448290+0000","last_active":"2024-04-25T14:39:15.985001+0000","last_peered":"2024-04-25T14:39:15.985001+0000","last_clean":"2024-04-25T14:39:15.985001+0000","last_became_active":"2024-04-25T14:38:10.446147+0000","last_became_peered":"2024-04-25T14:38:10.446147+0000","last_unstale":"2024-04-25T14:39:15.985001+0000","last_undegraded":"2024-04-25T14:39:15.985001+0000","last_fullsized":"2024-04-25T14:39:15.985001+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:12.775551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.971273+0000","last_change":"2024-04-25T14:38:09.545209+0000","last_active":"2024-04-25T14:39:15.971273+0000","last_peered":"2024-04-25T14:39:15.971273+0000","last_clean":"2024-04-25T14:39:15.971273+0000","last_became_active":"2024-04-25T14:38:09.544861+0000","last_became_peered":"2024-04-25T14:38:09.544861+0000","last_unstale":"2024-04-25T14:39:15.971273+0000","last_undegraded":"2024-04-25T14:39:15.971273+0000","last_fullsized":"2024-04-25T14:39:15.971273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:25.829162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":354,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.973537+0000","last_change":"2024-04-25T14:36:48.250081+0000","last_active":"2024-04-25T14:39:15.973537+0000","last_peered":"2024-04-25T14:39:15.973537+0000","last_clean":"2024-04-25T14:39:15.973537+0000","last_became_active":"2024-04-25T14:36:48.249520+0000","last_became_peered":"2024-04-25T14:36:48.249520+0000","last_unstale":"2024-04-25T14:39:15.973537+0000","last_undegraded":"2024-04-25T14:39:15.973537+0000","last_fullsized":"2024-04-25T14:39:15.973537+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:33:03.061016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:33:03.061016+0000","last_clean_scrub_stamp":"2024-04-25T14:33:03.061016+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:04:06.653326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.973820+0000","last_change":"2024-04-25T14:38:10.422966+0000","last_active":"2024-04-25T14:39:15.973820+0000","last_peered":"2024-04-25T14:39:15.973820+0000","last_clean":"2024-04-25T14:39:15.973820+0000","last_became_active":"2024-04-25T14:38:10.422568+0000","last_became_peered":"2024-04-25T14:38:10.422568+0000","last_unstale":"2024-04-25T14:39:15.973820+0000","last_undegraded":"2024-04-25T14:39:15.973820+0000","last_fullsized":"2024-04-25T14:39:15.973820+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:17.405074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.982120+0000","last_change":"2024-04-25T14:39:15.982120+0000","last_active":"2024-04-25T14:39:15.982120+0000","last_peered":"2024-04-25T14:39:15.982120+0000","last_clean":"2024-04-25T14:39:15.982120+0000","last_became_active":"2024-04-25T14:39:15.975343+0000","last_became_peered":"2024-04-25T14:39:15.975343+0000","last_unstale":"2024-04-25T14:39:15.982120+0000","last_undegraded":"2024-04-25T14:39:15.982120+0000","last_fullsized":"2024-04-25T14:39:15.982120+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:38.632052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.999140+0000","last_change":"2024-04-25T14:39:15.998744+0000","last_active":"2024-04-25T14:39:15.999140+0000","last_peered":"2024-04-25T14:39:15.999140+0000","last_clean":"2024-04-25T14:39:15.999140+0000","last_became_active":"2024-04-25T14:39:15.993416+0000","last_became_peered":"2024-04-25T14:39:15.993416+0000","last_unstale":"2024-04-25T14:39:15.999140+0000","last_undegraded":"2024-04-25T14:39:15.999140+0000","last_fullsized":"2024-04-25T14:39:15.999140+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:24.019707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.976059+0000","last_change":"2024-04-25T14:38:10.429554+0000","last_active":"2024-04-25T14:39:15.976059+0000","last_peered":"2024-04-25T14:39:15.976059+0000","last_clean":"2024-04-25T14:39:15.976059+0000","last_became_active":"2024-04-25T14:38:10.428944+0000","last_became_peered":"2024-04-25T14:38:10.428944+0000","last_unstale":"2024-04-25T14:39:15.976059+0000","last_undegraded":"2024-04-25T14:39:15.976059+0000","last_fullsized":"2024-04-25T14:39:15.976059+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:38.013028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.978926+0000","last_change":"2024-04-25T14:38:09.564849+0000","last_active":"2024-04-25T14:39:15.978926+0000","last_peered":"2024-04-25T14:39:15.978926+0000","last_clean":"2024-04-25T14:39:15.978926+0000","last_became_active":"2024-04-25T14:38:09.564354+0000","last_became_peered":"2024-04-25T14:38:09.564354+0000","last_unstale":"2024-04-25T14:39:15.978926+0000","last_undegraded":"2024-04-25T14:39:15.978926+0000","last_fullsized":"2024-04-25T14:39:15.978926+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:44.308366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.989359+0000","last_change":"2024-04-25T14:38:10.438178+0000","last_active":"2024-04-25T14:39:15.989359+0000","last_peered":"2024-04-25T14:39:15.989359+0000","last_clean":"2024-04-25T14:39:15.989359+0000","last_became_active":"2024-04-25T14:38:10.429936+0000","last_became_peered":"2024-04-25T14:38:10.429936+0000","last_unstale":"2024-04-25T14:39:15.989359+0000","last_undegraded":"2024-04-25T14:39:15.989359+0000","last_fullsized":"2024-04-25T14:39:15.989359+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:33.598747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.970600+0000","last_change":"2024-04-25T14:38:09.547680+0000","last_active":"2024-04-25T14:39:15.970600+0000","last_peered":"2024-04-25T14:39:15.970600+0000","last_clean":"2024-04-25T14:39:15.970600+0000","last_became_active":"2024-04-25T14:38:09.547337+0000","last_became_peered":"2024-04-25T14:38:09.547337+0000","last_unstale":"2024-04-25T14:39:15.970600+0000","last_undegraded":"2024-04-25T14:39:15.970600+0000","last_fullsized":"2024-04-25T14:39:15.970600+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:32.754439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.991740+0000","last_change":"2024-04-25T14:38:10.447115+0000","last_active":"2024-04-25T14:39:15.991740+0000","last_peered":"2024-04-25T14:39:15.991740+0000","last_clean":"2024-04-25T14:39:15.991740+0000","last_became_active":"2024-04-25T14:38:10.446681+0000","last_became_peered":"2024-04-25T14:38:10.446681+0000","last_unstale":"2024-04-25T14:39:15.991740+0000","last_undegraded":"2024-04-25T14:39:15.991740+0000","last_fullsized":"2024-04-25T14:39:15.991740+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:02:58.942053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:16.534479+0000","last_change":"2024-04-25T14:38:09.551523+0000","last_active":"2024-04-25T14:39:16.534479+0000","last_peered":"2024-04-25T14:39:16.534479+0000","last_clean":"2024-04-25T14:39:16.534479+0000","last_became_active":"2024-04-25T14:38:09.551213+0000","last_became_peered":"2024-04-25T14:38:09.551213+0000","last_unstale":"2024-04-25T14:39:16.534479+0000","last_undegraded":"2024-04-25T14:39:16.534479+0000","last_fullsized":"2024-04-25T14:39:16.534479+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:48.948571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.975775+0000","last_change":"2024-04-25T14:38:10.420032+0000","last_active":"2024-04-25T14:39:15.975775+0000","last_peered":"2024-04-25T14:39:15.975775+0000","last_clean":"2024-04-25T14:39:15.975775+0000","last_became_active":"2024-04-25T14:38:10.418349+0000","last_became_peered":"2024-04-25T14:38:10.418349+0000","last_unstale":"2024-04-25T14:39:15.975775+0000","last_undegraded":"2024-04-25T14:39:15.975775+0000","last_fullsized":"2024-04-25T14:39:15.975775+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:38:55.669414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.984235+0000","last_change":"2024-04-25T14:38:09.567097+0000","last_active":"2024-04-25T14:39:15.984235+0000","last_peered":"2024-04-25T14:39:15.984235+0000","last_clean":"2024-04-25T14:39:15.984235+0000","last_became_active":"2024-04-25T14:38:09.566852+0000","last_became_peered":"2024-04-25T14:38:09.566852+0000","last_unstale":"2024-04-25T14:39:15.984235+0000","last_undegraded":"2024-04-25T14:39:15.984235+0000","last_fullsized":"2024-04-25T14:39:15.984235+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:41:55.773005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.978187+0000","last_change":"2024-04-25T14:38:10.432691+0000","last_active":"2024-04-25T14:39:15.978187+0000","last_peered":"2024-04-25T14:39:15.978187+0000","last_clean":"2024-04-25T14:39:15.978187+0000","last_became_active":"2024-04-25T14:38:10.432321+0000","last_became_peered":"2024-04-25T14:38:10.432321+0000","last_unstale":"2024-04-25T14:39:15.978187+0000","last_undegraded":"2024-04-25T14:39:15.978187+0000","last_fullsized":"2024-04-25T14:39:15.978187+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:47:58.026331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.972673+0000","last_change":"2024-04-25T14:38:09.563692+0000","last_active":"2024-04-25T14:39:15.972673+0000","last_peered":"2024-04-25T14:39:15.972673+0000","last_clean":"2024-04-25T14:39:15.972673+0000","last_became_active":"2024-04-25T14:38:09.563345+0000","last_became_peered":"2024-04-25T14:38:09.563345+0000","last_unstale":"2024-04-25T14:39:15.972673+0000","last_undegraded":"2024-04-25T14:39:15.972673+0000","last_fullsized":"2024-04-25T14:39:15.972673+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:43.221429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.984407+0000","last_change":"2024-04-25T14:38:10.442519+0000","last_active":"2024-04-25T14:39:15.984407+0000","last_peered":"2024-04-25T14:39:15.984407+0000","last_clean":"2024-04-25T14:39:15.984407+0000","last_became_active":"2024-04-25T14:38:10.439045+0000","last_became_peered":"2024-04-25T14:38:10.439045+0000","last_unstale":"2024-04-25T14:39:15.984407+0000","last_undegraded":"2024-04-25T14:39:15.984407+0000","last_fullsized":"2024-04-25T14:39:15.984407+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:42.425106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.976059+0000","last_change":"2024-04-25T14:38:09.550363+0000","last_active":"2024-04-25T14:39:15.976059+0000","last_peered":"2024-04-25T14:39:15.976059+0000","last_clean":"2024-04-25T14:39:15.976059+0000","last_became_active":"2024-04-25T14:38:09.550044+0000","last_became_peered":"2024-04-25T14:38:09.550044+0000","last_unstale":"2024-04-25T14:39:15.976059+0000","last_undegraded":"2024-04-25T14:39:15.976059+0000","last_fullsized":"2024-04-25T14:39:15.976059+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:25.573353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.972986+0000","last_change":"2024-04-25T14:38:10.428736+0000","last_active":"2024-04-25T14:39:15.972986+0000","last_peered":"2024-04-25T14:39:15.972986+0000","last_clean":"2024-04-25T14:39:15.972986+0000","last_became_active":"2024-04-25T14:38:10.428410+0000","last_became_peered":"2024-04-25T14:38:10.428410+0000","last_unstale":"2024-04-25T14:39:15.972986+0000","last_undegraded":"2024-04-25T14:39:15.972986+0000","last_fullsized":"2024-04-25T14:39:15.972986+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:02:51.474220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.972844+0000","last_change":"2024-04-25T14:38:09.552800+0000","last_active":"2024-04-25T14:39:15.972844+0000","last_peered":"2024-04-25T14:39:15.972844+0000","last_clean":"2024-04-25T14:39:15.972844+0000","last_became_active":"2024-04-25T14:38:09.552460+0000","last_became_peered":"2024-04-25T14:38:09.552460+0000","last_unstale":"2024-04-25T14:39:15.972844+0000","last_undegraded":"2024-04-25T14:39:15.972844+0000","last_fullsized":"2024-04-25T14:39:15.972844+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:00.800770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.973424+0000","last_change":"2024-04-25T14:38:10.424359+0000","last_active":"2024-04-25T14:39:15.973424+0000","last_peered":"2024-04-25T14:39:15.973424+0000","last_clean":"2024-04-25T14:39:15.973424+0000","last_became_active":"2024-04-25T14:38:10.424020+0000","last_became_peered":"2024-04-25T14:38:10.424020+0000","last_unstale":"2024-04-25T14:39:15.973424+0000","last_undegraded":"2024-04-25T14:39:15.973424+0000","last_fullsized":"2024-04-25T14:39:15.973424+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:14.700772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.970176+0000","last_change":"2024-04-25T14:38:09.545637+0000","last_active":"2024-04-25T14:39:15.970176+0000","last_peered":"2024-04-25T14:39:15.970176+0000","last_clean":"2024-04-25T14:39:15.970176+0000","last_became_active":"2024-04-25T14:38:09.545319+0000","last_became_peered":"2024-04-25T14:38:09.545319+0000","last_unstale":"2024-04-25T14:39:15.970176+0000","last_undegraded":"2024-04-25T14:39:15.970176+0000","last_fullsized":"2024-04-25T14:39:15.970176+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:02.284715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.973079+0000","last_change":"2024-04-25T14:38:10.446928+0000","last_active":"2024-04-25T14:39:15.973079+0000","last_peered":"2024-04-25T14:39:15.973079+0000","last_clean":"2024-04-25T14:39:15.973079+0000","last_became_active":"2024-04-25T14:38:10.446518+0000","last_became_peered":"2024-04-25T14:38:10.446518+0000","last_unstale":"2024-04-25T14:39:15.973079+0000","last_undegraded":"2024-04-25T14:39:15.973079+0000","last_fullsized":"2024-04-25T14:39:15.973079+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:40.769059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.976728+0000","last_change":"2024-04-25T14:38:09.557381+0000","last_active":"2024-04-25T14:39:15.976728+0000","last_peered":"2024-04-25T14:39:15.976728+0000","last_clean":"2024-04-25T14:39:15.976728+0000","last_became_active":"2024-04-25T14:38:09.556504+0000","last_became_peered":"2024-04-25T14:38:09.556504+0000","last_unstale":"2024-04-25T14:39:15.976728+0000","last_undegraded":"2024-04-25T14:39:15.976728+0000","last_fullsized":"2024-04-25T14:39:15.976728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:01:02.761249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.973396+0000","last_change":"2024-04-25T14:38:10.418104+0000","last_active":"2024-04-25T14:39:15.973396+0000","last_peered":"2024-04-25T14:39:15.973396+0000","last_clean":"2024-04-25T14:39:15.973396+0000","last_became_active":"2024-04-25T14:38:10.417774+0000","last_became_peered":"2024-04-25T14:38:10.417774+0000","last_unstale":"2024-04-25T14:39:15.973396+0000","last_undegraded":"2024-04-25T14:39:15.973396+0000","last_fullsized":"2024-04-25T14:39:15.973396+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:32:44.229899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.978293+0000","last_change":"2024-04-25T14:38:10.423546+0000","last_active":"2024-04-25T14:39:15.978293+0000","last_peered":"2024-04-25T14:39:15.978293+0000","last_clean":"2024-04-25T14:39:15.978293+0000","last_became_active":"2024-04-25T14:38:10.423216+0000","last_became_peered":"2024-04-25T14:38:10.423216+0000","last_unstale":"2024-04-25T14:39:15.978293+0000","last_undegraded":"2024-04-25T14:39:15.978293+0000","last_fullsized":"2024-04-25T14:39:15.978293+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:58.640669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.972779+0000","last_change":"2024-04-25T14:38:10.422504+0000","last_active":"2024-04-25T14:39:15.972779+0000","last_peered":"2024-04-25T14:39:15.972779+0000","last_clean":"2024-04-25T14:39:15.972779+0000","last_became_active":"2024-04-25T14:38:10.422095+0000","last_became_peered":"2024-04-25T14:38:10.422095+0000","last_unstale":"2024-04-25T14:39:15.972779+0000","last_undegraded":"2024-04-25T14:39:15.972779+0000","last_fullsized":"2024-04-25T14:39:15.972779+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:25.795962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.972439+0000","last_change":"2024-04-25T14:38:10.447446+0000","last_active":"2024-04-25T14:39:15.972439+0000","last_peered":"2024-04-25T14:39:15.972439+0000","last_clean":"2024-04-25T14:39:15.972439+0000","last_became_active":"2024-04-25T14:38:10.447056+0000","last_became_peered":"2024-04-25T14:38:10.447056+0000","last_unstale":"2024-04-25T14:39:15.972439+0000","last_undegraded":"2024-04-25T14:39:15.972439+0000","last_fullsized":"2024-04-25T14:39:15.972439+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:40.462340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.990523+0000","last_change":"2024-04-25T14:38:10.433444+0000","last_active":"2024-04-25T14:39:15.990523+0000","last_peered":"2024-04-25T14:39:15.990523+0000","last_clean":"2024-04-25T14:39:15.990523+0000","last_became_active":"2024-04-25T14:38:10.433153+0000","last_became_peered":"2024-04-25T14:38:10.433153+0000","last_unstale":"2024-04-25T14:39:15.990523+0000","last_undegraded":"2024-04-25T14:39:15.990523+0000","last_fullsized":"2024-04-25T14:39:15.990523+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:10.722650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.975940+0000","last_change":"2024-04-25T14:38:10.438322+0000","last_active":"2024-04-25T14:39:15.975940+0000","last_peered":"2024-04-25T14:39:15.975940+0000","last_clean":"2024-04-25T14:39:15.975940+0000","last_became_active":"2024-04-25T14:38:10.436103+0000","last_became_peered":"2024-04-25T14:38:10.436103+0000","last_unstale":"2024-04-25T14:39:15.975940+0000","last_undegraded":"2024-04-25T14:39:15.975940+0000","last_fullsized":"2024-04-25T14:39:15.975940+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:16.790202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.978611+0000","last_change":"2024-04-25T14:38:10.419710+0000","last_active":"2024-04-25T14:39:15.978611+0000","last_peered":"2024-04-25T14:39:15.978611+0000","last_clean":"2024-04-25T14:39:15.978611+0000","last_became_active":"2024-04-25T14:38:10.419406+0000","last_became_peered":"2024-04-25T14:38:10.419406+0000","last_unstale":"2024-04-25T14:39:15.978611+0000","last_undegraded":"2024-04-25T14:39:15.978611+0000","last_fullsized":"2024-04-25T14:39:15.978611+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:37.964698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.972816+0000","last_change":"2024-04-25T14:38:10.429510+0000","last_active":"2024-04-25T14:39:15.972816+0000","last_peered":"2024-04-25T14:39:15.972816+0000","last_clean":"2024-04-25T14:39:15.972816+0000","last_became_active":"2024-04-25T14:38:10.429219+0000","last_became_peered":"2024-04-25T14:38:10.429219+0000","last_unstale":"2024-04-25T14:39:15.972816+0000","last_undegraded":"2024-04-25T14:39:15.972816+0000","last_fullsized":"2024-04-25T14:39:15.972816+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:52:46.673126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.989155+0000","last_change":"2024-04-25T14:38:10.441487+0000","last_active":"2024-04-25T14:39:15.989155+0000","last_peered":"2024-04-25T14:39:15.989155+0000","last_clean":"2024-04-25T14:39:15.989155+0000","last_became_active":"2024-04-25T14:38:10.441111+0000","last_became_peered":"2024-04-25T14:38:10.441111+0000","last_unstale":"2024-04-25T14:39:15.989155+0000","last_undegraded":"2024-04-25T14:39:15.989155+0000","last_fullsized":"2024-04-25T14:39:15.989155+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:59.842687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.970065+0000","last_change":"2024-04-25T14:38:10.436511+0000","last_active":"2024-04-25T14:39:15.970065+0000","last_peered":"2024-04-25T14:39:15.970065+0000","last_clean":"2024-04-25T14:39:15.970065+0000","last_became_active":"2024-04-25T14:38:10.436171+0000","last_became_peered":"2024-04-25T14:38:10.436171+0000","last_unstale":"2024-04-25T14:39:15.970065+0000","last_undegraded":"2024-04-25T14:39:15.970065+0000","last_fullsized":"2024-04-25T14:39:15.970065+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:27:19.605907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.983017+0000","last_change":"2024-04-25T14:38:10.433624+0000","last_active":"2024-04-25T14:39:15.983017+0000","last_peered":"2024-04-25T14:39:15.983017+0000","last_clean":"2024-04-25T14:39:15.983017+0000","last_became_active":"2024-04-25T14:38:10.433258+0000","last_became_peered":"2024-04-25T14:38:10.433258+0000","last_unstale":"2024-04-25T14:39:15.983017+0000","last_undegraded":"2024-04-25T14:39:15.983017+0000","last_fullsized":"2024-04-25T14:39:15.983017+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:53.563110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:16.534632+0000","last_change":"2024-04-25T14:38:10.446924+0000","last_active":"2024-04-25T14:39:16.534632+0000","last_peered":"2024-04-25T14:39:16.534632+0000","last_clean":"2024-04-25T14:39:16.534632+0000","last_became_active":"2024-04-25T14:38:10.446553+0000","last_became_peered":"2024-04-25T14:38:10.446553+0000","last_unstale":"2024-04-25T14:39:16.534632+0000","last_undegraded":"2024-04-25T14:39:16.534632+0000","last_fullsized":"2024-04-25T14:39:16.534632+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:00.203796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.974877+0000","last_change":"2024-04-25T14:38:10.429952+0000","last_active":"2024-04-25T14:39:15.974877+0000","last_peered":"2024-04-25T14:39:15.974877+0000","last_clean":"2024-04-25T14:39:15.974877+0000","last_became_active":"2024-04-25T14:38:10.429626+0000","last_became_peered":"2024-04-25T14:38:10.429626+0000","last_unstale":"2024-04-25T14:39:15.974877+0000","last_undegraded":"2024-04-25T14:39:15.974877+0000","last_fullsized":"2024-04-25T14:39:15.974877+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:07.287113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T14:39:15.978351+0000","last_change":"2024-04-25T14:38:10.418292+0000","last_active":"2024-04-25T14:39:15.978351+0000","last_peered":"2024-04-25T14:39:15.978351+0000","last_clean":"2024-04-25T14:39:15.978351+0000","last_became_active":"2024-04-25T14:38:10.418005+0000","last_became_peered":"2024-04-25T14:38:10.418005+0000","last_unstale":"2024-04-25T14:39:15.978351+0000","last_undegraded":"2024-04-25T14:39:15.978351+0000","last_fullsized":"2024-04-25T14:39:15.978351+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:25.486993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28,"ondisk_log_size":28,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66288,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":541,"num_read_kb":339,"num_write":161,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1035,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":198864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285602,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":987136,"data_stored":469673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547238,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28664,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716488,"statfs":{"total":95995035648,"available":95965683712,"internally_reserved":0,"allocated":1679360,"data_stored":1123419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710764,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":469887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972403,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":995328,"data_stored":469785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266742,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28576,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716576,"statfs":{"total":95995035648,"available":95965773824,"internally_reserved":0,"allocated":1593344,"data_stored":1062128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430267,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":469276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":530965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1007616,"data_stored":470547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1064960,"data_stored":531693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149773,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":469587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888138,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":999424,"data_stored":471117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691907,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32748,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712404,"statfs":{"total":95995035648,"available":95961501696,"internally_reserved":0,"allocated":1605632,"data_stored":1061421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":62624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:39:45.645 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T14:39:45.646 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T14:39:45.646 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-25T14:39:45.646 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-25T14:39:46.168 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:39:46.392 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:46 smithi053 ceph-mon[27773]: pgmap v365: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T14:39:46.393 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:46 smithi053 ceph-mon[27773]: from='client.24902 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:39:46.393 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:46 smithi053 ceph-mon[27773]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-25T14:39:46.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:46 smithi183 ceph-mon[32584]: pgmap v365: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T14:39:46.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:46 smithi183 ceph-mon[32584]: from='client.24902 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:39:46.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:46 smithi183 ceph-mon[32584]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-25T14:39:46.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:46 smithi150 ceph-mon[32504]: pgmap v365: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T14:39:46.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:46 smithi150 ceph-mon[32504]: from='client.24902 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:39:46.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:46 smithi150 ceph-mon[32504]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-25T14:39:48.346 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:48 smithi053 ceph-mon[27773]: pgmap v366: 97 pgs: 97 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T14:39:48.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:48 smithi183 ceph-mon[32584]: pgmap v366: 97 pgs: 97 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T14:39:48.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:48 smithi150 ceph-mon[32504]: pgmap v366: 97 pgs: 97 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T14:39:48.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.635+0000 7f7e88c00700 1 -- 172.21.15.53:0/485531554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e8410f4f0 msgr2=0x7f7e841118e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:48.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.635+0000 7f7e88c00700 1 --2- 172.21.15.53:0/485531554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e8410f4f0 0x7f7e841118e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c00b0a0 tx=0x7f7e7c00b3b0 comp rx=0 tx=0).stop 2024-04-25T14:39:48.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.636+0000 7f7e88c00700 1 -- 172.21.15.53:0/485531554 shutdown_connections 2024-04-25T14:39:48.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.636+0000 7f7e88c00700 1 --2- 172.21.15.53:0/485531554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e8410f4f0 0x7f7e841118e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:48.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.636+0000 7f7e88c00700 1 --2- 172.21.15.53:0/485531554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e8406a750 0x7f7e8406ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:48.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.636+0000 7f7e88c00700 1 --2- 172.21.15.53:0/485531554 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e8406b070 0x7f7e8410efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:48.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.636+0000 7f7e88c00700 1 -- 172.21.15.53:0/485531554 >> 172.21.15.53:0/485531554 conn(0x7f7e840788f0 msgr2=0x7f7e84078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:48.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.636+0000 7f7e88c00700 1 -- 172.21.15.53:0/485531554 shutdown_connections 2024-04-25T14:39:48.638 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.636+0000 7f7e88c00700 1 -- 172.21.15.53:0/485531554 wait complete. 2024-04-25T14:39:48.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.637+0000 7f7e88c00700 1 Processor -- start 2024-04-25T14:39:48.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.637+0000 7f7e88c00700 1 -- start start 2024-04-25T14:39:48.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.637+0000 7f7e88c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e8406a750 0x7f7e8419b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:48.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.637+0000 7f7e88c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e8406b070 0x7f7e8419b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:48.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.637+0000 7f7e88c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e8410f4f0 0x7f7e8419fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:48.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.637+0000 7f7e88c00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7e84114e40 con 0x7f7e8410f4f0 2024-04-25T14:39:48.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.637+0000 7f7e88c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7e84114cc0 con 0x7f7e8406a750 2024-04-25T14:39:48.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.637+0000 7f7e88c00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7e84114fc0 con 0x7f7e8406b070 2024-04-25T14:39:48.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.638+0000 7f7e82800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e8410f4f0 0x7f7e8419fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:48.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.638+0000 7f7e81400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e8406b070 0x7f7e8419b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:48.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.638+0000 7f7e81e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e8406a750 0x7f7e8419b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:48.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.638+0000 7f7e82800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e8410f4f0 0x7f7e8419fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40334/0 (socket says 172.21.15.53:40334) 2024-04-25T14:39:48.640 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.638+0000 7f7e81400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e8406b070 0x7f7e8419b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:46052/0 (socket says 172.21.15.53:46052) 2024-04-25T14:39:48.640 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.638+0000 7f7e82800700 1 -- 172.21.15.53:0/3211727750 learned_addr learned my addr 172.21.15.53:0/3211727750 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:48.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.638+0000 7f7e81400700 1 -- 172.21.15.53:0/3211727750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e8406a750 msgr2=0x7f7e8419b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:48.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.638+0000 7f7e81400700 1 --2- 172.21.15.53:0/3211727750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e8406a750 0x7f7e8419b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:48.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.638+0000 7f7e81400700 1 -- 172.21.15.53:0/3211727750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e8410f4f0 msgr2=0x7f7e8419fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:48.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.638+0000 7f7e81400700 1 --2- 172.21.15.53:0/3211727750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e8410f4f0 0x7f7e8419fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:48.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.638+0000 7f7e81400700 1 -- 172.21.15.53:0/3211727750 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e841a0310 con 0x7f7e8406b070 2024-04-25T14:39:48.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.638+0000 7f7e82800700 1 --2- 172.21.15.53:0/3211727750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e8410f4f0 0x7f7e8419fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:39:48.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.639+0000 7f7e81400700 1 --2- 172.21.15.53:0/3211727750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e8406b070 0x7f7e8419b890 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7e7000e9f0 tx=0x7f7e7000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:48.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.639+0000 7f7e7aa00700 1 -- 172.21.15.53:0/3211727750 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7000c9c0 con 0x7f7e8406b070 2024-04-25T14:39:48.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.639+0000 7f7e88c00700 1 -- 172.21.15.53:0/3211727750 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e841a0600 con 0x7f7e8406b070 2024-04-25T14:39:48.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.639+0000 7f7e7aa00700 1 -- 172.21.15.53:0/3211727750 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e7000cb60 con 0x7f7e8406b070 2024-04-25T14:39:48.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.639+0000 7f7e7aa00700 1 -- 172.21.15.53:0/3211727750 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e70018680 con 0x7f7e8406b070 2024-04-25T14:39:48.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.639+0000 7f7e88c00700 1 -- 172.21.15.53:0/3211727750 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e841a7e80 con 0x7f7e8406b070 2024-04-25T14:39:48.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.640+0000 7f7e88c00700 1 -- 172.21.15.53:0/3211727750 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e84069730 con 0x7f7e8406b070 2024-04-25T14:39:48.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.642+0000 7f7e7aa00700 1 -- 172.21.15.53:0/3211727750 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7e70010430 con 0x7f7e8406b070 2024-04-25T14:39:48.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.644+0000 7f7e7aa00700 1 --2- 172.21.15.53:0/3211727750 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f7e680a3c40 0x7f7e680a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:48.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.644+0000 7f7e7aa00700 1 -- 172.21.15.53:0/3211727750 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f7e700d0510 con 0x7f7e8406b070 2024-04-25T14:39:48.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.644+0000 7f7e81e00700 1 --2- 172.21.15.53:0/3211727750 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f7e680a3c40 0x7f7e680a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:48.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.645+0000 7f7e81e00700 1 --2- 172.21.15.53:0/3211727750 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f7e680a3c40 0x7f7e680a6100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7e74005950 tx=0x7f7e7400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:48.649 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.648+0000 7f7e7aa00700 1 -- 172.21.15.53:0/3211727750 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7e7009ce80 con 0x7f7e8406b070 2024-04-25T14:39:48.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.933+0000 7f7e88c00700 1 -- 172.21.15.53:0/3211727750 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f7e8402df80 con 0x7f7e8406b070 2024-04-25T14:39:48.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.934+0000 7f7e7aa00700 1 -- 172.21.15.53:0/3211727750 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f7e7009c780 con 0x7f7e8406b070 2024-04-25T14:39:48.935 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:39:48.935 INFO:teuthology.orchestra.run.smithi053.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T14:39:48.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.937+0000 7f7e88c00700 1 -- 172.21.15.53:0/3211727750 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f7e680a3c40 msgr2=0x7f7e680a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:48.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.937+0000 7f7e88c00700 1 --2- 172.21.15.53:0/3211727750 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f7e680a3c40 0x7f7e680a6100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7e74005950 tx=0x7f7e7400b410 comp rx=0 tx=0).stop 2024-04-25T14:39:48.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.937+0000 7f7e88c00700 1 -- 172.21.15.53:0/3211727750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e8406b070 msgr2=0x7f7e8419b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:48.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.937+0000 7f7e88c00700 1 --2- 172.21.15.53:0/3211727750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e8406b070 0x7f7e8419b890 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7e7000e9f0 tx=0x7f7e7000ed00 comp rx=0 tx=0).stop 2024-04-25T14:39:48.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.937+0000 7f7e88c00700 1 -- 172.21.15.53:0/3211727750 shutdown_connections 2024-04-25T14:39:48.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.937+0000 7f7e88c00700 1 --2- 172.21.15.53:0/3211727750 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f7e680a3c40 0x7f7e680a6100 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:48.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.937+0000 7f7e88c00700 1 --2- 172.21.15.53:0/3211727750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e8410f4f0 0x7f7e8419fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:48.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.937+0000 7f7e88c00700 1 --2- 172.21.15.53:0/3211727750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e8406a750 0x7f7e8419b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:48.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.937+0000 7f7e88c00700 1 --2- 172.21.15.53:0/3211727750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e8406b070 0x7f7e8419b890 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:48.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.937+0000 7f7e88c00700 1 -- 172.21.15.53:0/3211727750 >> 172.21.15.53:0/3211727750 conn(0x7f7e840788f0 msgr2=0x7f7e841109e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:48.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.937+0000 7f7e88c00700 1 -- 172.21.15.53:0/3211727750 shutdown_connections 2024-04-25T14:39:48.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:48.937+0000 7f7e88c00700 1 -- 172.21.15.53:0/3211727750 wait complete. 2024-04-25T14:39:49.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:49 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/3211727750' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T14:39:49.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:49 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/3211727750' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T14:39:49.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:49 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/3211727750' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T14:39:49.813 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-25T14:39:49.814 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:39:50.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.033+0000 7f9168000640 1 -- 172.21.15.53:0/3733750644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91601041b0 msgr2=0x7f91601045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:50.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.033+0000 7f9168000640 1 --2- 172.21.15.53:0/3733750644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91601041b0 0x7f91601045d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9154009860 tx=0x7f915402a340 comp rx=0 tx=0).stop 2024-04-25T14:39:50.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.034+0000 7f9168000640 1 -- 172.21.15.53:0/3733750644 shutdown_connections 2024-04-25T14:39:50.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.034+0000 7f9168000640 1 --2- 172.21.15.53:0/3733750644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9160105d60 0x7f916010c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:50.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.034+0000 7f9168000640 1 --2- 172.21.15.53:0/3733750644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91601053a0 0x7f9160105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:50.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.034+0000 7f9168000640 1 --2- 172.21.15.53:0/3733750644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91601041b0 0x7f91601045d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:50.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.034+0000 7f9168000640 1 -- 172.21.15.53:0/3733750644 >> 172.21.15.53:0/3733750644 conn(0x7f91600ff740 msgr2=0x7f9160101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:50.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.034+0000 7f9168000640 1 -- 172.21.15.53:0/3733750644 shutdown_connections 2024-04-25T14:39:50.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.034+0000 7f9168000640 1 -- 172.21.15.53:0/3733750644 wait complete. 2024-04-25T14:39:50.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.035+0000 7f9168000640 1 Processor -- start 2024-04-25T14:39:50.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.035+0000 7f9168000640 1 -- start start 2024-04-25T14:39:50.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.035+0000 7f9168000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91601041b0 0x7f916019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:50.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.035+0000 7f9168000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91601053a0 0x7f916019a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:50.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.035+0000 7f9168000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9160105d60 0x7f91601a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:50.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.035+0000 7f9168000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f916010fcc0 con 0x7f9160105d60 2024-04-25T14:39:50.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.035+0000 7f9168000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f916010fb40 con 0x7f91601053a0 2024-04-25T14:39:50.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.035+0000 7f9168000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f916010fe40 con 0x7f91601041b0 2024-04-25T14:39:50.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.036+0000 7f9165600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91601041b0 0x7f916019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:50.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.036+0000 7f9164c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91601053a0 0x7f916019a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:50.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.036+0000 7f9165600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91601041b0 0x7f916019a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:46074/0 (socket says 172.21.15.53:46074) 2024-04-25T14:39:50.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.036+0000 7f9165600640 1 -- 172.21.15.53:0/2193553570 learned_addr learned my addr 172.21.15.53:0/2193553570 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:50.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.036+0000 7f9166000640 1 --2- 172.21.15.53:0/2193553570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9160105d60 0x7f91601a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:50.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.036+0000 7f9164c00640 1 -- 172.21.15.53:0/2193553570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91601041b0 msgr2=0x7f916019a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:50.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.036+0000 7f9164c00640 1 --2- 172.21.15.53:0/2193553570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91601041b0 0x7f916019a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:50.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.036+0000 7f9164c00640 1 -- 172.21.15.53:0/2193553570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9160105d60 msgr2=0x7f91601a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:50.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.036+0000 7f9164c00640 1 --2- 172.21.15.53:0/2193553570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9160105d60 0x7f91601a19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:50.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.036+0000 7f9164c00640 1 -- 172.21.15.53:0/2193553570 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91601a2010 con 0x7f91601053a0 2024-04-25T14:39:50.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.036+0000 7f9165600640 1 --2- 172.21.15.53:0/2193553570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91601041b0 0x7f916019a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:39:50.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.036+0000 7f9164c00640 1 --2- 172.21.15.53:0/2193553570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91601053a0 0x7f916019a930 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f914c00e9f0 tx=0x7f914c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:50.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.037+0000 7f915e000640 1 -- 172.21.15.53:0/2193553570 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f914c00ccc0 con 0x7f91601053a0 2024-04-25T14:39:50.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.037+0000 7f9168000640 1 -- 172.21.15.53:0/2193553570 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91601a2300 con 0x7f91601053a0 2024-04-25T14:39:50.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.037+0000 7f9168000640 1 -- 172.21.15.53:0/2193553570 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91601a28a0 con 0x7f91601053a0 2024-04-25T14:39:50.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.037+0000 7f915e000640 1 -- 172.21.15.53:0/2193553570 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f914c0108e0 con 0x7f91601053a0 2024-04-25T14:39:50.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.037+0000 7f915e000640 1 -- 172.21.15.53:0/2193553570 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f914c018750 con 0x7f91601053a0 2024-04-25T14:39:50.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.038+0000 7f9168000640 1 -- 172.21.15.53:0/2193553570 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91400052c0 con 0x7f91601053a0 2024-04-25T14:39:50.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.041+0000 7f915e000640 1 -- 172.21.15.53:0/2193553570 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f914c010430 con 0x7f91601053a0 2024-04-25T14:39:50.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.042+0000 7f915e000640 1 --2- 172.21.15.53:0/2193553570 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f91480a3c30 0x7f91480a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:50.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.042+0000 7f915e000640 1 -- 172.21.15.53:0/2193553570 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f914c0d05f0 con 0x7f91601053a0 2024-04-25T14:39:50.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.043+0000 7f9165600640 1 --2- 172.21.15.53:0/2193553570 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f91480a3c30 0x7f91480a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:50.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.043+0000 7f9165600640 1 --2- 172.21.15.53:0/2193553570 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f91480a3c30 0x7f91480a60f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f91540095f0 tx=0x7f9154030450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:50.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.044+0000 7f915e000640 1 -- 172.21.15.53:0/2193553570 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f914c09c910 con 0x7f91601053a0 2024-04-25T14:39:50.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.254+0000 7f9168000640 1 -- 172.21.15.53:0/2193553570 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9140005880 con 0x7f91601053a0 2024-04-25T14:39:50.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.256+0000 7f915e000640 1 -- 172.21.15.53:0/2193553570 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22668 (secure 0 0 0) 0x7f914c0a0c10 con 0x7f91601053a0 2024-04-25T14:39:50.258 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:39:50.258 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":90,"fsid":"fd30e6ce-030f-11ef-bc93-c7b262605968","created":"2024-04-25T14:28:54.330326+0000","modified":"2024-04-25T14:39:15.964081+0000","last_up_change":"2024-04-25T14:37:10.921148+0000","last_in_change":"2024-04-25T14:36:49.566459+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:33:02.338115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:38:07.517974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:38:08.994122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aecfbaf0-39da-4749-a49b-c7205dabd4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6803","nonce":464176015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6805","nonce":464176015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6809","nonce":464176015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6807","nonce":464176015}]},"public_addr":"172.21.15.53:6803/464176015","cluster_addr":"172.21.15.53:6805/464176015","heartbeat_back_addr":"172.21.15.53:6809/464176015","heartbeat_front_addr":"172.21.15.53:6807/464176015","state":["exists","up"]},{"osd":1,"uuid":"2c4a21ae-c43e-4839-8a56-c3fc63aeb7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6801","nonce":1304621982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6803","nonce":1304621982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6807","nonce":1304621982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6805","nonce":1304621982}]},"public_addr":"172.21.15.150:6801/1304621982","cluster_addr":"172.21.15.150:6803/1304621982","heartbeat_back_addr":"172.21.15.150:6807/1304621982","heartbeat_front_addr":"172.21.15.150:6805/1304621982","state":["exists","up"]},{"osd":2,"uuid":"e8effd43-adc0-472a-9eed-a9273b06dc0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6801","nonce":574509311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6803","nonce":574509311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6807","nonce":574509311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6805","nonce":574509311}]},"public_addr":"172.21.15.183:6801/574509311","cluster_addr":"172.21.15.183:6803/574509311","heartbeat_back_addr":"172.21.15.183:6807/574509311","heartbeat_front_addr":"172.21.15.183:6805/574509311","state":["exists","up"]},{"osd":3,"uuid":"e7cfced4-9b40-40de-a303-75ea8d5df4a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6811","nonce":570672219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6813","nonce":570672219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6817","nonce":570672219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6815","nonce":570672219}]},"public_addr":"172.21.15.53:6811/570672219","cluster_addr":"172.21.15.53:6813/570672219","heartbeat_back_addr":"172.21.15.53:6817/570672219","heartbeat_front_addr":"172.21.15.53:6815/570672219","state":["exists","up"]},{"osd":4,"uuid":"44a9e070-2282-43f1-9f05-b1f8ff81e540","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6809","nonce":1526850103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6811","nonce":1526850103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6815","nonce":1526850103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6813","nonce":1526850103}]},"public_addr":"172.21.15.150:6809/1526850103","cluster_addr":"172.21.15.150:6811/1526850103","heartbeat_back_addr":"172.21.15.150:6815/1526850103","heartbeat_front_addr":"172.21.15.150:6813/1526850103","state":["exists","up"]},{"osd":5,"uuid":"2c5619d2-c0f9-47d6-bf7e-2f4afd1b09f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6809","nonce":773726576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6811","nonce":773726576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6815","nonce":773726576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6813","nonce":773726576}]},"public_addr":"172.21.15.183:6809/773726576","cluster_addr":"172.21.15.183:6811/773726576","heartbeat_back_addr":"172.21.15.183:6815/773726576","heartbeat_front_addr":"172.21.15.183:6813/773726576","state":["exists","up"]},{"osd":6,"uuid":"1cf74d94-c118-4738-b6db-040af458a8eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6819","nonce":355884319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6821","nonce":355884319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6825","nonce":355884319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6823","nonce":355884319}]},"public_addr":"172.21.15.53:6819/355884319","cluster_addr":"172.21.15.53:6821/355884319","heartbeat_back_addr":"172.21.15.53:6825/355884319","heartbeat_front_addr":"172.21.15.53:6823/355884319","state":["exists","up"]},{"osd":7,"uuid":"0ce97c04-cd41-4080-bbbc-be9cf5fadc02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6817","nonce":229022562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6819","nonce":229022562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6823","nonce":229022562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6821","nonce":229022562}]},"public_addr":"172.21.15.150:6817/229022562","cluster_addr":"172.21.15.150:6819/229022562","heartbeat_back_addr":"172.21.15.150:6823/229022562","heartbeat_front_addr":"172.21.15.150:6821/229022562","state":["exists","up"]},{"osd":8,"uuid":"5d57787d-2a7c-4514-bd01-357ced413f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6817","nonce":301003844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6819","nonce":301003844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6823","nonce":301003844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6821","nonce":301003844}]},"public_addr":"172.21.15.183:6817/301003844","cluster_addr":"172.21.15.183:6819/301003844","heartbeat_back_addr":"172.21.15.183:6823/301003844","heartbeat_front_addr":"172.21.15.183:6821/301003844","state":["exists","up"]},{"osd":9,"uuid":"e11dd8eb-c55a-4d70-8c58-457540b7a50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6827","nonce":3449260978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6829","nonce":3449260978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6833","nonce":3449260978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6831","nonce":3449260978}]},"public_addr":"172.21.15.53:6827/3449260978","cluster_addr":"172.21.15.53:6829/3449260978","heartbeat_back_addr":"172.21.15.53:6833/3449260978","heartbeat_front_addr":"172.21.15.53:6831/3449260978","state":["exists","up"]},{"osd":10,"uuid":"5de4803d-ad37-4a90-add5-0048f44fc4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6825","nonce":3435992308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6827","nonce":3435992308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6831","nonce":3435992308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6829","nonce":3435992308}]},"public_addr":"172.21.15.150:6825/3435992308","cluster_addr":"172.21.15.150:6827/3435992308","heartbeat_back_addr":"172.21.15.150:6831/3435992308","heartbeat_front_addr":"172.21.15.150:6829/3435992308","state":["exists","up"]},{"osd":11,"uuid":"0e819a68-ed56-477e-b9c0-81e8cc0bce11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6825","nonce":1957842686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6827","nonce":1957842686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6831","nonce":1957842686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6829","nonce":1957842686}]},"public_addr":"172.21.15.183:6825/1957842686","cluster_addr":"172.21.15.183:6827/1957842686","heartbeat_back_addr":"172.21.15.183:6831/1957842686","heartbeat_front_addr":"172.21.15.183:6829/1957842686","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:09.840629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:34.769537+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:58.112969+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:33:29.656790+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:33:55.181287+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:34:19.772645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:34:53.833921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:35:19.076733+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:35:43.163390+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:36:16.568340+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:36:42.492768+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:37:07.944639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"3.38","mappings":[{"from":5,"to":7}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6800/3887954889":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/1014501022":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/2576091441":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/1233527569":"2024-04-26T14:29:38.106242+0000","172.21.15.53:6801/3887954889":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/931889810":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/177196141":"2024-04-26T14:30:14.162383+0000","172.21.15.53:6800/1722497390":"2024-04-26T14:29:38.106242+0000","172.21.15.53:6801/1722497390":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/1927549198":"2024-04-26T14:30:14.162383+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:39:50.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.259+0000 7f9168000640 1 -- 172.21.15.53:0/2193553570 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f91480a3c30 msgr2=0x7f91480a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:50.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.259+0000 7f9168000640 1 --2- 172.21.15.53:0/2193553570 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f91480a3c30 0x7f91480a60f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f91540095f0 tx=0x7f9154030450 comp rx=0 tx=0).stop 2024-04-25T14:39:50.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.259+0000 7f9168000640 1 -- 172.21.15.53:0/2193553570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91601053a0 msgr2=0x7f916019a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:50.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.259+0000 7f9168000640 1 --2- 172.21.15.53:0/2193553570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91601053a0 0x7f916019a930 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f914c00e9f0 tx=0x7f914c00eeb0 comp rx=0 tx=0).stop 2024-04-25T14:39:50.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.259+0000 7f9168000640 1 -- 172.21.15.53:0/2193553570 shutdown_connections 2024-04-25T14:39:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.259+0000 7f9168000640 1 --2- 172.21.15.53:0/2193553570 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f91480a3c30 0x7f91480a60f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.259+0000 7f9168000640 1 --2- 172.21.15.53:0/2193553570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9160105d60 0x7f91601a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.259+0000 7f9168000640 1 --2- 172.21.15.53:0/2193553570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91601053a0 0x7f916019a930 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.259+0000 7f9168000640 1 --2- 172.21.15.53:0/2193553570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91601041b0 0x7f916019a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.259+0000 7f9168000640 1 -- 172.21.15.53:0/2193553570 >> 172.21.15.53:0/2193553570 conn(0x7f91600ff740 msgr2=0x7f9160108770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.259+0000 7f9168000640 1 -- 172.21.15.53:0/2193553570 shutdown_connections 2024-04-25T14:39:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.260+0000 7f9168000640 1 -- 172.21.15.53:0/2193553570 wait complete. 2024-04-25T14:39:50.281 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:33:02.338115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:38:07.517974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:38:08.994122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:39:50.282 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:39:50.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:50 smithi183 ceph-mon[32584]: pgmap v367: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-25T14:39:50.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:50 smithi150 ceph-mon[32504]: pgmap v367: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-25T14:39:50.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.503+0000 7f521e400640 1 -- 172.21.15.53:0/1033512919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5218105210 msgr2=0x7f5218107600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:50.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.503+0000 7f521e400640 1 --2- 172.21.15.53:0/1033512919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5218105210 0x7f5218107600 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5208009860 tx=0x7f520802a340 comp rx=0 tx=0).stop 2024-04-25T14:39:50.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.503+0000 7f521e400640 1 -- 172.21.15.53:0/1033512919 shutdown_connections 2024-04-25T14:39:50.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.503+0000 7f521e400640 1 --2- 172.21.15.53:0/1033512919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5218107b40 0x7f5218109f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:50.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.503+0000 7f521e400640 1 --2- 172.21.15.53:0/1033512919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5218105210 0x7f5218107600 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:50.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.503+0000 7f521e400640 1 --2- 172.21.15.53:0/1033512919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52181028e0 0x7f5218104cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:50.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.503+0000 7f521e400640 1 -- 172.21.15.53:0/1033512919 >> 172.21.15.53:0/1033512919 conn(0x7f52180fc2f0 msgr2=0x7f52180fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:50.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.504+0000 7f521e400640 1 -- 172.21.15.53:0/1033512919 shutdown_connections 2024-04-25T14:39:50.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.504+0000 7f521e400640 1 -- 172.21.15.53:0/1033512919 wait complete. 2024-04-25T14:39:50.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.505+0000 7f521e400640 1 Processor -- start 2024-04-25T14:39:50.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.505+0000 7f521e400640 1 -- start start 2024-04-25T14:39:50.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.505+0000 7f521e400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f52181028e0 0x7f521819a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:50.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.505+0000 7f521e400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5218105210 0x7f521819a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:50.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.506+0000 7f521e400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5218107b40 0x7f52181a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:50.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.506+0000 7f521e400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f521810d990 con 0x7f5218107b40 2024-04-25T14:39:50.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.506+0000 7f521e400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f521810d810 con 0x7f5218105210 2024-04-25T14:39:50.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.506+0000 7f521e400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f521810db10 con 0x7f52181028e0 2024-04-25T14:39:50.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.506+0000 7f521d000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f52181028e0 0x7f521819a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:50.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.506+0000 7f521d000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f52181028e0 0x7f521819a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:46094/0 (socket says 172.21.15.53:46094) 2024-04-25T14:39:50.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.506+0000 7f521d000640 1 -- 172.21.15.53:0/46344486 learned_addr learned my addr 172.21.15.53:0/46344486 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:50.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.506+0000 7f521da00640 1 --2- 172.21.15.53:0/46344486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5218107b40 0x7f52181a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:50.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.506+0000 7f520fe00640 1 --2- 172.21.15.53:0/46344486 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5218105210 0x7f521819a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:50.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.506+0000 7f521da00640 1 -- 172.21.15.53:0/46344486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f52181028e0 msgr2=0x7f521819a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:50.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.506+0000 7f521da00640 1 --2- 172.21.15.53:0/46344486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f52181028e0 0x7f521819a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:50.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.506+0000 7f521da00640 1 -- 172.21.15.53:0/46344486 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5218105210 msgr2=0x7f521819a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:50.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.506+0000 7f521da00640 1 --2- 172.21.15.53:0/46344486 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5218105210 0x7f521819a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:50.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.506+0000 7f521da00640 1 -- 172.21.15.53:0/46344486 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52181a1ee0 con 0x7f5218107b40 2024-04-25T14:39:50.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.506+0000 7f521d000640 1 --2- 172.21.15.53:0/46344486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f52181028e0 0x7f521819a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:39:50.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.507+0000 7f521da00640 1 --2- 172.21.15.53:0/46344486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5218107b40 0x7f52181a1870 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5210009fd0 tx=0x7f521000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:50.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.507+0000 7f5216000640 1 -- 172.21.15.53:0/46344486 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521000ce20 con 0x7f5218107b40 2024-04-25T14:39:50.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.507+0000 7f5216000640 1 -- 172.21.15.53:0/46344486 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5210005020 con 0x7f5218107b40 2024-04-25T14:39:50.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.507+0000 7f521e400640 1 -- 172.21.15.53:0/46344486 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52181a21d0 con 0x7f5218107b40 2024-04-25T14:39:50.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.507+0000 7f5216000640 1 -- 172.21.15.53:0/46344486 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5210010690 con 0x7f5218107b40 2024-04-25T14:39:50.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.507+0000 7f521e400640 1 -- 172.21.15.53:0/46344486 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52181a26f0 con 0x7f5218107b40 2024-04-25T14:39:50.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.508+0000 7f521e400640 1 -- 172.21.15.53:0/46344486 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5218109690 con 0x7f5218107b40 2024-04-25T14:39:50.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.510+0000 7f5216000640 1 -- 172.21.15.53:0/46344486 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f52100054d0 con 0x7f5218107b40 2024-04-25T14:39:50.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.512+0000 7f5216000640 1 --2- 172.21.15.53:0/46344486 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f51f80a3bb0 0x7f51f80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:50.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.512+0000 7f5216000640 1 -- 172.21.15.53:0/46344486 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f52100d0b40 con 0x7f5218107b40 2024-04-25T14:39:50.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.512+0000 7f521d000640 1 --2- 172.21.15.53:0/46344486 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f51f80a3bb0 0x7f51f80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:50.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.513+0000 7f521d000640 1 --2- 172.21.15.53:0/46344486 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f51f80a3bb0 0x7f51f80a6070 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f52000096c0 tx=0x7f5200009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:50.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.516+0000 7f5216000640 1 -- 172.21.15.53:0/46344486 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f521009d4b0 con 0x7f5218107b40 2024-04-25T14:39:50.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:50 smithi053 ceph-mon[27773]: pgmap v367: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-25T14:39:50.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.726+0000 7f521e400640 1 -- 172.21.15.53:0/46344486 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5218062b50 con 0x7f5218107b40 2024-04-25T14:39:50.727 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.727+0000 7f5216000640 1 -- 172.21.15.53:0/46344486 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f521009cdb0 con 0x7f5218107b40 2024-04-25T14:39:50.728 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2024-04-25T14:39:50.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.729+0000 7f521e400640 1 -- 172.21.15.53:0/46344486 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f51f80a3bb0 msgr2=0x7f51f80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:50.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.729+0000 7f521e400640 1 --2- 172.21.15.53:0/46344486 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f51f80a3bb0 0x7f51f80a6070 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f52000096c0 tx=0x7f5200009260 comp rx=0 tx=0).stop 2024-04-25T14:39:50.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.729+0000 7f521e400640 1 -- 172.21.15.53:0/46344486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5218107b40 msgr2=0x7f52181a1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:50.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.729+0000 7f521e400640 1 --2- 172.21.15.53:0/46344486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5218107b40 0x7f52181a1870 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5210009fd0 tx=0x7f521000ef10 comp rx=0 tx=0).stop 2024-04-25T14:39:50.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.730+0000 7f521e400640 1 -- 172.21.15.53:0/46344486 shutdown_connections 2024-04-25T14:39:50.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.730+0000 7f521e400640 1 --2- 172.21.15.53:0/46344486 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f51f80a3bb0 0x7f51f80a6070 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:50.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.730+0000 7f521e400640 1 --2- 172.21.15.53:0/46344486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5218107b40 0x7f52181a1870 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:50.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.730+0000 7f521e400640 1 --2- 172.21.15.53:0/46344486 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5218105210 0x7f521819a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:50.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.730+0000 7f521e400640 1 --2- 172.21.15.53:0/46344486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f52181028e0 0x7f521819a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:50.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.730+0000 7f521e400640 1 -- 172.21.15.53:0/46344486 >> 172.21.15.53:0/46344486 conn(0x7f52180fc2f0 msgr2=0x7f5218101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:50.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.730+0000 7f521e400640 1 -- 172.21.15.53:0/46344486 shutdown_connections 2024-04-25T14:39:50.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.730+0000 7f521e400640 1 -- 172.21.15.53:0/46344486 wait complete. 2024-04-25T14:39:50.750 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:39:50.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.959+0000 7f5fe3200640 1 -- 172.21.15.53:0/2720102686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe4105280 msgr2=0x7f5fe4105700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:50.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.959+0000 7f5fe3200640 1 --2- 172.21.15.53:0/2720102686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe4105280 0x7f5fe4105700 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0009900 tx=0x7f5fd002a410 comp rx=0 tx=0).stop 2024-04-25T14:39:50.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.960+0000 7f5fe3200640 1 -- 172.21.15.53:0/2720102686 shutdown_connections 2024-04-25T14:39:50.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.960+0000 7f5fe3200640 1 --2- 172.21.15.53:0/2720102686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fe4105c40 0x7f5fe410c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:50.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.960+0000 7f5fe3200640 1 --2- 172.21.15.53:0/2720102686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe4105280 0x7f5fe4105700 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:50.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.960+0000 7f5fe3200640 1 --2- 172.21.15.53:0/2720102686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5fe4104090 0x7f5fe41044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:50.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.960+0000 7f5fe3200640 1 -- 172.21.15.53:0/2720102686 >> 172.21.15.53:0/2720102686 conn(0x7f5fe40ff600 msgr2=0x7f5fe4101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:50.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.960+0000 7f5fe3200640 1 -- 172.21.15.53:0/2720102686 shutdown_connections 2024-04-25T14:39:50.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.960+0000 7f5fe3200640 1 -- 172.21.15.53:0/2720102686 wait complete. 2024-04-25T14:39:50.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.961+0000 7f5fe3200640 1 Processor -- start 2024-04-25T14:39:50.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.961+0000 7f5fe3200640 1 -- start start 2024-04-25T14:39:50.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.961+0000 7f5fe3200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe4104090 0x7f5fe419a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:50.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.961+0000 7f5fe3200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fe4105280 0x7f5fe419a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:50.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.961+0000 7f5fe3200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5fe4105c40 0x7f5fe41a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:50.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.961+0000 7f5fe3200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe410fc20 con 0x7f5fe4104090 2024-04-25T14:39:50.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.961+0000 7f5fe3200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe410faa0 con 0x7f5fe4105c40 2024-04-25T14:39:50.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.961+0000 7f5fe3200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe410fda0 con 0x7f5fe4105280 2024-04-25T14:39:50.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.962+0000 7f5fe1e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe4104090 0x7f5fe419a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:50.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.962+0000 7f5fe1e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe4104090 0x7f5fe419a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40396/0 (socket says 172.21.15.53:40396) 2024-04-25T14:39:50.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.962+0000 7f5fe1e00640 1 -- 172.21.15.53:0/4237769903 learned_addr learned my addr 172.21.15.53:0/4237769903 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:50.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.962+0000 7f5fe1400640 1 --2- 172.21.15.53:0/4237769903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fe4105280 0x7f5fe419a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:50.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.962+0000 7f5fe2800640 1 --2- 172.21.15.53:0/4237769903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5fe4105c40 0x7f5fe41a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:50.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.962+0000 7f5fe1e00640 1 -- 172.21.15.53:0/4237769903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fe4105280 msgr2=0x7f5fe419a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:50.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.962+0000 7f5fe1e00640 1 --2- 172.21.15.53:0/4237769903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fe4105280 0x7f5fe419a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:50.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.962+0000 7f5fe1e00640 1 -- 172.21.15.53:0/4237769903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5fe4105c40 msgr2=0x7f5fe41a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:50.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.962+0000 7f5fe1e00640 1 --2- 172.21.15.53:0/4237769903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5fe4105c40 0x7f5fe41a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:50.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.962+0000 7f5fe1e00640 1 -- 172.21.15.53:0/4237769903 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe41a1ec0 con 0x7f5fe4104090 2024-04-25T14:39:50.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.962+0000 7f5fe1e00640 1 --2- 172.21.15.53:0/4237769903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe4104090 0x7f5fe419a2a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400e9c0 tx=0x7f5fd400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:50.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.963+0000 7f5fdaa00640 1 -- 172.21.15.53:0/4237769903 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd400cde0 con 0x7f5fe4104090 2024-04-25T14:39:50.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.963+0000 7f5fdaa00640 1 -- 172.21.15.53:0/4237769903 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fd40108e0 con 0x7f5fe4104090 2024-04-25T14:39:50.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.963+0000 7f5fe3200640 1 -- 172.21.15.53:0/4237769903 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fe41a21b0 con 0x7f5fe4104090 2024-04-25T14:39:50.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.963+0000 7f5fdaa00640 1 -- 172.21.15.53:0/4237769903 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4018960 con 0x7f5fe4104090 2024-04-25T14:39:50.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.963+0000 7f5fe3200640 1 -- 172.21.15.53:0/4237769903 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fe41a2720 con 0x7f5fe4104090 2024-04-25T14:39:50.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.964+0000 7f5fe3200640 1 -- 172.21.15.53:0/4237769903 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fe410b480 con 0x7f5fe4104090 2024-04-25T14:39:50.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.966+0000 7f5fdaa00640 1 -- 172.21.15.53:0/4237769903 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5fd401f020 con 0x7f5fe4104090 2024-04-25T14:39:50.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.967+0000 7f5fdaa00640 1 --2- 172.21.15.53:0/4237769903 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5fc80a3c30 0x7f5fc80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:50.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.968+0000 7f5fdaa00640 1 -- 172.21.15.53:0/4237769903 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f5fd40d06f0 con 0x7f5fe4104090 2024-04-25T14:39:50.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.968+0000 7f5fe1400640 1 --2- 172.21.15.53:0/4237769903 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5fc80a3c30 0x7f5fc80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:50.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.968+0000 7f5fe1400640 1 --2- 172.21.15.53:0/4237769903 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5fc80a3c30 0x7f5fc80a60f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0031500 tx=0x7f5fd0005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:50.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:50.972+0000 7f5fdaa00640 1 -- 172.21.15.53:0/4237769903 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5fd4053ff0 con 0x7f5fe4104090 2024-04-25T14:39:51.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.169+0000 7f5fe3200640 1 -- 172.21.15.53:0/4237769903 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5fe4062b50 con 0x7f5fe4104090 2024-04-25T14:39:51.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.171+0000 7f5fdaa00640 1 -- 172.21.15.53:0/4237769903 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f5fd409d060 con 0x7f5fe4104090 2024-04-25T14:39:51.172 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2024-04-25T14:39:51.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.174+0000 7f5fe3200640 1 -- 172.21.15.53:0/4237769903 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5fc80a3c30 msgr2=0x7f5fc80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:51.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.174+0000 7f5fe3200640 1 --2- 172.21.15.53:0/4237769903 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5fc80a3c30 0x7f5fc80a60f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0031500 tx=0x7f5fd0005a20 comp rx=0 tx=0).stop 2024-04-25T14:39:51.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.174+0000 7f5fe3200640 1 -- 172.21.15.53:0/4237769903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe4104090 msgr2=0x7f5fe419a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:51.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.174+0000 7f5fe3200640 1 --2- 172.21.15.53:0/4237769903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe4104090 0x7f5fe419a2a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400e9c0 tx=0x7f5fd400ee80 comp rx=0 tx=0).stop 2024-04-25T14:39:51.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.174+0000 7f5fe3200640 1 -- 172.21.15.53:0/4237769903 shutdown_connections 2024-04-25T14:39:51.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.174+0000 7f5fe3200640 1 --2- 172.21.15.53:0/4237769903 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5fc80a3c30 0x7f5fc80a60f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:51.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.174+0000 7f5fe3200640 1 --2- 172.21.15.53:0/4237769903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5fe4105c40 0x7f5fe41a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:51.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.174+0000 7f5fe3200640 1 --2- 172.21.15.53:0/4237769903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fe4105280 0x7f5fe419a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:51.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.174+0000 7f5fe3200640 1 --2- 172.21.15.53:0/4237769903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe4104090 0x7f5fe419a2a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:51.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.174+0000 7f5fe3200640 1 -- 172.21.15.53:0/4237769903 >> 172.21.15.53:0/4237769903 conn(0x7f5fe40ff600 msgr2=0x7f5fe4108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:51.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.174+0000 7f5fe3200640 1 -- 172.21.15.53:0/4237769903 shutdown_connections 2024-04-25T14:39:51.176 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.174+0000 7f5fe3200640 1 -- 172.21.15.53:0/4237769903 wait complete. 2024-04-25T14:39:51.194 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:39:51.405 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.403+0000 7fc4f1400640 1 -- 172.21.15.53:0/3190656251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4ec103990 msgr2=0x7fc4ec10a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:51.405 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.403+0000 7fc4f1400640 1 --2- 172.21.15.53:0/3190656251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4ec103990 0x7fc4ec10a2b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4009900 tx=0x7fc4e402a3f0 comp rx=0 tx=0).stop 2024-04-25T14:39:51.405 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.404+0000 7fc4f1400640 1 -- 172.21.15.53:0/3190656251 shutdown_connections 2024-04-25T14:39:51.405 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.404+0000 7fc4f1400640 1 --2- 172.21.15.53:0/3190656251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4ec103990 0x7fc4ec10a2b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:51.405 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.404+0000 7fc4f1400640 1 --2- 172.21.15.53:0/3190656251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4ec102fd0 0x7fc4ec103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:51.405 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.404+0000 7fc4f1400640 1 --2- 172.21.15.53:0/3190656251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4ec101e80 0x7fc4ec1022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:51.405 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.404+0000 7fc4f1400640 1 -- 172.21.15.53:0/3190656251 >> 172.21.15.53:0/3190656251 conn(0x7fc4ec0fd3f0 msgr2=0x7fc4ec0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:51.405 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.404+0000 7fc4f1400640 1 -- 172.21.15.53:0/3190656251 shutdown_connections 2024-04-25T14:39:51.405 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.404+0000 7fc4f1400640 1 -- 172.21.15.53:0/3190656251 wait complete. 2024-04-25T14:39:51.407 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.405+0000 7fc4f1400640 1 Processor -- start 2024-04-25T14:39:51.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.405+0000 7fc4f1400640 1 -- start start 2024-04-25T14:39:51.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.405+0000 7fc4f1400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4ec101e80 0x7fc4ec197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:51.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.405+0000 7fc4f1400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4ec102fd0 0x7fc4ec198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:51.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.405+0000 7fc4f1400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4ec103990 0x7fc4ec19f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:51.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.405+0000 7fc4f1400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec10da10 con 0x7fc4ec103990 2024-04-25T14:39:51.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.405+0000 7fc4f1400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec10d890 con 0x7fc4ec102fd0 2024-04-25T14:39:51.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.405+0000 7fc4f1400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec10db90 con 0x7fc4ec101e80 2024-04-25T14:39:51.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.406+0000 7fc4f0a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4ec103990 0x7fc4ec19f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:51.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.406+0000 7fc4ebe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4ec101e80 0x7fc4ec197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:51.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.406+0000 7fc4ebe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4ec101e80 0x7fc4ec197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:46142/0 (socket says 172.21.15.53:46142) 2024-04-25T14:39:51.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.406+0000 7fc4f0a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4ec103990 0x7fc4ec19f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40412/0 (socket says 172.21.15.53:40412) 2024-04-25T14:39:51.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.406+0000 7fc4ebe00640 1 -- 172.21.15.53:0/2394446187 learned_addr learned my addr 172.21.15.53:0/2394446187 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:51.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.406+0000 7fc4eb400640 1 --2- 172.21.15.53:0/2394446187 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4ec102fd0 0x7fc4ec198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:51.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.406+0000 7fc4ebe00640 1 -- 172.21.15.53:0/2394446187 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4ec102fd0 msgr2=0x7fc4ec198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:51.409 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.406+0000 7fc4ebe00640 1 --2- 172.21.15.53:0/2394446187 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4ec102fd0 0x7fc4ec198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:51.409 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.406+0000 7fc4ebe00640 1 -- 172.21.15.53:0/2394446187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4ec103990 msgr2=0x7fc4ec19f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:51.409 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.406+0000 7fc4ebe00640 1 --2- 172.21.15.53:0/2394446187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4ec103990 0x7fc4ec19f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:51.409 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.406+0000 7fc4ebe00640 1 -- 172.21.15.53:0/2394446187 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4ec19fc10 con 0x7fc4ec101e80 2024-04-25T14:39:51.409 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.406+0000 7fc4eb400640 1 --2- 172.21.15.53:0/2394446187 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4ec102fd0 0x7fc4ec198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:39:51.409 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.406+0000 7fc4ebe00640 1 --2- 172.21.15.53:0/2394446187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4ec101e80 0x7fc4ec197ff0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400e9f0 tx=0x7fc4d400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:51.409 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.407+0000 7fc4e8c00640 1 -- 172.21.15.53:0/2394446187 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d400cd80 con 0x7fc4ec101e80 2024-04-25T14:39:51.409 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.407+0000 7fc4e8c00640 1 -- 172.21.15.53:0/2394446187 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4d40108e0 con 0x7fc4ec101e80 2024-04-25T14:39:51.409 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.407+0000 7fc4f1400640 1 -- 172.21.15.53:0/2394446187 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4ec0741e0 con 0x7fc4ec101e80 2024-04-25T14:39:51.409 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.407+0000 7fc4e8c00640 1 -- 172.21.15.53:0/2394446187 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4018980 con 0x7fc4ec101e80 2024-04-25T14:39:51.409 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.407+0000 7fc4f1400640 1 -- 172.21.15.53:0/2394446187 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4ec074730 con 0x7fc4ec101e80 2024-04-25T14:39:51.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.408+0000 7fc4f1400640 1 -- 172.21.15.53:0/2394446187 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4ec109130 con 0x7fc4ec101e80 2024-04-25T14:39:51.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.411+0000 7fc4e8c00640 1 -- 172.21.15.53:0/2394446187 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc4d4010430 con 0x7fc4ec101e80 2024-04-25T14:39:51.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.412+0000 7fc4e8c00640 1 --2- 172.21.15.53:0/2394446187 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fc4d00a3bb0 0x7fc4d00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:51.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.412+0000 7fc4e8c00640 1 -- 172.21.15.53:0/2394446187 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fc4d40d0740 con 0x7fc4ec101e80 2024-04-25T14:39:51.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.412+0000 7fc4eb400640 1 --2- 172.21.15.53:0/2394446187 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fc4d00a3bb0 0x7fc4d00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:51.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.413+0000 7fc4eb400640 1 --2- 172.21.15.53:0/2394446187 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fc4d00a3bb0 0x7fc4d00a6070 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec199500 tx=0x7fc4dc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:51.417 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.417+0000 7fc4e8c00640 1 -- 172.21.15.53:0/2394446187 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc4d409ca60 con 0x7fc4ec101e80 2024-04-25T14:39:51.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:51 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/2193553570' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:39:51.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:51 smithi183 ceph-mon[32584]: pgmap v368: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T14:39:51.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:51 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/46344486' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:39:51.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:51 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/2193553570' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:39:51.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:51 smithi150 ceph-mon[32504]: pgmap v368: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T14:39:51.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:51 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/46344486' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:39:51.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:51 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/2193553570' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:39:51.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:51 smithi053 ceph-mon[27773]: pgmap v368: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T14:39:51.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:51 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/46344486' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:39:51.623 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.623+0000 7fc4f1400640 1 -- 172.21.15.53:0/2394446187 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc4ec075b10 con 0x7fc4ec101e80 2024-04-25T14:39:51.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.624+0000 7fc4e8c00640 1 -- 172.21.15.53:0/2394446187 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7fc4d40a0d60 con 0x7fc4ec101e80 2024-04-25T14:39:51.625 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2024-04-25T14:39:51.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.626+0000 7fc4f1400640 1 -- 172.21.15.53:0/2394446187 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fc4d00a3bb0 msgr2=0x7fc4d00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:51.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.626+0000 7fc4f1400640 1 --2- 172.21.15.53:0/2394446187 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fc4d00a3bb0 0x7fc4d00a6070 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec199500 tx=0x7fc4dc009500 comp rx=0 tx=0).stop 2024-04-25T14:39:51.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.627+0000 7fc4f1400640 1 -- 172.21.15.53:0/2394446187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4ec101e80 msgr2=0x7fc4ec197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:51.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.627+0000 7fc4f1400640 1 --2- 172.21.15.53:0/2394446187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4ec101e80 0x7fc4ec197ff0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400e9f0 tx=0x7fc4d400eeb0 comp rx=0 tx=0).stop 2024-04-25T14:39:51.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.627+0000 7fc4f1400640 1 -- 172.21.15.53:0/2394446187 shutdown_connections 2024-04-25T14:39:51.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.627+0000 7fc4f1400640 1 --2- 172.21.15.53:0/2394446187 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fc4d00a3bb0 0x7fc4d00a6070 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:51.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.627+0000 7fc4f1400640 1 --2- 172.21.15.53:0/2394446187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4ec103990 0x7fc4ec19f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:51.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.627+0000 7fc4f1400640 1 --2- 172.21.15.53:0/2394446187 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4ec102fd0 0x7fc4ec198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:51.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.627+0000 7fc4f1400640 1 --2- 172.21.15.53:0/2394446187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4ec101e80 0x7fc4ec197ff0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:51.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.627+0000 7fc4f1400640 1 -- 172.21.15.53:0/2394446187 >> 172.21.15.53:0/2394446187 conn(0x7fc4ec0fd3f0 msgr2=0x7fc4ec106300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:51.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.627+0000 7fc4f1400640 1 -- 172.21.15.53:0/2394446187 shutdown_connections 2024-04-25T14:39:51.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.627+0000 7fc4f1400640 1 -- 172.21.15.53:0/2394446187 wait complete. 2024-04-25T14:39:51.647 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:39:51.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.860+0000 7fbfc3200640 1 -- 172.21.15.53:0/2800053214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfc4074950 msgr2=0x7fbfc4074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:51.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.860+0000 7fbfc3200640 1 --2- 172.21.15.53:0/2800053214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfc4074950 0x7fbfc4074db0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbfac009900 tx=0x7fbfac02a480 comp rx=0 tx=0).stop 2024-04-25T14:39:51.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.860+0000 7fbfc3200640 1 -- 172.21.15.53:0/2800053214 shutdown_connections 2024-04-25T14:39:51.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.860+0000 7fbfc3200640 1 --2- 172.21.15.53:0/2800053214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfc4109c80 0x7fbfc410c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:51.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.860+0000 7fbfc3200640 1 --2- 172.21.15.53:0/2800053214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfc4074950 0x7fbfc4074db0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:51.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.860+0000 7fbfc3200640 1 --2- 172.21.15.53:0/2800053214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbfc4075f30 0x7fbfc4074380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:51.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.860+0000 7fbfc3200640 1 -- 172.21.15.53:0/2800053214 >> 172.21.15.53:0/2800053214 conn(0x7fbfc40fd360 msgr2=0x7fbfc40ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:51.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.860+0000 7fbfc3200640 1 -- 172.21.15.53:0/2800053214 shutdown_connections 2024-04-25T14:39:51.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.860+0000 7fbfc3200640 1 -- 172.21.15.53:0/2800053214 wait complete. 2024-04-25T14:39:51.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.861+0000 7fbfc3200640 1 Processor -- start 2024-04-25T14:39:51.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.861+0000 7fbfc3200640 1 -- start start 2024-04-25T14:39:51.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.862+0000 7fbfc3200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfc4074950 0x7fbfc419a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:51.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.862+0000 7fbfc3200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfc4075f30 0x7fbfc419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:51.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.862+0000 7fbfc3200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbfc4109c80 0x7fbfc41a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:51.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.862+0000 7fbfc3200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc410fb80 con 0x7fbfc4075f30 2024-04-25T14:39:51.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.862+0000 7fbfc3200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc410fa00 con 0x7fbfc4074950 2024-04-25T14:39:51.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.862+0000 7fbfc3200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc410fd00 con 0x7fbfc4109c80 2024-04-25T14:39:51.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.862+0000 7fbfc1400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfc4075f30 0x7fbfc419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:51.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.862+0000 7fbfc1400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfc4075f30 0x7fbfc419a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40430/0 (socket says 172.21.15.53:40430) 2024-04-25T14:39:51.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.862+0000 7fbfc1400640 1 -- 172.21.15.53:0/1274807484 learned_addr learned my addr 172.21.15.53:0/1274807484 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:51.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.862+0000 7fbfc1e00640 1 --2- 172.21.15.53:0/1274807484 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfc4074950 0x7fbfc419a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:51.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.862+0000 7fbfc2800640 1 --2- 172.21.15.53:0/1274807484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbfc4109c80 0x7fbfc41a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:51.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.863+0000 7fbfc1400640 1 -- 172.21.15.53:0/1274807484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbfc4109c80 msgr2=0x7fbfc41a17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:51.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.863+0000 7fbfc1400640 1 --2- 172.21.15.53:0/1274807484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbfc4109c80 0x7fbfc41a17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:51.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.863+0000 7fbfc1400640 1 -- 172.21.15.53:0/1274807484 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfc4074950 msgr2=0x7fbfc419a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:51.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.863+0000 7fbfc1400640 1 --2- 172.21.15.53:0/1274807484 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfc4074950 0x7fbfc419a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:51.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.863+0000 7fbfc1400640 1 -- 172.21.15.53:0/1274807484 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc41a1e40 con 0x7fbfc4075f30 2024-04-25T14:39:51.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.863+0000 7fbfc2800640 1 --2- 172.21.15.53:0/1274807484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbfc4109c80 0x7fbfc41a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:39:51.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.863+0000 7fbfc1400640 1 --2- 172.21.15.53:0/1274807484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfc4075f30 0x7fbfc419a760 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbfac005ab0 tx=0x7fbfac003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:51.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.863+0000 7fbfbaa00640 1 -- 172.21.15.53:0/1274807484 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfac03f070 con 0x7fbfc4075f30 2024-04-25T14:39:51.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.863+0000 7fbfc3200640 1 -- 172.21.15.53:0/1274807484 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfc41a20d0 con 0x7fbfc4075f30 2024-04-25T14:39:51.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.863+0000 7fbfbaa00640 1 -- 172.21.15.53:0/1274807484 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfac031c50 con 0x7fbfc4075f30 2024-04-25T14:39:51.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.863+0000 7fbfbaa00640 1 -- 172.21.15.53:0/1274807484 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfac035030 con 0x7fbfc4075f30 2024-04-25T14:39:51.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.864+0000 7fbfc3200640 1 -- 172.21.15.53:0/1274807484 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfc41a2610 con 0x7fbfc4075f30 2024-04-25T14:39:51.868 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.865+0000 7fbfc3200640 1 -- 172.21.15.53:0/1274807484 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfc4108a40 con 0x7fbfc4075f30 2024-04-25T14:39:51.868 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.867+0000 7fbfbaa00640 1 -- 172.21.15.53:0/1274807484 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbfac042070 con 0x7fbfc4075f30 2024-04-25T14:39:51.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.868+0000 7fbfbaa00640 1 --2- 172.21.15.53:0/1274807484 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fbfa80a3c30 0x7fbfa80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:51.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.868+0000 7fbfbaa00640 1 -- 172.21.15.53:0/1274807484 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fbfac0f0b90 con 0x7fbfc4075f30 2024-04-25T14:39:51.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.868+0000 7fbfc1e00640 1 --2- 172.21.15.53:0/1274807484 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fbfa80a3c30 0x7fbfa80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:51.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.869+0000 7fbfc1e00640 1 --2- 172.21.15.53:0/1274807484 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fbfa80a3c30 0x7fbfa80a60f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4005fd0 tx=0x7fbfb4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:51.873 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:51.872+0000 7fbfbaa00640 1 -- 172.21.15.53:0/1274807484 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbfac0bceb0 con 0x7fbfc4075f30 2024-04-25T14:39:52.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.117+0000 7fbfc3200640 1 -- 172.21.15.53:0/1274807484 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbfc4062b50 con 0x7fbfc4075f30 2024-04-25T14:39:52.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.119+0000 7fbfbaa00640 1 -- 172.21.15.53:0/1274807484 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11862 (secure 0 0 0) 0x7fbfac0c11b0 con 0x7fbfc4075f30 2024-04-25T14:39:52.120 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:39:52.120 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14460,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6841/41110727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6840","nonce":41110727},{"type":"v1","addr":"172.21.15.53:6841","nonce":41110727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2966992472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2966992472},{"type":"v1","addr":"172.21.15.150:6833","nonce":2966992472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/94143643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":94143643},{"type":"v1","addr":"172.21.15.183:6833","nonce":94143643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24470,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/506682943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":506682943},{"type":"v1","addr":"172.21.15.53:6835","nonce":506682943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1369488844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1369488844},{"type":"v1","addr":"172.21.15.150:6835","nonce":1369488844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3786232682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3786232682},{"type":"v1","addr":"172.21.15.53:6837","nonce":3786232682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T14:38:10.037451+0000","modified":"2024-04-25T14:38:15.629854+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24584,"mds_1":24565,"mds_2":24560},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24532":{"gid":24532,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1189328602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1189328602},{"type":"v1","addr":"172.21.15.183:6835","nonce":1189328602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/2344343627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2344343627},{"type":"v1","addr":"172.21.15.150:6837","nonce":2344343627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24544":{"gid":24544,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1163336384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1163336384},{"type":"v1","addr":"172.21.15.183:6837","nonce":1163336384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6839/570000540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":570000540},{"type":"v1","addr":"172.21.15.53:6839","nonce":570000540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.183:6839/1554258915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1554258915},{"type":"v1","addr":"172.21.15.183:6839","nonce":1554258915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/149573616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":149573616},{"type":"v1","addr":"172.21.15.150:6839","nonce":149573616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24565,24584,24560]},"id":1}]} 2024-04-25T14:39:52.120 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 21 2024-04-25T14:39:52.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.121+0000 7fbfc3200640 1 -- 172.21.15.53:0/1274807484 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fbfa80a3c30 msgr2=0x7fbfa80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:52.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.121+0000 7fbfc3200640 1 --2- 172.21.15.53:0/1274807484 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fbfa80a3c30 0x7fbfa80a60f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4005fd0 tx=0x7fbfb4009500 comp rx=0 tx=0).stop 2024-04-25T14:39:52.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.121+0000 7fbfc3200640 1 -- 172.21.15.53:0/1274807484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfc4075f30 msgr2=0x7fbfc419a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:52.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.121+0000 7fbfc3200640 1 --2- 172.21.15.53:0/1274807484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfc4075f30 0x7fbfc419a760 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbfac005ab0 tx=0x7fbfac003b90 comp rx=0 tx=0).stop 2024-04-25T14:39:52.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.121+0000 7fbfc3200640 1 -- 172.21.15.53:0/1274807484 shutdown_connections 2024-04-25T14:39:52.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.121+0000 7fbfc3200640 1 --2- 172.21.15.53:0/1274807484 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fbfa80a3c30 0x7fbfa80a60f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:52.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.121+0000 7fbfc3200640 1 --2- 172.21.15.53:0/1274807484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbfc4109c80 0x7fbfc41a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:52.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.121+0000 7fbfc3200640 1 --2- 172.21.15.53:0/1274807484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfc4075f30 0x7fbfc419a760 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:52.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.122+0000 7fbfc3200640 1 --2- 172.21.15.53:0/1274807484 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfc4074950 0x7fbfc419a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:52.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.122+0000 7fbfc3200640 1 -- 172.21.15.53:0/1274807484 >> 172.21.15.53:0/1274807484 conn(0x7fbfc40fd360 msgr2=0x7fbfc4105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:52.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.122+0000 7fbfc3200640 1 -- 172.21.15.53:0/1274807484 shutdown_connections 2024-04-25T14:39:52.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.122+0000 7fbfc3200640 1 -- 172.21.15.53:0/1274807484 wait complete. 2024-04-25T14:39:52.142 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:39:52.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.356+0000 7efc1a800640 1 -- 172.21.15.53:0/3330317270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc14105180 msgr2=0x7efc14107570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:52.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.356+0000 7efc1a800640 1 --2- 172.21.15.53:0/3330317270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc14105180 0x7efc14107570 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efc00009860 tx=0x7efc0002a340 comp rx=0 tx=0).stop 2024-04-25T14:39:52.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.356+0000 7efc1a800640 1 -- 172.21.15.53:0/3330317270 shutdown_connections 2024-04-25T14:39:52.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.356+0000 7efc1a800640 1 --2- 172.21.15.53:0/3330317270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc14107ab0 0x7efc14109f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:52.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.356+0000 7efc1a800640 1 --2- 172.21.15.53:0/3330317270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc14105180 0x7efc14107570 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:52.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.356+0000 7efc1a800640 1 --2- 172.21.15.53:0/3330317270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc14102850 0x7efc14104c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:52.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.356+0000 7efc1a800640 1 -- 172.21.15.53:0/3330317270 >> 172.21.15.53:0/3330317270 conn(0x7efc140fc220 msgr2=0x7efc140fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:52.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.356+0000 7efc1a800640 1 -- 172.21.15.53:0/3330317270 shutdown_connections 2024-04-25T14:39:52.358 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.356+0000 7efc1a800640 1 -- 172.21.15.53:0/3330317270 wait complete. 2024-04-25T14:39:52.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.357+0000 7efc1a800640 1 Processor -- start 2024-04-25T14:39:52.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.357+0000 7efc1a800640 1 -- start start 2024-04-25T14:39:52.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.357+0000 7efc1a800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc14102850 0x7efc141021c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:52.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.357+0000 7efc1a800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc14105180 0x7efc14100800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:52.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.357+0000 7efc1a800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc14107ab0 0x7efc14100d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:52.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.357+0000 7efc1a800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7efc1410d8f0 con 0x7efc14102850 2024-04-25T14:39:52.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.357+0000 7efc1a800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7efc1410d770 con 0x7efc14107ab0 2024-04-25T14:39:52.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.357+0000 7efc1a800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efc1410da70 con 0x7efc14105180 2024-04-25T14:39:52.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.358+0000 7efc19400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc14102850 0x7efc141021c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:52.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.358+0000 7efc18a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc14105180 0x7efc14100800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:52.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.358+0000 7efc19400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc14102850 0x7efc141021c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59506/0 (socket says 172.21.15.53:59506) 2024-04-25T14:39:52.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.358+0000 7efc19400640 1 -- 172.21.15.53:0/1691384282 learned_addr learned my addr 172.21.15.53:0/1691384282 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:52.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.358+0000 7efc19e00640 1 --2- 172.21.15.53:0/1691384282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc14107ab0 0x7efc14100d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:52.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.358+0000 7efc18a00640 1 -- 172.21.15.53:0/1691384282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc14107ab0 msgr2=0x7efc14100d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:52.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.358+0000 7efc18a00640 1 --2- 172.21.15.53:0/1691384282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc14107ab0 0x7efc14100d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:52.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.358+0000 7efc18a00640 1 -- 172.21.15.53:0/1691384282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc14102850 msgr2=0x7efc141021c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:52.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.358+0000 7efc18a00640 1 --2- 172.21.15.53:0/1691384282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc14102850 0x7efc141021c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:52.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.358+0000 7efc18a00640 1 -- 172.21.15.53:0/1691384282 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc141015d0 con 0x7efc14105180 2024-04-25T14:39:52.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.358+0000 7efc19400640 1 --2- 172.21.15.53:0/1691384282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc14102850 0x7efc141021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:39:52.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.358+0000 7efc18a00640 1 --2- 172.21.15.53:0/1691384282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc14105180 0x7efc14100800 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efc0002a860 tx=0x7efc00032e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:52.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.359+0000 7efc12000640 1 -- 172.21.15.53:0/1691384282 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc00037040 con 0x7efc14105180 2024-04-25T14:39:52.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.359+0000 7efc12000640 1 -- 172.21.15.53:0/1691384282 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc00030c50 con 0x7efc14105180 2024-04-25T14:39:52.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.359+0000 7efc1a800640 1 -- 172.21.15.53:0/1691384282 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc141a20c0 con 0x7efc14105180 2024-04-25T14:39:52.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.359+0000 7efc12000640 1 -- 172.21.15.53:0/1691384282 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc00003850 con 0x7efc14105180 2024-04-25T14:39:52.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.359+0000 7efc1a800640 1 -- 172.21.15.53:0/1691384282 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc141a2600 con 0x7efc14105180 2024-04-25T14:39:52.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.360+0000 7efc1a800640 1 -- 172.21.15.53:0/1691384282 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc140fdd10 con 0x7efc14105180 2024-04-25T14:39:52.363 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.363+0000 7efc12000640 1 -- 172.21.15.53:0/1691384282 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7efc00043020 con 0x7efc14105180 2024-04-25T14:39:52.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.364+0000 7efc12000640 1 --2- 172.21.15.53:0/1691384282 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7efbf80a3c30 0x7efbf80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:52.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.364+0000 7efc12000640 1 -- 172.21.15.53:0/1691384282 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7efc000ef680 con 0x7efc14105180 2024-04-25T14:39:52.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.364+0000 7efc19400640 1 --2- 172.21.15.53:0/1691384282 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7efbf80a3c30 0x7efbf80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:52.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.365+0000 7efc19400640 1 --2- 172.21.15.53:0/1691384282 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7efbf80a3c30 0x7efbf80a60f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efc0400e860 tx=0x7efc04010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:52.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.369+0000 7efc12000640 1 -- 172.21.15.53:0/1691384282 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efc0004a060 con 0x7efc14105180 2024-04-25T14:39:52.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:52 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/4237769903' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:39:52.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:52 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/2394446187' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:39:52.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:52 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/1274807484' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:39:52.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:52 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/4237769903' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:39:52.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:52 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/2394446187' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:39:52.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:52 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/1274807484' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:39:52.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:52 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/4237769903' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:39:52.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:52 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/2394446187' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:39:52.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:52 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1274807484' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:39:52.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.579+0000 7efc1a800640 1 -- 172.21.15.53:0/1691384282 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc14062b50 con 0x7efc14105180 2024-04-25T14:39:52.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.581+0000 7efc12000640 1 -- 172.21.15.53:0/1691384282 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22668 (secure 0 0 0) 0x7efc000bbff0 con 0x7efc14105180 2024-04-25T14:39:52.583 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:39:52.583 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":90,"fsid":"fd30e6ce-030f-11ef-bc93-c7b262605968","created":"2024-04-25T14:28:54.330326+0000","modified":"2024-04-25T14:39:15.964081+0000","last_up_change":"2024-04-25T14:37:10.921148+0000","last_in_change":"2024-04-25T14:36:49.566459+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:33:02.338115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:38:07.517974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:38:08.994122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aecfbaf0-39da-4749-a49b-c7205dabd4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6803","nonce":464176015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6805","nonce":464176015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6809","nonce":464176015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6807","nonce":464176015}]},"public_addr":"172.21.15.53:6803/464176015","cluster_addr":"172.21.15.53:6805/464176015","heartbeat_back_addr":"172.21.15.53:6809/464176015","heartbeat_front_addr":"172.21.15.53:6807/464176015","state":["exists","up"]},{"osd":1,"uuid":"2c4a21ae-c43e-4839-8a56-c3fc63aeb7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6801","nonce":1304621982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6803","nonce":1304621982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6807","nonce":1304621982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6805","nonce":1304621982}]},"public_addr":"172.21.15.150:6801/1304621982","cluster_addr":"172.21.15.150:6803/1304621982","heartbeat_back_addr":"172.21.15.150:6807/1304621982","heartbeat_front_addr":"172.21.15.150:6805/1304621982","state":["exists","up"]},{"osd":2,"uuid":"e8effd43-adc0-472a-9eed-a9273b06dc0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6801","nonce":574509311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6803","nonce":574509311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6807","nonce":574509311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6805","nonce":574509311}]},"public_addr":"172.21.15.183:6801/574509311","cluster_addr":"172.21.15.183:6803/574509311","heartbeat_back_addr":"172.21.15.183:6807/574509311","heartbeat_front_addr":"172.21.15.183:6805/574509311","state":["exists","up"]},{"osd":3,"uuid":"e7cfced4-9b40-40de-a303-75ea8d5df4a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6811","nonce":570672219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6813","nonce":570672219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6817","nonce":570672219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6815","nonce":570672219}]},"public_addr":"172.21.15.53:6811/570672219","cluster_addr":"172.21.15.53:6813/570672219","heartbeat_back_addr":"172.21.15.53:6817/570672219","heartbeat_front_addr":"172.21.15.53:6815/570672219","state":["exists","up"]},{"osd":4,"uuid":"44a9e070-2282-43f1-9f05-b1f8ff81e540","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6809","nonce":1526850103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6811","nonce":1526850103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6815","nonce":1526850103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6813","nonce":1526850103}]},"public_addr":"172.21.15.150:6809/1526850103","cluster_addr":"172.21.15.150:6811/1526850103","heartbeat_back_addr":"172.21.15.150:6815/1526850103","heartbeat_front_addr":"172.21.15.150:6813/1526850103","state":["exists","up"]},{"osd":5,"uuid":"2c5619d2-c0f9-47d6-bf7e-2f4afd1b09f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6809","nonce":773726576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6811","nonce":773726576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6815","nonce":773726576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6813","nonce":773726576}]},"public_addr":"172.21.15.183:6809/773726576","cluster_addr":"172.21.15.183:6811/773726576","heartbeat_back_addr":"172.21.15.183:6815/773726576","heartbeat_front_addr":"172.21.15.183:6813/773726576","state":["exists","up"]},{"osd":6,"uuid":"1cf74d94-c118-4738-b6db-040af458a8eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6819","nonce":355884319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6821","nonce":355884319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6825","nonce":355884319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6823","nonce":355884319}]},"public_addr":"172.21.15.53:6819/355884319","cluster_addr":"172.21.15.53:6821/355884319","heartbeat_back_addr":"172.21.15.53:6825/355884319","heartbeat_front_addr":"172.21.15.53:6823/355884319","state":["exists","up"]},{"osd":7,"uuid":"0ce97c04-cd41-4080-bbbc-be9cf5fadc02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6817","nonce":229022562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6819","nonce":229022562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6823","nonce":229022562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6821","nonce":229022562}]},"public_addr":"172.21.15.150:6817/229022562","cluster_addr":"172.21.15.150:6819/229022562","heartbeat_back_addr":"172.21.15.150:6823/229022562","heartbeat_front_addr":"172.21.15.150:6821/229022562","state":["exists","up"]},{"osd":8,"uuid":"5d57787d-2a7c-4514-bd01-357ced413f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6817","nonce":301003844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6819","nonce":301003844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6823","nonce":301003844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6821","nonce":301003844}]},"public_addr":"172.21.15.183:6817/301003844","cluster_addr":"172.21.15.183:6819/301003844","heartbeat_back_addr":"172.21.15.183:6823/301003844","heartbeat_front_addr":"172.21.15.183:6821/301003844","state":["exists","up"]},{"osd":9,"uuid":"e11dd8eb-c55a-4d70-8c58-457540b7a50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6827","nonce":3449260978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6829","nonce":3449260978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6833","nonce":3449260978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6831","nonce":3449260978}]},"public_addr":"172.21.15.53:6827/3449260978","cluster_addr":"172.21.15.53:6829/3449260978","heartbeat_back_addr":"172.21.15.53:6833/3449260978","heartbeat_front_addr":"172.21.15.53:6831/3449260978","state":["exists","up"]},{"osd":10,"uuid":"5de4803d-ad37-4a90-add5-0048f44fc4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6825","nonce":3435992308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6827","nonce":3435992308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6831","nonce":3435992308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6829","nonce":3435992308}]},"public_addr":"172.21.15.150:6825/3435992308","cluster_addr":"172.21.15.150:6827/3435992308","heartbeat_back_addr":"172.21.15.150:6831/3435992308","heartbeat_front_addr":"172.21.15.150:6829/3435992308","state":["exists","up"]},{"osd":11,"uuid":"0e819a68-ed56-477e-b9c0-81e8cc0bce11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6825","nonce":1957842686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6827","nonce":1957842686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6831","nonce":1957842686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6829","nonce":1957842686}]},"public_addr":"172.21.15.183:6825/1957842686","cluster_addr":"172.21.15.183:6827/1957842686","heartbeat_back_addr":"172.21.15.183:6831/1957842686","heartbeat_front_addr":"172.21.15.183:6829/1957842686","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:09.840629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:34.769537+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:58.112969+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:33:29.656790+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:33:55.181287+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:34:19.772645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:34:53.833921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:35:19.076733+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:35:43.163390+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:36:16.568340+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:36:42.492768+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:37:07.944639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"3.38","mappings":[{"from":5,"to":7}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6800/3887954889":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/1014501022":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/2576091441":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/1233527569":"2024-04-26T14:29:38.106242+0000","172.21.15.53:6801/3887954889":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/931889810":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/177196141":"2024-04-26T14:30:14.162383+0000","172.21.15.53:6800/1722497390":"2024-04-26T14:29:38.106242+0000","172.21.15.53:6801/1722497390":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/1927549198":"2024-04-26T14:30:14.162383+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:39:52.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.585+0000 7efc1a800640 1 -- 172.21.15.53:0/1691384282 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7efbf80a3c30 msgr2=0x7efbf80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:52.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.585+0000 7efc1a800640 1 --2- 172.21.15.53:0/1691384282 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7efbf80a3c30 0x7efbf80a60f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efc0400e860 tx=0x7efc04010040 comp rx=0 tx=0).stop 2024-04-25T14:39:52.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.585+0000 7efc1a800640 1 -- 172.21.15.53:0/1691384282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc14105180 msgr2=0x7efc14100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:52.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.585+0000 7efc1a800640 1 --2- 172.21.15.53:0/1691384282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc14105180 0x7efc14100800 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efc0002a860 tx=0x7efc00032e90 comp rx=0 tx=0).stop 2024-04-25T14:39:52.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.585+0000 7efc1a800640 1 -- 172.21.15.53:0/1691384282 shutdown_connections 2024-04-25T14:39:52.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.585+0000 7efc1a800640 1 --2- 172.21.15.53:0/1691384282 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7efbf80a3c30 0x7efbf80a60f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:52.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.585+0000 7efc1a800640 1 --2- 172.21.15.53:0/1691384282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc14107ab0 0x7efc14100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:52.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.585+0000 7efc1a800640 1 --2- 172.21.15.53:0/1691384282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc14105180 0x7efc14100800 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:52.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.585+0000 7efc1a800640 1 --2- 172.21.15.53:0/1691384282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc14102850 0x7efc141021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:52.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.585+0000 7efc1a800640 1 -- 172.21.15.53:0/1691384282 >> 172.21.15.53:0/1691384282 conn(0x7efc140fc220 msgr2=0x7efc141030f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:52.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.585+0000 7efc1a800640 1 -- 172.21.15.53:0/1691384282 shutdown_connections 2024-04-25T14:39:52.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.585+0000 7efc1a800640 1 -- 172.21.15.53:0/1691384282 wait complete. 2024-04-25T14:39:52.607 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:33:02.338115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:38:07.517974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:38:08.994122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:39:52.608 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:39:52.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.826+0000 7ff6a2000640 1 -- 172.21.15.53:0/1073253262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff69c105340 msgr2=0x7ff69c107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:52.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.826+0000 7ff6a2000640 1 --2- 172.21.15.53:0/1073253262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff69c105340 0x7ff69c107730 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff688009860 tx=0x7ff68802a340 comp rx=0 tx=0).stop 2024-04-25T14:39:52.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.826+0000 7ff6a2000640 1 -- 172.21.15.53:0/1073253262 shutdown_connections 2024-04-25T14:39:52.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.826+0000 7ff6a2000640 1 --2- 172.21.15.53:0/1073253262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff69c107c70 0x7ff69c10a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:52.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.826+0000 7ff6a2000640 1 --2- 172.21.15.53:0/1073253262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff69c105340 0x7ff69c107730 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:52.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.826+0000 7ff6a2000640 1 --2- 172.21.15.53:0/1073253262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff69c102a10 0x7ff69c104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:52.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.826+0000 7ff6a2000640 1 -- 172.21.15.53:0/1073253262 >> 172.21.15.53:0/1073253262 conn(0x7ff69c0fc400 msgr2=0x7ff69c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:52.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.826+0000 7ff6a2000640 1 -- 172.21.15.53:0/1073253262 shutdown_connections 2024-04-25T14:39:52.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.827+0000 7ff6a2000640 1 -- 172.21.15.53:0/1073253262 wait complete. 2024-04-25T14:39:52.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.828+0000 7ff6a2000640 1 Processor -- start 2024-04-25T14:39:52.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.828+0000 7ff6a2000640 1 -- start start 2024-04-25T14:39:52.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.828+0000 7ff6a2000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff69c102a10 0x7ff69c19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:52.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.828+0000 7ff6a2000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff69c105340 0x7ff69c19a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:52.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.828+0000 7ff6a2000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff69c107c70 0x7ff69c1a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:52.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.828+0000 7ff6a2000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c10dab0 con 0x7ff69c102a10 2024-04-25T14:39:52.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.828+0000 7ff6a2000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c10d930 con 0x7ff69c107c70 2024-04-25T14:39:52.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.828+0000 7ff6a2000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c10dc30 con 0x7ff69c105340 2024-04-25T14:39:52.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.829+0000 7ff69b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff69c102a10 0x7ff69c19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:52.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.829+0000 7ff69b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff69c102a10 0x7ff69c19a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59524/0 (socket says 172.21.15.53:59524) 2024-04-25T14:39:52.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.829+0000 7ff69b400640 1 -- 172.21.15.53:0/3357729083 learned_addr learned my addr 172.21.15.53:0/3357729083 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:52.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.829+0000 7ff69aa00640 1 --2- 172.21.15.53:0/3357729083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff69c105340 0x7ff69c19a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:52.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.829+0000 7ff69be00640 1 --2- 172.21.15.53:0/3357729083 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff69c107c70 0x7ff69c1a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:52.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.829+0000 7ff69b400640 1 -- 172.21.15.53:0/3357729083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff69c105340 msgr2=0x7ff69c19a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:52.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.829+0000 7ff69b400640 1 --2- 172.21.15.53:0/3357729083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff69c105340 0x7ff69c19a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:52.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.829+0000 7ff69b400640 1 -- 172.21.15.53:0/3357729083 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff69c107c70 msgr2=0x7ff69c1a1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:52.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.829+0000 7ff69b400640 1 --2- 172.21.15.53:0/3357729083 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff69c107c70 0x7ff69c1a1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:52.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.829+0000 7ff69b400640 1 -- 172.21.15.53:0/3357729083 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff69c1a2070 con 0x7ff69c102a10 2024-04-25T14:39:52.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.829+0000 7ff69b400640 1 --2- 172.21.15.53:0/3357729083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff69c102a10 0x7ff69c19a450 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff68c00e9c0 tx=0x7ff68c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:52.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.830+0000 7ff693e00640 1 -- 172.21.15.53:0/3357729083 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68c00ccc0 con 0x7ff69c102a10 2024-04-25T14:39:52.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.830+0000 7ff6a2000640 1 -- 172.21.15.53:0/3357729083 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff69c1a2360 con 0x7ff69c102a10 2024-04-25T14:39:52.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.830+0000 7ff693e00640 1 -- 172.21.15.53:0/3357729083 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff68c0054d0 con 0x7ff69c102a10 2024-04-25T14:39:52.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.830+0000 7ff693e00640 1 -- 172.21.15.53:0/3357729083 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68c010430 con 0x7ff69c102a10 2024-04-25T14:39:52.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.830+0000 7ff6a2000640 1 -- 172.21.15.53:0/3357729083 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff69c1a2880 con 0x7ff69c102a10 2024-04-25T14:39:52.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.831+0000 7ff6a2000640 1 -- 172.21.15.53:0/3357729083 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff69c1097c0 con 0x7ff69c102a10 2024-04-25T14:39:52.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.833+0000 7ff693e00640 1 -- 172.21.15.53:0/3357729083 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff68c004f80 con 0x7ff69c102a10 2024-04-25T14:39:52.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.835+0000 7ff693e00640 1 --2- 172.21.15.53:0/3357729083 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff6800a3bb0 0x7ff6800a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:52.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.835+0000 7ff693e00640 1 -- 172.21.15.53:0/3357729083 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7ff68c0d0870 con 0x7ff69c102a10 2024-04-25T14:39:52.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.835+0000 7ff69aa00640 1 --2- 172.21.15.53:0/3357729083 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff6800a3bb0 0x7ff6800a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:52.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.836+0000 7ff69aa00640 1 --2- 172.21.15.53:0/3357729083 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff6800a3bb0 0x7ff6800a6070 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff688009830 tx=0x7ff688030450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:52.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:52.839+0000 7ff693e00640 1 -- 172.21.15.53:0/3357729083 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff68c09d1e0 con 0x7ff69c102a10 2024-04-25T14:39:53.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.046+0000 7ff6a2000640 1 -- 172.21.15.53:0/3357729083 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff69c062b50 con 0x7ff69c102a10 2024-04-25T14:39:53.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.047+0000 7ff693e00640 1 -- 172.21.15.53:0/3357729083 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7ff68c09cae0 con 0x7ff69c102a10 2024-04-25T14:39:53.048 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2024-04-25T14:39:53.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.049+0000 7ff6a2000640 1 -- 172.21.15.53:0/3357729083 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff6800a3bb0 msgr2=0x7ff6800a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:53.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.049+0000 7ff6a2000640 1 --2- 172.21.15.53:0/3357729083 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff6800a3bb0 0x7ff6800a6070 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff688009830 tx=0x7ff688030450 comp rx=0 tx=0).stop 2024-04-25T14:39:53.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.049+0000 7ff6a2000640 1 -- 172.21.15.53:0/3357729083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff69c102a10 msgr2=0x7ff69c19a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:53.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.049+0000 7ff6a2000640 1 --2- 172.21.15.53:0/3357729083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff69c102a10 0x7ff69c19a450 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff68c00e9c0 tx=0x7ff68c00ee80 comp rx=0 tx=0).stop 2024-04-25T14:39:53.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.050+0000 7ff6a2000640 1 -- 172.21.15.53:0/3357729083 shutdown_connections 2024-04-25T14:39:53.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.050+0000 7ff6a2000640 1 --2- 172.21.15.53:0/3357729083 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff6800a3bb0 0x7ff6800a6070 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:53.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.050+0000 7ff6a2000640 1 --2- 172.21.15.53:0/3357729083 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff69c107c70 0x7ff69c1a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:53.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.050+0000 7ff6a2000640 1 --2- 172.21.15.53:0/3357729083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff69c105340 0x7ff69c19a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:53.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.050+0000 7ff6a2000640 1 --2- 172.21.15.53:0/3357729083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff69c102a10 0x7ff69c19a450 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:53.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.050+0000 7ff6a2000640 1 -- 172.21.15.53:0/3357729083 >> 172.21.15.53:0/3357729083 conn(0x7ff69c0fc400 msgr2=0x7ff69c102260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:53.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.050+0000 7ff6a2000640 1 -- 172.21.15.53:0/3357729083 shutdown_connections 2024-04-25T14:39:53.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.050+0000 7ff6a2000640 1 -- 172.21.15.53:0/3357729083 wait complete. 2024-04-25T14:39:53.072 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:39:53.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.289+0000 7f96bfe00640 1 -- 172.21.15.53:0/2531073485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96c0108490 msgr2=0x7f96c01088b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:53.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.289+0000 7f96bfe00640 1 --2- 172.21.15.53:0/2531073485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96c0108490 0x7f96c01088b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f96b0009900 tx=0x7f96b002a410 comp rx=0 tx=0).stop 2024-04-25T14:39:53.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.289+0000 7f96bfe00640 1 -- 172.21.15.53:0/2531073485 shutdown_connections 2024-04-25T14:39:53.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.289+0000 7f96bfe00640 1 --2- 172.21.15.53:0/2531073485 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96c010a040 0x7f96c01108d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:53.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.289+0000 7f96bfe00640 1 --2- 172.21.15.53:0/2531073485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96c0109680 0x7f96c0109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:53.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.289+0000 7f96bfe00640 1 --2- 172.21.15.53:0/2531073485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96c0108490 0x7f96c01088b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:53.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.289+0000 7f96bfe00640 1 -- 172.21.15.53:0/2531073485 >> 172.21.15.53:0/2531073485 conn(0x7f96c0077420 msgr2=0x7f96c0079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:53.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.289+0000 7f96bfe00640 1 -- 172.21.15.53:0/2531073485 shutdown_connections 2024-04-25T14:39:53.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.289+0000 7f96bfe00640 1 -- 172.21.15.53:0/2531073485 wait complete. 2024-04-25T14:39:53.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.290+0000 7f96bfe00640 1 Processor -- start 2024-04-25T14:39:53.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.290+0000 7f96bfe00640 1 -- start start 2024-04-25T14:39:53.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.290+0000 7f96bfe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96c0108490 0x7f96c019e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:53.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.291+0000 7f96bfe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96c0109680 0x7f96c019ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:53.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.291+0000 7f96bfe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96c010a040 0x7f96c01a5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:53.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.291+0000 7f96bfe00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f96c0113ff0 con 0x7f96c0108490 2024-04-25T14:39:53.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.291+0000 7f96bfe00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f96c0113e70 con 0x7f96c010a040 2024-04-25T14:39:53.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.291+0000 7f96bfe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f96c0114170 con 0x7f96c0109680 2024-04-25T14:39:53.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.291+0000 7f96bea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96c0108490 0x7f96c019e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:53.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.291+0000 7f96bea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96c0108490 0x7f96c019e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59546/0 (socket says 172.21.15.53:59546) 2024-04-25T14:39:53.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.291+0000 7f96bea00640 1 -- 172.21.15.53:0/3434038252 learned_addr learned my addr 172.21.15.53:0/3434038252 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:53.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.291+0000 7f96b7e00640 1 --2- 172.21.15.53:0/3434038252 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96c0109680 0x7f96c019ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:53.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.291+0000 7f96bf400640 1 --2- 172.21.15.53:0/3434038252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96c010a040 0x7f96c01a5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:53.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.291+0000 7f96bea00640 1 -- 172.21.15.53:0/3434038252 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96c0109680 msgr2=0x7f96c019ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:53.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.291+0000 7f96bea00640 1 --2- 172.21.15.53:0/3434038252 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96c0109680 0x7f96c019ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:53.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.291+0000 7f96bea00640 1 -- 172.21.15.53:0/3434038252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96c010a040 msgr2=0x7f96c01a5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:53.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.291+0000 7f96bea00640 1 --2- 172.21.15.53:0/3434038252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96c010a040 0x7f96c01a5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:53.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.291+0000 7f96bea00640 1 -- 172.21.15.53:0/3434038252 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c01a62f0 con 0x7f96c0108490 2024-04-25T14:39:53.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.292+0000 7f96bea00640 1 --2- 172.21.15.53:0/3434038252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96c0108490 0x7f96c019e720 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f96b0031500 tx=0x7f96b0003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:53.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.292+0000 7f96c4a00640 1 -- 172.21.15.53:0/3434038252 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b003f070 con 0x7f96c0108490 2024-04-25T14:39:53.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.292+0000 7f96c4a00640 1 -- 172.21.15.53:0/3434038252 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96b0003e70 con 0x7f96c0108490 2024-04-25T14:39:53.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.292+0000 7f96c4a00640 1 -- 172.21.15.53:0/3434038252 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b0035030 con 0x7f96c0108490 2024-04-25T14:39:53.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.292+0000 7f96bfe00640 1 -- 172.21.15.53:0/3434038252 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96c01a6580 con 0x7f96c0108490 2024-04-25T14:39:53.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.292+0000 7f96bfe00640 1 -- 172.21.15.53:0/3434038252 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96c01a6ac0 con 0x7f96c0108490 2024-04-25T14:39:53.295 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.293+0000 7f96bfe00640 1 -- 172.21.15.53:0/3434038252 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96c010f8d0 con 0x7f96c0108490 2024-04-25T14:39:53.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.295+0000 7f96c4a00640 1 -- 172.21.15.53:0/3434038252 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f96b0042070 con 0x7f96c0108490 2024-04-25T14:39:53.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.297+0000 7f96c4a00640 1 --2- 172.21.15.53:0/3434038252 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f96a40a3c30 0x7f96a40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:53.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.297+0000 7f96c4a00640 1 -- 172.21.15.53:0/3434038252 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f96b00f0fa0 con 0x7f96c0108490 2024-04-25T14:39:53.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.297+0000 7f96b7e00640 1 --2- 172.21.15.53:0/3434038252 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f96a40a3c30 0x7f96a40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:53.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.298+0000 7f96b7e00640 1 --2- 172.21.15.53:0/3434038252 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f96a40a3c30 0x7f96a40a60f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f96c019fbe0 tx=0x7f96a800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:53.302 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.301+0000 7f96c4a00640 1 -- 172.21.15.53:0/3434038252 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f96b00bd2c0 con 0x7f96c0108490 2024-04-25T14:39:53.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:53 smithi183 ceph-mon[32584]: pgmap v369: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-25T14:39:53.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:53 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/1691384282' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:39:53.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:53 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/3357729083' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:39:53.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:53 smithi150 ceph-mon[32504]: pgmap v369: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-25T14:39:53.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:53 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/1691384282' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:39:53.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:53 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/3357729083' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:39:53.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.509+0000 7f96bfe00640 1 -- 172.21.15.53:0/3434038252 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f96c0062b50 con 0x7f96c0108490 2024-04-25T14:39:53.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.509+0000 7f96c4a00640 1 -- 172.21.15.53:0/3434038252 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f96b00c15c0 con 0x7f96c0108490 2024-04-25T14:39:53.510 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2024-04-25T14:39:53.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.513+0000 7f96bfe00640 1 -- 172.21.15.53:0/3434038252 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f96a40a3c30 msgr2=0x7f96a40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:53.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.513+0000 7f96bfe00640 1 --2- 172.21.15.53:0/3434038252 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f96a40a3c30 0x7f96a40a60f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f96c019fbe0 tx=0x7f96a800b390 comp rx=0 tx=0).stop 2024-04-25T14:39:53.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.513+0000 7f96bfe00640 1 -- 172.21.15.53:0/3434038252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96c0108490 msgr2=0x7f96c019e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:53.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.513+0000 7f96bfe00640 1 --2- 172.21.15.53:0/3434038252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96c0108490 0x7f96c019e720 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f96b0031500 tx=0x7f96b0003c80 comp rx=0 tx=0).stop 2024-04-25T14:39:53.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.513+0000 7f96bfe00640 1 -- 172.21.15.53:0/3434038252 shutdown_connections 2024-04-25T14:39:53.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.513+0000 7f96bfe00640 1 --2- 172.21.15.53:0/3434038252 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f96a40a3c30 0x7f96a40a60f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:53.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.513+0000 7f96bfe00640 1 --2- 172.21.15.53:0/3434038252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96c010a040 0x7f96c01a5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:53.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.513+0000 7f96bfe00640 1 --2- 172.21.15.53:0/3434038252 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96c0109680 0x7f96c019ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:53.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.513+0000 7f96bfe00640 1 --2- 172.21.15.53:0/3434038252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96c0108490 0x7f96c019e720 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:53.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.513+0000 7f96bfe00640 1 -- 172.21.15.53:0/3434038252 >> 172.21.15.53:0/3434038252 conn(0x7f96c0077420 msgr2=0x7f96c010caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:53.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.513+0000 7f96bfe00640 1 -- 172.21.15.53:0/3434038252 shutdown_connections 2024-04-25T14:39:53.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.513+0000 7f96bfe00640 1 -- 172.21.15.53:0/3434038252 wait complete. 2024-04-25T14:39:53.531 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:53 smithi053 ceph-mon[27773]: pgmap v369: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-25T14:39:53.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:53 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1691384282' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:39:53.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:53 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/3357729083' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:39:53.535 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:39:53.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.751+0000 7fcd90e00640 1 -- 172.21.15.53:0/1775106809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd8c10a160 msgr2=0x7fcd8c1109f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:53.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.751+0000 7fcd90e00640 1 --2- 172.21.15.53:0/1775106809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd8c10a160 0x7fcd8c1109f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcd84009be0 tx=0x7fcd8402a6c0 comp rx=0 tx=0).stop 2024-04-25T14:39:53.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.751+0000 7fcd90e00640 1 -- 172.21.15.53:0/1775106809 shutdown_connections 2024-04-25T14:39:53.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.751+0000 7fcd90e00640 1 --2- 172.21.15.53:0/1775106809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd8c10a160 0x7fcd8c1109f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:53.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.751+0000 7fcd90e00640 1 --2- 172.21.15.53:0/1775106809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd8c1097a0 0x7fcd8c109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:53.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.751+0000 7fcd90e00640 1 --2- 172.21.15.53:0/1775106809 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd8c1085b0 0x7fcd8c1089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:53.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.751+0000 7fcd90e00640 1 -- 172.21.15.53:0/1775106809 >> 172.21.15.53:0/1775106809 conn(0x7fcd8c077550 msgr2=0x7fcd8c079990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:53.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.751+0000 7fcd90e00640 1 -- 172.21.15.53:0/1775106809 shutdown_connections 2024-04-25T14:39:53.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.751+0000 7fcd90e00640 1 -- 172.21.15.53:0/1775106809 wait complete. 2024-04-25T14:39:53.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.752+0000 7fcd90e00640 1 Processor -- start 2024-04-25T14:39:53.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.752+0000 7fcd90e00640 1 -- start start 2024-04-25T14:39:53.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.752+0000 7fcd90e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd8c1085b0 0x7fcd8c19e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:53.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.752+0000 7fcd90e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd8c1097a0 0x7fcd8c19ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:53.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.752+0000 7fcd90e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd8c10a160 0x7fcd8c1a5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:53.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.753+0000 7fcd90e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8c1140c0 con 0x7fcd8c1097a0 2024-04-25T14:39:53.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.753+0000 7fcd90e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8c113f40 con 0x7fcd8c10a160 2024-04-25T14:39:53.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.753+0000 7fcd90e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8c114240 con 0x7fcd8c1085b0 2024-04-25T14:39:53.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.753+0000 7fcd83e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd8c1097a0 0x7fcd8c19ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:53.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.753+0000 7fcd89e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd8c1085b0 0x7fcd8c19e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:53.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.753+0000 7fcd83e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd8c1097a0 0x7fcd8c19ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59578/0 (socket says 172.21.15.53:59578) 2024-04-25T14:39:53.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.753+0000 7fcd8a800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd8c10a160 0x7fcd8c1a5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:53.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.753+0000 7fcd89e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd8c1085b0 0x7fcd8c19e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:35286/0 (socket says 172.21.15.53:35286) 2024-04-25T14:39:53.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.753+0000 7fcd83e00640 1 -- 172.21.15.53:0/1694341170 learned_addr learned my addr 172.21.15.53:0/1694341170 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:53.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.753+0000 7fcd83e00640 1 -- 172.21.15.53:0/1694341170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd8c1085b0 msgr2=0x7fcd8c19e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:53.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.753+0000 7fcd83e00640 1 --2- 172.21.15.53:0/1694341170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd8c1085b0 0x7fcd8c19e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:53.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.753+0000 7fcd83e00640 1 -- 172.21.15.53:0/1694341170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd8c10a160 msgr2=0x7fcd8c1a5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:53.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.753+0000 7fcd83e00640 1 --2- 172.21.15.53:0/1694341170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd8c10a160 0x7fcd8c1a5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:53.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.753+0000 7fcd83e00640 1 -- 172.21.15.53:0/1694341170 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd8c1a63d0 con 0x7fcd8c1097a0 2024-04-25T14:39:53.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.753+0000 7fcd89e00640 1 --2- 172.21.15.53:0/1694341170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd8c1085b0 0x7fcd8c19e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:39:53.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.754+0000 7fcd83e00640 1 --2- 172.21.15.53:0/1694341170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd8c1097a0 0x7fcd8c19ecf0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcd7400e9c0 tx=0x7fcd7400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:53.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.754+0000 7fcd82a00640 1 -- 172.21.15.53:0/1694341170 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7400cde0 con 0x7fcd8c1097a0 2024-04-25T14:39:53.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.754+0000 7fcd90e00640 1 -- 172.21.15.53:0/1694341170 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd8c1a66c0 con 0x7fcd8c1097a0 2024-04-25T14:39:53.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.754+0000 7fcd82a00640 1 -- 172.21.15.53:0/1694341170 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd740035f0 con 0x7fcd8c1097a0 2024-04-25T14:39:53.757 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.754+0000 7fcd82a00640 1 -- 172.21.15.53:0/1694341170 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd74010690 con 0x7fcd8c1097a0 2024-04-25T14:39:53.757 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.754+0000 7fcd90e00640 1 -- 172.21.15.53:0/1694341170 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd8c1a6c60 con 0x7fcd8c1097a0 2024-04-25T14:39:53.757 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.755+0000 7fcd90e00640 1 -- 172.21.15.53:0/1694341170 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd8c10f8d0 con 0x7fcd8c1097a0 2024-04-25T14:39:53.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.758+0000 7fcd82a00640 1 -- 172.21.15.53:0/1694341170 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fcd74004000 con 0x7fcd8c1097a0 2024-04-25T14:39:53.761 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.760+0000 7fcd82a00640 1 --2- 172.21.15.53:0/1694341170 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fcd6c0a3bb0 0x7fcd6c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:53.761 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.761+0000 7fcd82a00640 1 -- 172.21.15.53:0/1694341170 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fcd740d0760 con 0x7fcd8c1097a0 2024-04-25T14:39:53.761 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.761+0000 7fcd89e00640 1 --2- 172.21.15.53:0/1694341170 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fcd6c0a3bb0 0x7fcd6c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:53.762 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.761+0000 7fcd89e00640 1 --2- 172.21.15.53:0/1694341170 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fcd6c0a3bb0 0x7fcd6c0a6070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c109600 tx=0x7fcd7c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:53.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.765+0000 7fcd82a00640 1 -- 172.21.15.53:0/1694341170 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcd74009510 con 0x7fcd8c1097a0 2024-04-25T14:39:53.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.973+0000 7fcd90e00640 1 -- 172.21.15.53:0/1694341170 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcd8c062b50 con 0x7fcd8c1097a0 2024-04-25T14:39:53.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.974+0000 7fcd82a00640 1 -- 172.21.15.53:0/1694341170 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7fcd7409d0d0 con 0x7fcd8c1097a0 2024-04-25T14:39:53.975 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2024-04-25T14:39:53.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.977+0000 7fcd90e00640 1 -- 172.21.15.53:0/1694341170 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fcd6c0a3bb0 msgr2=0x7fcd6c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:53.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.977+0000 7fcd90e00640 1 --2- 172.21.15.53:0/1694341170 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fcd6c0a3bb0 0x7fcd6c0a6070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c109600 tx=0x7fcd7c005f90 comp rx=0 tx=0).stop 2024-04-25T14:39:53.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.977+0000 7fcd90e00640 1 -- 172.21.15.53:0/1694341170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd8c1097a0 msgr2=0x7fcd8c19ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:53.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.977+0000 7fcd90e00640 1 --2- 172.21.15.53:0/1694341170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd8c1097a0 0x7fcd8c19ecf0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcd7400e9c0 tx=0x7fcd7400ee80 comp rx=0 tx=0).stop 2024-04-25T14:39:53.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.978+0000 7fcd90e00640 1 -- 172.21.15.53:0/1694341170 shutdown_connections 2024-04-25T14:39:53.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.978+0000 7fcd90e00640 1 --2- 172.21.15.53:0/1694341170 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fcd6c0a3bb0 0x7fcd6c0a6070 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:53.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.978+0000 7fcd90e00640 1 --2- 172.21.15.53:0/1694341170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd8c10a160 0x7fcd8c1a5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:53.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.978+0000 7fcd90e00640 1 --2- 172.21.15.53:0/1694341170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd8c1097a0 0x7fcd8c19ecf0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:53.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.978+0000 7fcd90e00640 1 --2- 172.21.15.53:0/1694341170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd8c1085b0 0x7fcd8c19e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:53.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.978+0000 7fcd90e00640 1 -- 172.21.15.53:0/1694341170 >> 172.21.15.53:0/1694341170 conn(0x7fcd8c077550 msgr2=0x7fcd8c10caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:53.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.978+0000 7fcd90e00640 1 -- 172.21.15.53:0/1694341170 shutdown_connections 2024-04-25T14:39:53.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:53.978+0000 7fcd90e00640 1 -- 172.21.15.53:0/1694341170 wait complete. 2024-04-25T14:39:53.998 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:39:54.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.212+0000 7f438a000640 1 -- 172.21.15.53:0/3421212101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4384103fd0 msgr2=0x7f43841043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:54.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.212+0000 7f438a000640 1 --2- 172.21.15.53:0/3421212101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4384103fd0 0x7f43841043f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4374009900 tx=0x7f437402a410 comp rx=0 tx=0).stop 2024-04-25T14:39:54.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.212+0000 7f438a000640 1 -- 172.21.15.53:0/3421212101 shutdown_connections 2024-04-25T14:39:54.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.212+0000 7f438a000640 1 --2- 172.21.15.53:0/3421212101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4384105b80 0x7f438410c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:54.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.212+0000 7f438a000640 1 --2- 172.21.15.53:0/3421212101 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43841051c0 0x7f4384105640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:54.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.213+0000 7f438a000640 1 --2- 172.21.15.53:0/3421212101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4384103fd0 0x7f43841043f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:54.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.213+0000 7f438a000640 1 -- 172.21.15.53:0/3421212101 >> 172.21.15.53:0/3421212101 conn(0x7f43840ff560 msgr2=0x7f4384101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:54.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.213+0000 7f438a000640 1 -- 172.21.15.53:0/3421212101 shutdown_connections 2024-04-25T14:39:54.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.213+0000 7f438a000640 1 -- 172.21.15.53:0/3421212101 wait complete. 2024-04-25T14:39:54.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.214+0000 7f438a000640 1 Processor -- start 2024-04-25T14:39:54.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.214+0000 7f438a000640 1 -- start start 2024-04-25T14:39:54.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.214+0000 7f438a000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4384103fd0 0x7f438419a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:54.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.214+0000 7f438a000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43841051c0 0x7f438419a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:54.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.214+0000 7f438a000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4384105b80 0x7f43841a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:54.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.214+0000 7f438a000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f438410fb60 con 0x7f4384105b80 2024-04-25T14:39:54.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.214+0000 7f438a000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f438410f9e0 con 0x7f4384103fd0 2024-04-25T14:39:54.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.214+0000 7f438a000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f438410fce0 con 0x7f43841051c0 2024-04-25T14:39:54.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.214+0000 7f4388c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4384103fd0 0x7f438419a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:54.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.214+0000 7f4388c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4384103fd0 0x7f438419a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.53:60642/0 (socket says 172.21.15.53:60642) 2024-04-25T14:39:54.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.214+0000 7f4388c00640 1 -- 172.21.15.53:0/1828003099 learned_addr learned my addr 172.21.15.53:0/1828003099 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:54.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.214+0000 7f4389600640 1 --2- 172.21.15.53:0/1828003099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4384105b80 0x7f43841a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:54.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.214+0000 7f437be00640 1 --2- 172.21.15.53:0/1828003099 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43841051c0 0x7f438419a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:54.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.215+0000 7f4388c00640 1 -- 172.21.15.53:0/1828003099 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43841051c0 msgr2=0x7f438419a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:54.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.215+0000 7f4388c00640 1 --2- 172.21.15.53:0/1828003099 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43841051c0 0x7f438419a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:54.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.215+0000 7f4388c00640 1 -- 172.21.15.53:0/1828003099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4384105b80 msgr2=0x7f43841a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:54.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.215+0000 7f4388c00640 1 --2- 172.21.15.53:0/1828003099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4384105b80 0x7f43841a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:54.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.215+0000 7f4388c00640 1 -- 172.21.15.53:0/1828003099 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43841a1e70 con 0x7f4384103fd0 2024-04-25T14:39:54.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.215+0000 7f437be00640 1 --2- 172.21.15.53:0/1828003099 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43841051c0 0x7f438419a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:39:54.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.215+0000 7f4388c00640 1 --2- 172.21.15.53:0/1828003099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4384103fd0 0x7f438419a250 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4374031500 tx=0x7f4374003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:54.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.215+0000 7f4382000640 1 -- 172.21.15.53:0/1828003099 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437403f070 con 0x7f4384103fd0 2024-04-25T14:39:54.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.215+0000 7f4382000640 1 -- 172.21.15.53:0/1828003099 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4374031c50 con 0x7f4384103fd0 2024-04-25T14:39:54.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.215+0000 7f438a000640 1 -- 172.21.15.53:0/1828003099 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43841a2100 con 0x7f4384103fd0 2024-04-25T14:39:54.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.215+0000 7f4382000640 1 -- 172.21.15.53:0/1828003099 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4374035030 con 0x7f4384103fd0 2024-04-25T14:39:54.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.216+0000 7f438a000640 1 -- 172.21.15.53:0/1828003099 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43841a25f0 con 0x7f4384103fd0 2024-04-25T14:39:54.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.217+0000 7f438a000640 1 -- 172.21.15.53:0/1828003099 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f438410b340 con 0x7f4384103fd0 2024-04-25T14:39:54.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.219+0000 7f4382000640 1 -- 172.21.15.53:0/1828003099 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4374042070 con 0x7f4384103fd0 2024-04-25T14:39:54.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.220+0000 7f4382000640 1 --2- 172.21.15.53:0/1828003099 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f43680a3bb0 0x7f43680a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:54.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.220+0000 7f4382000640 1 -- 172.21.15.53:0/1828003099 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f43740f0c30 con 0x7f4384103fd0 2024-04-25T14:39:54.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.220+0000 7f437be00640 1 --2- 172.21.15.53:0/1828003099 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f43680a3bb0 0x7f43680a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:54.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.221+0000 7f437be00640 1 --2- 172.21.15.53:0/1828003099 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f43680a3bb0 0x7f43680a6070 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f438419b760 tx=0x7f436c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:54.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.225+0000 7f4382000640 1 -- 172.21.15.53:0/1828003099 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f43740bd5a0 con 0x7f4384103fd0 2024-04-25T14:39:54.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:54 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/3434038252' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:39:54.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:54 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1694341170' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:39:54.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:54 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/3434038252' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:39:54.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:54 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/1694341170' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:39:54.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:54 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/3434038252' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:39:54.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:54 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/1694341170' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:39:54.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.483+0000 7f438a000640 1 -- 172.21.15.53:0/1828003099 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f438402d5c0 con 0x7f4384103fd0 2024-04-25T14:39:54.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.484+0000 7f4382000640 1 -- 172.21.15.53:0/1828003099 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11862 (secure 0 0 0) 0x7f43740bcea0 con 0x7f4384103fd0 2024-04-25T14:39:54.485 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:39:54.485 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14460,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6841/41110727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6840","nonce":41110727},{"type":"v1","addr":"172.21.15.53:6841","nonce":41110727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2966992472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2966992472},{"type":"v1","addr":"172.21.15.150:6833","nonce":2966992472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/94143643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":94143643},{"type":"v1","addr":"172.21.15.183:6833","nonce":94143643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24470,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/506682943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":506682943},{"type":"v1","addr":"172.21.15.53:6835","nonce":506682943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1369488844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1369488844},{"type":"v1","addr":"172.21.15.150:6835","nonce":1369488844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3786232682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3786232682},{"type":"v1","addr":"172.21.15.53:6837","nonce":3786232682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T14:38:10.037451+0000","modified":"2024-04-25T14:38:15.629854+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24584,"mds_1":24565,"mds_2":24560},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24532":{"gid":24532,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1189328602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1189328602},{"type":"v1","addr":"172.21.15.183:6835","nonce":1189328602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/2344343627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2344343627},{"type":"v1","addr":"172.21.15.150:6837","nonce":2344343627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24544":{"gid":24544,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1163336384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1163336384},{"type":"v1","addr":"172.21.15.183:6837","nonce":1163336384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6839/570000540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":570000540},{"type":"v1","addr":"172.21.15.53:6839","nonce":570000540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.183:6839/1554258915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1554258915},{"type":"v1","addr":"172.21.15.183:6839","nonce":1554258915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/149573616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":149573616},{"type":"v1","addr":"172.21.15.150:6839","nonce":149573616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24565,24584,24560]},"id":1}]} 2024-04-25T14:39:54.486 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 21 2024-04-25T14:39:54.488 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.487+0000 7f438a000640 1 -- 172.21.15.53:0/1828003099 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f43680a3bb0 msgr2=0x7f43680a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:54.488 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.487+0000 7f438a000640 1 --2- 172.21.15.53:0/1828003099 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f43680a3bb0 0x7f43680a6070 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f438419b760 tx=0x7f436c005f90 comp rx=0 tx=0).stop 2024-04-25T14:39:54.488 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.487+0000 7f438a000640 1 -- 172.21.15.53:0/1828003099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4384103fd0 msgr2=0x7f438419a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:54.488 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.487+0000 7f438a000640 1 --2- 172.21.15.53:0/1828003099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4384103fd0 0x7f438419a250 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4374031500 tx=0x7f4374003c10 comp rx=0 tx=0).stop 2024-04-25T14:39:54.488 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.487+0000 7f438a000640 1 -- 172.21.15.53:0/1828003099 shutdown_connections 2024-04-25T14:39:54.488 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.487+0000 7f438a000640 1 --2- 172.21.15.53:0/1828003099 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f43680a3bb0 0x7f43680a6070 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:54.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.487+0000 7f438a000640 1 --2- 172.21.15.53:0/1828003099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4384105b80 0x7f43841a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:54.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.487+0000 7f438a000640 1 --2- 172.21.15.53:0/1828003099 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43841051c0 0x7f438419a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:54.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.487+0000 7f438a000640 1 --2- 172.21.15.53:0/1828003099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4384103fd0 0x7f438419a250 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:54.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.487+0000 7f438a000640 1 -- 172.21.15.53:0/1828003099 >> 172.21.15.53:0/1828003099 conn(0x7f43840ff560 msgr2=0x7f4384108510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:54.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.487+0000 7f438a000640 1 -- 172.21.15.53:0/1828003099 shutdown_connections 2024-04-25T14:39:54.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.487+0000 7f438a000640 1 -- 172.21.15.53:0/1828003099 wait complete. 2024-04-25T14:39:54.509 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:39:54.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.702+0000 7f60d1400640 1 -- 172.21.15.53:0/287523871 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60cc1041b0 msgr2=0x7f60cc1045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:54.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.702+0000 7f60d1400640 1 --2- 172.21.15.53:0/287523871 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60cc1041b0 0x7f60cc1045d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f60bc009ca0 tx=0x7f60bc02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:39:54.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.703+0000 7f60d1400640 1 -- 172.21.15.53:0/287523871 shutdown_connections 2024-04-25T14:39:54.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.703+0000 7f60d1400640 1 --2- 172.21.15.53:0/287523871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60cc105d60 0x7f60cc10c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:54.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.703+0000 7f60d1400640 1 --2- 172.21.15.53:0/287523871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60cc1053a0 0x7f60cc105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:54.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.703+0000 7f60d1400640 1 --2- 172.21.15.53:0/287523871 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60cc1041b0 0x7f60cc1045d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:54.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.703+0000 7f60d1400640 1 -- 172.21.15.53:0/287523871 >> 172.21.15.53:0/287523871 conn(0x7f60cc0ff740 msgr2=0x7f60cc101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:54.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.703+0000 7f60d1400640 1 -- 172.21.15.53:0/287523871 shutdown_connections 2024-04-25T14:39:54.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.703+0000 7f60d1400640 1 -- 172.21.15.53:0/287523871 wait complete. 2024-04-25T14:39:54.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.704+0000 7f60d1400640 1 Processor -- start 2024-04-25T14:39:54.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.704+0000 7f60d1400640 1 -- start start 2024-04-25T14:39:54.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.704+0000 7f60d1400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60cc1041b0 0x7f60cc19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:54.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.704+0000 7f60d1400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60cc1053a0 0x7f60cc19a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:54.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.704+0000 7f60d1400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60cc105d60 0x7f60cc1a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:54.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.705+0000 7f60d1400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f60cc10fcf0 con 0x7f60cc1053a0 2024-04-25T14:39:54.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.705+0000 7f60d1400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f60cc10fb70 con 0x7f60cc1041b0 2024-04-25T14:39:54.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.705+0000 7f60d1400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f60cc10fe70 con 0x7f60cc105d60 2024-04-25T14:39:54.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.705+0000 7f60c9e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60cc1053a0 0x7f60cc19a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:54.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.705+0000 7f60c9e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60cc1053a0 0x7f60cc19a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59610/0 (socket says 172.21.15.53:59610) 2024-04-25T14:39:54.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.705+0000 7f60c9e00640 1 -- 172.21.15.53:0/2113084387 learned_addr learned my addr 172.21.15.53:0/2113084387 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:54.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.705+0000 7f60ca800640 1 --2- 172.21.15.53:0/2113084387 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60cc1041b0 0x7f60cc19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:54.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.705+0000 7f60cb200640 1 --2- 172.21.15.53:0/2113084387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60cc105d60 0x7f60cc1a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:54.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.705+0000 7f60c9e00640 1 -- 172.21.15.53:0/2113084387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60cc105d60 msgr2=0x7f60cc1a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:54.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.705+0000 7f60c9e00640 1 --2- 172.21.15.53:0/2113084387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60cc105d60 0x7f60cc1a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:54.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.705+0000 7f60c9e00640 1 -- 172.21.15.53:0/2113084387 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60cc1041b0 msgr2=0x7f60cc19a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:54.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.705+0000 7f60c9e00640 1 --2- 172.21.15.53:0/2113084387 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60cc1041b0 0x7f60cc19a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:54.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.705+0000 7f60c9e00640 1 -- 172.21.15.53:0/2113084387 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60cc1a2040 con 0x7f60cc1053a0 2024-04-25T14:39:54.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.706+0000 7f60c9e00640 1 --2- 172.21.15.53:0/2113084387 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60cc1053a0 0x7f60cc19a960 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f60b800e9f0 tx=0x7f60b800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:54.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.706+0000 7f60c3400640 1 -- 172.21.15.53:0/2113084387 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b800ccc0 con 0x7f60cc1053a0 2024-04-25T14:39:54.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.706+0000 7f60d1400640 1 -- 172.21.15.53:0/2113084387 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60cc1a2330 con 0x7f60cc1053a0 2024-04-25T14:39:54.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.706+0000 7f60c3400640 1 -- 172.21.15.53:0/2113084387 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60b800ce60 con 0x7f60cc1053a0 2024-04-25T14:39:54.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.706+0000 7f60c3400640 1 -- 172.21.15.53:0/2113084387 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b8010430 con 0x7f60cc1053a0 2024-04-25T14:39:54.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.706+0000 7f60d1400640 1 -- 172.21.15.53:0/2113084387 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60cc1a2880 con 0x7f60cc1053a0 2024-04-25T14:39:54.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.707+0000 7f60d1400640 1 -- 172.21.15.53:0/2113084387 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60cc10b570 con 0x7f60cc1053a0 2024-04-25T14:39:54.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.710+0000 7f60c3400640 1 -- 172.21.15.53:0/2113084387 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f60b80046c0 con 0x7f60cc1053a0 2024-04-25T14:39:54.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.711+0000 7f60c3400640 1 --2- 172.21.15.53:0/2113084387 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f60b00a3bb0 0x7f60b00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:54.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.712+0000 7f60c3400640 1 -- 172.21.15.53:0/2113084387 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f60b80d0840 con 0x7f60cc1053a0 2024-04-25T14:39:54.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.712+0000 7f60ca800640 1 --2- 172.21.15.53:0/2113084387 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f60b00a3bb0 0x7f60b00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:54.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.712+0000 7f60ca800640 1 --2- 172.21.15.53:0/2113084387 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f60b00a3bb0 0x7f60b00a6070 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f60bc000f80 tx=0x7f60bc005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:54.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.716+0000 7f60c3400640 1 -- 172.21.15.53:0/2113084387 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f60b809d1b0 con 0x7f60cc1053a0 2024-04-25T14:39:54.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.924+0000 7f60d1400640 1 -- 172.21.15.53:0/2113084387 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f60cc062b50 con 0x7f60cc1053a0 2024-04-25T14:39:54.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.926+0000 7f60c3400640 1 -- 172.21.15.53:0/2113084387 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22668 (secure 0 0 0) 0x7f60b809cab0 con 0x7f60cc1053a0 2024-04-25T14:39:54.928 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:39:54.928 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":90,"fsid":"fd30e6ce-030f-11ef-bc93-c7b262605968","created":"2024-04-25T14:28:54.330326+0000","modified":"2024-04-25T14:39:15.964081+0000","last_up_change":"2024-04-25T14:37:10.921148+0000","last_in_change":"2024-04-25T14:36:49.566459+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:33:02.338115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:38:07.517974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:38:08.994122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aecfbaf0-39da-4749-a49b-c7205dabd4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6803","nonce":464176015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6805","nonce":464176015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6809","nonce":464176015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6807","nonce":464176015}]},"public_addr":"172.21.15.53:6803/464176015","cluster_addr":"172.21.15.53:6805/464176015","heartbeat_back_addr":"172.21.15.53:6809/464176015","heartbeat_front_addr":"172.21.15.53:6807/464176015","state":["exists","up"]},{"osd":1,"uuid":"2c4a21ae-c43e-4839-8a56-c3fc63aeb7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6801","nonce":1304621982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6803","nonce":1304621982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6807","nonce":1304621982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6805","nonce":1304621982}]},"public_addr":"172.21.15.150:6801/1304621982","cluster_addr":"172.21.15.150:6803/1304621982","heartbeat_back_addr":"172.21.15.150:6807/1304621982","heartbeat_front_addr":"172.21.15.150:6805/1304621982","state":["exists","up"]},{"osd":2,"uuid":"e8effd43-adc0-472a-9eed-a9273b06dc0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6801","nonce":574509311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6803","nonce":574509311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6807","nonce":574509311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6805","nonce":574509311}]},"public_addr":"172.21.15.183:6801/574509311","cluster_addr":"172.21.15.183:6803/574509311","heartbeat_back_addr":"172.21.15.183:6807/574509311","heartbeat_front_addr":"172.21.15.183:6805/574509311","state":["exists","up"]},{"osd":3,"uuid":"e7cfced4-9b40-40de-a303-75ea8d5df4a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6811","nonce":570672219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6813","nonce":570672219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6817","nonce":570672219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6815","nonce":570672219}]},"public_addr":"172.21.15.53:6811/570672219","cluster_addr":"172.21.15.53:6813/570672219","heartbeat_back_addr":"172.21.15.53:6817/570672219","heartbeat_front_addr":"172.21.15.53:6815/570672219","state":["exists","up"]},{"osd":4,"uuid":"44a9e070-2282-43f1-9f05-b1f8ff81e540","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6809","nonce":1526850103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6811","nonce":1526850103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6815","nonce":1526850103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6813","nonce":1526850103}]},"public_addr":"172.21.15.150:6809/1526850103","cluster_addr":"172.21.15.150:6811/1526850103","heartbeat_back_addr":"172.21.15.150:6815/1526850103","heartbeat_front_addr":"172.21.15.150:6813/1526850103","state":["exists","up"]},{"osd":5,"uuid":"2c5619d2-c0f9-47d6-bf7e-2f4afd1b09f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6809","nonce":773726576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6811","nonce":773726576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6815","nonce":773726576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6813","nonce":773726576}]},"public_addr":"172.21.15.183:6809/773726576","cluster_addr":"172.21.15.183:6811/773726576","heartbeat_back_addr":"172.21.15.183:6815/773726576","heartbeat_front_addr":"172.21.15.183:6813/773726576","state":["exists","up"]},{"osd":6,"uuid":"1cf74d94-c118-4738-b6db-040af458a8eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6819","nonce":355884319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6821","nonce":355884319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6825","nonce":355884319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6823","nonce":355884319}]},"public_addr":"172.21.15.53:6819/355884319","cluster_addr":"172.21.15.53:6821/355884319","heartbeat_back_addr":"172.21.15.53:6825/355884319","heartbeat_front_addr":"172.21.15.53:6823/355884319","state":["exists","up"]},{"osd":7,"uuid":"0ce97c04-cd41-4080-bbbc-be9cf5fadc02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6817","nonce":229022562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6819","nonce":229022562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6823","nonce":229022562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6821","nonce":229022562}]},"public_addr":"172.21.15.150:6817/229022562","cluster_addr":"172.21.15.150:6819/229022562","heartbeat_back_addr":"172.21.15.150:6823/229022562","heartbeat_front_addr":"172.21.15.150:6821/229022562","state":["exists","up"]},{"osd":8,"uuid":"5d57787d-2a7c-4514-bd01-357ced413f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6817","nonce":301003844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6819","nonce":301003844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6823","nonce":301003844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6821","nonce":301003844}]},"public_addr":"172.21.15.183:6817/301003844","cluster_addr":"172.21.15.183:6819/301003844","heartbeat_back_addr":"172.21.15.183:6823/301003844","heartbeat_front_addr":"172.21.15.183:6821/301003844","state":["exists","up"]},{"osd":9,"uuid":"e11dd8eb-c55a-4d70-8c58-457540b7a50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6827","nonce":3449260978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6829","nonce":3449260978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6833","nonce":3449260978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6831","nonce":3449260978}]},"public_addr":"172.21.15.53:6827/3449260978","cluster_addr":"172.21.15.53:6829/3449260978","heartbeat_back_addr":"172.21.15.53:6833/3449260978","heartbeat_front_addr":"172.21.15.53:6831/3449260978","state":["exists","up"]},{"osd":10,"uuid":"5de4803d-ad37-4a90-add5-0048f44fc4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6825","nonce":3435992308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6827","nonce":3435992308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6831","nonce":3435992308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6829","nonce":3435992308}]},"public_addr":"172.21.15.150:6825/3435992308","cluster_addr":"172.21.15.150:6827/3435992308","heartbeat_back_addr":"172.21.15.150:6831/3435992308","heartbeat_front_addr":"172.21.15.150:6829/3435992308","state":["exists","up"]},{"osd":11,"uuid":"0e819a68-ed56-477e-b9c0-81e8cc0bce11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6825","nonce":1957842686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6827","nonce":1957842686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6831","nonce":1957842686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6829","nonce":1957842686}]},"public_addr":"172.21.15.183:6825/1957842686","cluster_addr":"172.21.15.183:6827/1957842686","heartbeat_back_addr":"172.21.15.183:6831/1957842686","heartbeat_front_addr":"172.21.15.183:6829/1957842686","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:09.840629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:34.769537+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:58.112969+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:33:29.656790+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:33:55.181287+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:34:19.772645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:34:53.833921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:35:19.076733+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:35:43.163390+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:36:16.568340+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:36:42.492768+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:37:07.944639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"3.38","mappings":[{"from":5,"to":7}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6800/3887954889":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/1014501022":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/2576091441":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/1233527569":"2024-04-26T14:29:38.106242+0000","172.21.15.53:6801/3887954889":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/931889810":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/177196141":"2024-04-26T14:30:14.162383+0000","172.21.15.53:6800/1722497390":"2024-04-26T14:29:38.106242+0000","172.21.15.53:6801/1722497390":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/1927549198":"2024-04-26T14:30:14.162383+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:39:54.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.930+0000 7f60d1400640 1 -- 172.21.15.53:0/2113084387 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f60b00a3bb0 msgr2=0x7f60b00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:54.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.930+0000 7f60d1400640 1 --2- 172.21.15.53:0/2113084387 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f60b00a3bb0 0x7f60b00a6070 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f60bc000f80 tx=0x7f60bc005dc0 comp rx=0 tx=0).stop 2024-04-25T14:39:54.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.930+0000 7f60d1400640 1 -- 172.21.15.53:0/2113084387 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60cc1053a0 msgr2=0x7f60cc19a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:54.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.930+0000 7f60d1400640 1 --2- 172.21.15.53:0/2113084387 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60cc1053a0 0x7f60cc19a960 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f60b800e9f0 tx=0x7f60b800edf0 comp rx=0 tx=0).stop 2024-04-25T14:39:54.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.930+0000 7f60d1400640 1 -- 172.21.15.53:0/2113084387 shutdown_connections 2024-04-25T14:39:54.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.930+0000 7f60d1400640 1 --2- 172.21.15.53:0/2113084387 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f60b00a3bb0 0x7f60b00a6070 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:54.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.930+0000 7f60d1400640 1 --2- 172.21.15.53:0/2113084387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60cc105d60 0x7f60cc1a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:54.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.930+0000 7f60d1400640 1 --2- 172.21.15.53:0/2113084387 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60cc1053a0 0x7f60cc19a960 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:54.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.930+0000 7f60d1400640 1 --2- 172.21.15.53:0/2113084387 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60cc1041b0 0x7f60cc19a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:54.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.930+0000 7f60d1400640 1 -- 172.21.15.53:0/2113084387 >> 172.21.15.53:0/2113084387 conn(0x7f60cc0ff740 msgr2=0x7f60cc108740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:54.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.930+0000 7f60d1400640 1 -- 172.21.15.53:0/2113084387 shutdown_connections 2024-04-25T14:39:54.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:54.930+0000 7f60d1400640 1 -- 172.21.15.53:0/2113084387 wait complete. 2024-04-25T14:39:54.950 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:39:55.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.158+0000 7fb70a000640 1 -- 172.21.15.53:0/270519978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb704102850 msgr2=0x7fb704104c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:55.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.158+0000 7fb70a000640 1 --2- 172.21.15.53:0/270519978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb704102850 0x7fb704104c40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4009ca0 tx=0x7fb6f402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.158+0000 7fb70a000640 1 -- 172.21.15.53:0/270519978 shutdown_connections 2024-04-25T14:39:55.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.158+0000 7fb70a000640 1 --2- 172.21.15.53:0/270519978 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb704107ab0 0x7fb704109f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.158+0000 7fb70a000640 1 --2- 172.21.15.53:0/270519978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb704105180 0x7fb704107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.158+0000 7fb70a000640 1 --2- 172.21.15.53:0/270519978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb704102850 0x7fb704104c40 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.158+0000 7fb70a000640 1 -- 172.21.15.53:0/270519978 >> 172.21.15.53:0/270519978 conn(0x7fb7040fc220 msgr2=0x7fb7040fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:55.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.159+0000 7fb70a000640 1 -- 172.21.15.53:0/270519978 shutdown_connections 2024-04-25T14:39:55.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.159+0000 7fb70a000640 1 -- 172.21.15.53:0/270519978 wait complete. 2024-04-25T14:39:55.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.160+0000 7fb70a000640 1 Processor -- start 2024-04-25T14:39:55.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.160+0000 7fb70a000640 1 -- start start 2024-04-25T14:39:55.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.160+0000 7fb70a000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb704102850 0x7fb70419a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:55.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.160+0000 7fb70a000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb704105180 0x7fb70419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:55.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.160+0000 7fb70a000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb704107ab0 0x7fb7041a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:55.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.160+0000 7fb70a000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb70410d900 con 0x7fb704105180 2024-04-25T14:39:55.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.160+0000 7fb70a000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb70410d780 con 0x7fb704102850 2024-04-25T14:39:55.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.160+0000 7fb70a000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb70410da80 con 0x7fb704107ab0 2024-04-25T14:39:55.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.161+0000 7fb703e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb704105180 0x7fb70419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:55.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.161+0000 7fb709600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb704107ab0 0x7fb7041a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:55.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.161+0000 7fb709600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb704107ab0 0x7fb7041a17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:35322/0 (socket says 172.21.15.53:35322) 2024-04-25T14:39:55.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.161+0000 7fb708c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb704102850 0x7fb70419a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:55.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.161+0000 7fb709600640 1 -- 172.21.15.53:0/1199173754 learned_addr learned my addr 172.21.15.53:0/1199173754 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:55.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.161+0000 7fb709600640 1 -- 172.21.15.53:0/1199173754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb704102850 msgr2=0x7fb70419a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:55.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.161+0000 7fb709600640 1 --2- 172.21.15.53:0/1199173754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb704102850 0x7fb70419a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.161+0000 7fb709600640 1 -- 172.21.15.53:0/1199173754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb704105180 msgr2=0x7fb70419a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:55.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.161+0000 7fb709600640 1 --2- 172.21.15.53:0/1199173754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb704105180 0x7fb70419a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.161+0000 7fb709600640 1 -- 172.21.15.53:0/1199173754 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7041a1e40 con 0x7fb704107ab0 2024-04-25T14:39:55.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.161+0000 7fb703e00640 1 --2- 172.21.15.53:0/1199173754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb704105180 0x7fb70419a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:39:55.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.161+0000 7fb709600640 1 --2- 172.21.15.53:0/1199173754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb704107ab0 0x7fb7041a17d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc00e970 tx=0x7fb6fc00ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:55.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.162+0000 7fb701600640 1 -- 172.21.15.53:0/1199173754 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc00ccc0 con 0x7fb704107ab0 2024-04-25T14:39:55.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.162+0000 7fb701600640 1 -- 172.21.15.53:0/1199173754 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6fc00ce60 con 0x7fb704107ab0 2024-04-25T14:39:55.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.162+0000 7fb701600640 1 -- 172.21.15.53:0/1199173754 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc010430 con 0x7fb704107ab0 2024-04-25T14:39:55.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.162+0000 7fb70a000640 1 -- 172.21.15.53:0/1199173754 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7041a2130 con 0x7fb704107ab0 2024-04-25T14:39:55.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.162+0000 7fb70a000640 1 -- 172.21.15.53:0/1199173754 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7041a2630 con 0x7fb704107ab0 2024-04-25T14:39:55.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.163+0000 7fb70a000640 1 -- 172.21.15.53:0/1199173754 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb704109600 con 0x7fb704107ab0 2024-04-25T14:39:55.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.165+0000 7fb701600640 1 -- 172.21.15.53:0/1199173754 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb6fc004ff0 con 0x7fb704107ab0 2024-04-25T14:39:55.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.166+0000 7fb701600640 1 --2- 172.21.15.53:0/1199173754 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fb6e80a3bb0 0x7fb6e80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:55.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.167+0000 7fb701600640 1 -- 172.21.15.53:0/1199173754 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fb6fc0d0870 con 0x7fb704107ab0 2024-04-25T14:39:55.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.167+0000 7fb708c00640 1 --2- 172.21.15.53:0/1199173754 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fb6e80a3bb0 0x7fb6e80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:55.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.167+0000 7fb708c00640 1 --2- 172.21.15.53:0/1199173754 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fb6e80a3bb0 0x7fb6e80a6070 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4005e30 tx=0x7fb6f4005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:55.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.169+0000 7fb701600640 1 -- 172.21.15.53:0/1199173754 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb6fc09d1e0 con 0x7fb704107ab0 2024-04-25T14:39:55.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:55 smithi053 ceph-mon[27773]: pgmap v370: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-25T14:39:55.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:55 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1828003099' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:39:55.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:55 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/2113084387' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:39:55.422 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.421+0000 7fb70a000640 1 -- 172.21.15.53:0/1199173754 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb70402d5c0 con 0x7fb704107ab0 2024-04-25T14:39:55.423 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.422+0000 7fb701600640 1 -- 172.21.15.53:0/1199173754 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11862 (secure 0 0 0) 0x7fb6fc09cae0 con 0x7fb704107ab0 2024-04-25T14:39:55.423 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:39:55.424 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14460,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6841/41110727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6840","nonce":41110727},{"type":"v1","addr":"172.21.15.53:6841","nonce":41110727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2966992472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2966992472},{"type":"v1","addr":"172.21.15.150:6833","nonce":2966992472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/94143643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":94143643},{"type":"v1","addr":"172.21.15.183:6833","nonce":94143643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24470,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/506682943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":506682943},{"type":"v1","addr":"172.21.15.53:6835","nonce":506682943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1369488844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1369488844},{"type":"v1","addr":"172.21.15.150:6835","nonce":1369488844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3786232682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3786232682},{"type":"v1","addr":"172.21.15.53:6837","nonce":3786232682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T14:38:10.037451+0000","modified":"2024-04-25T14:38:15.629854+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24584,"mds_1":24565,"mds_2":24560},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24532":{"gid":24532,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1189328602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1189328602},{"type":"v1","addr":"172.21.15.183:6835","nonce":1189328602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/2344343627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2344343627},{"type":"v1","addr":"172.21.15.150:6837","nonce":2344343627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24544":{"gid":24544,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1163336384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1163336384},{"type":"v1","addr":"172.21.15.183:6837","nonce":1163336384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6839/570000540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":570000540},{"type":"v1","addr":"172.21.15.53:6839","nonce":570000540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.183:6839/1554258915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1554258915},{"type":"v1","addr":"172.21.15.183:6839","nonce":1554258915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/149573616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":149573616},{"type":"v1","addr":"172.21.15.150:6839","nonce":149573616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24565,24584,24560]},"id":1}]} 2024-04-25T14:39:55.424 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 21 2024-04-25T14:39:55.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.425+0000 7fb70a000640 1 -- 172.21.15.53:0/1199173754 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fb6e80a3bb0 msgr2=0x7fb6e80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:55.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.425+0000 7fb70a000640 1 --2- 172.21.15.53:0/1199173754 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fb6e80a3bb0 0x7fb6e80a6070 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4005e30 tx=0x7fb6f4005dc0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.425+0000 7fb70a000640 1 -- 172.21.15.53:0/1199173754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb704107ab0 msgr2=0x7fb7041a17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:55.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.425+0000 7fb70a000640 1 --2- 172.21.15.53:0/1199173754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb704107ab0 0x7fb7041a17d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc00e970 tx=0x7fb6fc00ed70 comp rx=0 tx=0).stop 2024-04-25T14:39:55.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.425+0000 7fb70a000640 1 -- 172.21.15.53:0/1199173754 shutdown_connections 2024-04-25T14:39:55.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.425+0000 7fb70a000640 1 --2- 172.21.15.53:0/1199173754 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fb6e80a3bb0 0x7fb6e80a6070 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.425+0000 7fb70a000640 1 --2- 172.21.15.53:0/1199173754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb704107ab0 0x7fb7041a17d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.425+0000 7fb70a000640 1 --2- 172.21.15.53:0/1199173754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb704105180 0x7fb70419a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.425+0000 7fb70a000640 1 --2- 172.21.15.53:0/1199173754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb704102850 0x7fb70419a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.425+0000 7fb70a000640 1 -- 172.21.15.53:0/1199173754 >> 172.21.15.53:0/1199173754 conn(0x7fb7040fc220 msgr2=0x7fb704101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:55.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.425+0000 7fb70a000640 1 -- 172.21.15.53:0/1199173754 shutdown_connections 2024-04-25T14:39:55.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.425+0000 7fb70a000640 1 -- 172.21.15.53:0/1199173754 wait complete. 2024-04-25T14:39:55.445 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T14:38:10.037451+0000', 'modified': '2024-04-25T14:38:15.629854+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24584, 'mds_1': 24565, 'mds_2': 24560}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24532': {'gid': 24532, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6835/1189328602', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6834', 'nonce': 1189328602}, {'type': 'v1', 'addr': '172.21.15.183:6835', 'nonce': 1189328602}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24533': {'gid': 24533, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.150:6837/2344343627', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6836', 'nonce': 2344343627}, {'type': 'v1', 'addr': '172.21.15.150:6837', 'nonce': 2344343627}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24544': {'gid': 24544, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6837/1163336384', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6836', 'nonce': 1163336384}, {'type': 'v1', 'addr': '172.21.15.183:6837', 'nonce': 1163336384}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24560': {'gid': 24560, 'name': 'g', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.53:6839/570000540', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6838', 'nonce': 570000540}, {'type': 'v1', 'addr': '172.21.15.53:6839', 'nonce': 570000540}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24565': {'gid': 24565, 'name': 'i', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.183:6839/1554258915', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6838', 'nonce': 1554258915}, {'type': 'v1', 'addr': '172.21.15.183:6839', 'nonce': 1554258915}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24584': {'gid': 24584, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.150:6839/149573616', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6838', 'nonce': 149573616}, {'type': 'v1', 'addr': '172.21.15.150:6839', 'nonce': 149573616}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24584, 'qdb_cluster': [24565, 24584, 24560]} 2024-04-25T14:39:55.445 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-25T14:39:55.445 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2024-04-25T14:39:55.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:55 smithi183 ceph-mon[32584]: pgmap v370: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-25T14:39:55.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:55 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/1828003099' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:39:55.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:55 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/2113084387' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:39:55.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:55 smithi150 ceph-mon[32504]: pgmap v370: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-25T14:39:55.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:55 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/1828003099' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:39:55.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:55 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/2113084387' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:39:55.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.656+0000 7f095be00640 1 -- 172.21.15.53:0/3711913723 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f095c104090 msgr2=0x7f095c1044b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:55.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.656+0000 7f095be00640 1 --2- 172.21.15.53:0/3711913723 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f095c104090 0x7f095c1044b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0944009900 tx=0x7f094402a3f0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.657+0000 7f095be00640 1 -- 172.21.15.53:0/3711913723 shutdown_connections 2024-04-25T14:39:55.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.657+0000 7f095be00640 1 --2- 172.21.15.53:0/3711913723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f095c105c40 0x7f095c10c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.657+0000 7f095be00640 1 --2- 172.21.15.53:0/3711913723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f095c105280 0x7f095c105700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.657+0000 7f095be00640 1 --2- 172.21.15.53:0/3711913723 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f095c104090 0x7f095c1044b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.657+0000 7f095be00640 1 -- 172.21.15.53:0/3711913723 >> 172.21.15.53:0/3711913723 conn(0x7f095c0ff600 msgr2=0x7f095c101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:55.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.657+0000 7f095be00640 1 -- 172.21.15.53:0/3711913723 shutdown_connections 2024-04-25T14:39:55.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.657+0000 7f095be00640 1 -- 172.21.15.53:0/3711913723 wait complete. 2024-04-25T14:39:55.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.658+0000 7f095be00640 1 Processor -- start 2024-04-25T14:39:55.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.658+0000 7f095be00640 1 -- start start 2024-04-25T14:39:55.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.658+0000 7f095be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f095c104090 0x7f095c19a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:55.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.658+0000 7f095be00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f095c105280 0x7f095c19a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:55.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.658+0000 7f095be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f095c105c40 0x7f095c1a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:55.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.658+0000 7f095be00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f095c10fc20 con 0x7f095c104090 2024-04-25T14:39:55.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.658+0000 7f095be00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f095c10faa0 con 0x7f095c105280 2024-04-25T14:39:55.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.658+0000 7f095be00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f095c10fda0 con 0x7f095c105c40 2024-04-25T14:39:55.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.659+0000 7f095aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f095c104090 0x7f095c19a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:55.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.659+0000 7f095aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f095c104090 0x7f095c19a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59656/0 (socket says 172.21.15.53:59656) 2024-04-25T14:39:55.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.659+0000 7f095aa00640 1 -- 172.21.15.53:0/774735561 learned_addr learned my addr 172.21.15.53:0/774735561 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:55.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.659+0000 7f095b400640 1 --2- 172.21.15.53:0/774735561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f095c105c40 0x7f095c1a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:55.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.659+0000 7f095a000640 1 --2- 172.21.15.53:0/774735561 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f095c105280 0x7f095c19a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:55.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.659+0000 7f095aa00640 1 -- 172.21.15.53:0/774735561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f095c105c40 msgr2=0x7f095c1a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:55.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.659+0000 7f095aa00640 1 --2- 172.21.15.53:0/774735561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f095c105c40 0x7f095c1a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.659+0000 7f095aa00640 1 -- 172.21.15.53:0/774735561 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f095c105280 msgr2=0x7f095c19a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:55.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.659+0000 7f095aa00640 1 --2- 172.21.15.53:0/774735561 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f095c105280 0x7f095c19a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.659+0000 7f095aa00640 1 -- 172.21.15.53:0/774735561 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f095c1a1f40 con 0x7f095c104090 2024-04-25T14:39:55.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.659+0000 7f095b400640 1 --2- 172.21.15.53:0/774735561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f095c105c40 0x7f095c1a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:39:55.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.660+0000 7f095aa00640 1 --2- 172.21.15.53:0/774735561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f095c104090 0x7f095c19a320 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0944031500 tx=0x7f0944003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:55.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.660+0000 7f0953400640 1 -- 172.21.15.53:0/774735561 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094403f070 con 0x7f095c104090 2024-04-25T14:39:55.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.660+0000 7f0953400640 1 -- 172.21.15.53:0/774735561 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0944031c50 con 0x7f095c104090 2024-04-25T14:39:55.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.660+0000 7f095be00640 1 -- 172.21.15.53:0/774735561 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f095c1a21d0 con 0x7f095c104090 2024-04-25T14:39:55.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.660+0000 7f0953400640 1 -- 172.21.15.53:0/774735561 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0944035030 con 0x7f095c104090 2024-04-25T14:39:55.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.660+0000 7f095be00640 1 -- 172.21.15.53:0/774735561 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f095c1a26c0 con 0x7f095c104090 2024-04-25T14:39:55.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.663+0000 7f0953400640 1 -- 172.21.15.53:0/774735561 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0944042070 con 0x7f095c104090 2024-04-25T14:39:55.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.663+0000 7f0964564740 1 Processor -- start 2024-04-25T14:39:55.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.664+0000 7f0964564740 1 -- start start 2024-04-25T14:39:55.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.664+0000 7f0964564740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56381c2b3150 0x56381c2b3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:55.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.664+0000 7f0964564740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56381c2b3b40 0x56381c2b8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:55.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.664+0000 7f0964564740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56381c2b8cb0 0x56381c2bb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:55.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.664+0000 7f0964564740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56381c217b10 con 0x56381c2b3b40 2024-04-25T14:39:55.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.664+0000 7f0964564740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56381c1be2b0 con 0x56381c2b3150 2024-04-25T14:39:55.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.664+0000 7f0964564740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56381c213ad0 con 0x56381c2b8cb0 2024-04-25T14:39:55.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.664+0000 7f095aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56381c2b3b40 0x56381c2b8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:55.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.664+0000 7f095b400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56381c2b3150 0x56381c2b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:55.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.664+0000 7f095b400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56381c2b3150 0x56381c2b3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.53:60714/0 (socket says 172.21.15.53:60714) 2024-04-25T14:39:55.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.664+0000 7f095aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56381c2b3b40 0x56381c2b8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59662/0 (socket says 172.21.15.53:59662) 2024-04-25T14:39:55.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.664+0000 7f095b400640 1 -- 172.21.15.53:0/597088905 learned_addr learned my addr 172.21.15.53:0/597088905 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:55.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.664+0000 7f095a000640 1 --2- 172.21.15.53:0/597088905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56381c2b8cb0 0x56381c2bb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:55.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.665+0000 7f095b400640 1 -- 172.21.15.53:0/597088905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56381c2b8cb0 msgr2=0x56381c2bb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:39:55.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.665+0000 7f095b400640 1 --2- 172.21.15.53:0/597088905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56381c2b8cb0 0x56381c2bb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.665+0000 7f095b400640 1 -- 172.21.15.53:0/597088905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56381c2b3b40 msgr2=0x56381c2b8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:39:55.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.665+0000 7f095b400640 1 --2- 172.21.15.53:0/597088905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56381c2b3b40 0x56381c2b8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.665+0000 7f095b400640 1 -- 172.21.15.53:0/597088905 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56381bf2a2d0 con 0x56381c2b3150 2024-04-25T14:39:55.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.665+0000 7f095b400640 1 --2- 172.21.15.53:0/597088905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56381c2b3150 0x56381c2b3570 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0954010400 tx=0x7f09540108c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8197e9ca32b685b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:55.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.665+0000 7f0953400640 1 --2- 172.21.15.53:0/774735561 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f09400a3bb0 0x7f09400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:55.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.665+0000 7f0953400640 1 -- 172.21.15.53:0/774735561 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f09440f0c10 con 0x7f095c104090 2024-04-25T14:39:55.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.665+0000 7f094be00640 1 -- 172.21.15.53:0/597088905 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954004230 con 0x56381c2b3150 2024-04-25T14:39:55.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.665+0000 7f095aa00640 1 --2- 172.21.15.53:0/774735561 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f09400a3bb0 0x7f09400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:55.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.665+0000 7f094be00640 1 -- 172.21.15.53:0/597088905 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09540043d0 con 0x56381c2b3150 2024-04-25T14:39:55.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.665+0000 7f094be00640 1 -- 172.21.15.53:0/597088905 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954003a60 con 0x56381c2b3150 2024-04-25T14:39:55.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.666+0000 7f0964564740 1 -- 172.21.15.53:0/597088905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56381c2b3150 msgr2=0x56381c2b3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:55.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.666+0000 7f095aa00640 1 --2- 172.21.15.53:0/774735561 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f09400a3bb0 0x7f09400a6070 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f09440ee6a0 tx=0x7f09440f4650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:55.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.666+0000 7f0964564740 1 --2- 172.21.15.53:0/597088905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56381c2b3150 0x56381c2b3570 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0954010400 tx=0x7f09540108c0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.666+0000 7f0964564740 1 -- 172.21.15.53:0/597088905 shutdown_connections 2024-04-25T14:39:55.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.666+0000 7f0964564740 1 --2- 172.21.15.53:0/597088905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56381c2b8cb0 0x56381c2bb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.666+0000 7f0964564740 1 --2- 172.21.15.53:0/597088905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56381c2b3b40 0x56381c2b8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.666+0000 7f0964564740 1 --2- 172.21.15.53:0/597088905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56381c2b3150 0x56381c2b3570 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.666+0000 7f0964564740 1 -- 172.21.15.53:0/597088905 >> 172.21.15.53:0/597088905 conn(0x56381c2ae7c0 msgr2=0x56381c2aebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:55.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.666+0000 7f0964564740 1 -- 172.21.15.53:0/597088905 shutdown_connections 2024-04-25T14:39:55.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.666+0000 7f0964564740 1 -- 172.21.15.53:0/597088905 wait complete. 2024-04-25T14:39:55.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.667+0000 7f0964564740 1 -- start start 2024-04-25T14:39:55.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.667+0000 7f0964564740 1 Processor -- start 2024-04-25T14:39:55.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.667+0000 7f0964564740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56381c2b3150 0x56381c2bd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:55.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.667+0000 7f0964564740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56381c2b3b40 0x56381c2bd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:55.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.667+0000 7f0964564740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56381c2b8cb0 0x56381c2c48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:55.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.667+0000 7f0964564740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56381c2bbc60 con 0x56381c2b8cb0 2024-04-25T14:39:55.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.668+0000 7f0964564740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56381c1d1570 con 0x56381c2b3150 2024-04-25T14:39:55.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.668+0000 7f0964564740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56381c2bbde0 con 0x56381c2b3b40 2024-04-25T14:39:55.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.668+0000 7f0964564740 10 client.0 _ll_register_callbacks cb 0x56381c19ffc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:39:55.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.668+0000 7f095b400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56381c2b3150 0x56381c2bd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:55.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.668+0000 7f095b400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56381c2b3150 0x56381c2bd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.53:60726/0 (socket says 172.21.15.53:60726) 2024-04-25T14:39:55.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.668+0000 7f095a000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56381c2b3b40 0x56381c2bd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:55.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.668+0000 7f095b400640 1 -- 172.21.15.53:0/3620655860 learned_addr learned my addr 172.21.15.53:0/3620655860 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:55.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.668+0000 7f095b400640 1 --2- 172.21.15.53:0/3620655860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56381c2b8cb0 0x56381c2c48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:55.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.668+0000 7f095a000640 1 -- 172.21.15.53:0/3620655860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56381c2b3150 msgr2=0x56381c2bd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:39:55.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.668+0000 7f095a000640 1 --2- 172.21.15.53:0/3620655860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56381c2b3150 0x56381c2bd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.668+0000 7f095a000640 1 -- 172.21.15.53:0/3620655860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56381c2b8cb0 msgr2=0x56381c2c48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:39:55.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.668+0000 7f095a000640 1 --2- 172.21.15.53:0/3620655860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56381c2b8cb0 0x56381c2c48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.668+0000 7f095a000640 1 -- 172.21.15.53:0/3620655860 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56381bff46b0 con 0x56381c2b3b40 2024-04-25T14:39:55.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.668+0000 7f095b400640 1 --2- 172.21.15.53:0/3620655860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56381c2b8cb0 0x56381c2c48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:39:55.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.669+0000 7f095a000640 1 --2- 172.21.15.53:0/3620655860 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56381c2b3b40 0x56381c2bd890 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f095c19b830 tx=0x7f094c012de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa5587db21426ade server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:55.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.669+0000 7f0948c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T14:39:55.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.669+0000 7f0948c00640 1 -- 172.21.15.53:0/3620655860 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c003970 con 0x56381c2b3b40 2024-04-25T14:39:55.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.669+0000 7f0948c00640 1 -- 172.21.15.53:0/3620655860 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f094c003b10 con 0x56381c2b3b40 2024-04-25T14:39:55.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.669+0000 7f0964564740 1 -- 172.21.15.53:0/3620655860 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09540043d0 con 0x56381c2b3b40 2024-04-25T14:39:55.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.669+0000 7f0948c00640 1 -- 172.21.15.53:0/3620655860 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f094c003d10 con 0x56381c2b3b40 2024-04-25T14:39:55.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.669+0000 7f0964564740 10 client.25006 fetch_fsmap learned FSMap version 21 2024-04-25T14:39:55.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.669+0000 7f0964564740 1 -- 172.21.15.53:0/3620655860 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56381c2c7d10 con 0x56381c2b3b40 2024-04-25T14:39:55.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.669+0000 7f0948c00640 1 -- 172.21.15.53:0/3620655860 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09540043d0 con 0x56381c2b3b40 2024-04-25T14:39:55.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.670+0000 7f0948c00640 1 -- 172.21.15.53:0/3620655860 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f094c004460 con 0x56381c2b3b40 2024-04-25T14:39:55.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.670+0000 7f0964564740 10 client.25006 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T14:39:55.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.670+0000 7f0964564740 10 client.25006 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-25T14:39:55.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.670+0000 7f0964564740 20 client.25006 populate_metadata read hostname 'smithi053' 2024-04-25T14:39:55.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.670+0000 7f0964564740 1 --2- 172.21.15.53:0/3620655860 >> [v2:172.21.15.53:6838/570000540,v1:172.21.15.53:6839/570000540] conn(0x56381c2ca360 0x56381c2cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:55.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.670+0000 7f0964564740 4 client.25006 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:39:55.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.670+0000 7f0964564740 1 -- 172.21.15.53:0/3620655860 --> [v2:172.21.15.53:6838/570000540,v1:172.21.15.53:6839/570000540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56381c225530 con 0x56381c2ca360 2024-04-25T14:39:55.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.670+0000 7f095b400640 1 --2- 172.21.15.53:0/3620655860 >> [v2:172.21.15.53:6838/570000540,v1:172.21.15.53:6839/570000540] conn(0x56381c2ca360 0x56381c2cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:55.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.671+0000 7f095b400640 1 --2- 172.21.15.53:0/3620655860 >> [v2:172.21.15.53:6838/570000540,v1:172.21.15.53:6839/570000540] conn(0x56381c2ca360 0x56381c2cb7d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5832f68192e0e700 server_cookie=a56cb5b8f26ce95a in_seq=0 out_seq=0 2024-04-25T14:39:55.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.671+0000 7f0948c00640 10 client.25006 ms_handle_connect on v2:172.21.15.53:6838/570000540 2024-04-25T14:39:55.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.672+0000 7f0948c00640 1 -- 172.21.15.53:0/3620655860 <== mds.2 v2:172.21.15.53:6838/570000540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56381c225530 con 0x56381c2ca360 2024-04-25T14:39:55.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.672+0000 7f0948c00640 10 client.25006 handle_command_reply: tid=0 2024-04-25T14:39:55.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.672+0000 7f0964564740 1 client.25006 shutdown 2024-04-25T14:39:55.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.672+0000 7f0964564740 1 -- 172.21.15.53:0/3620655860 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56381c2b3b40 msgr2=0x56381c2bd890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:55.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.672+0000 7f0964564740 1 --2- 172.21.15.53:0/3620655860 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56381c2b3b40 0x56381c2bd890 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f095c19b830 tx=0x7f094c012de0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.672+0000 7f0964564740 1 -- 172.21.15.53:0/3620655860 shutdown_connections 2024-04-25T14:39:55.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.672+0000 7f0964564740 1 --2- 172.21.15.53:0/3620655860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56381c2b8cb0 0x56381c2c48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.672+0000 7f0964564740 1 --2- 172.21.15.53:0/3620655860 >> [v2:172.21.15.53:6838/570000540,v1:172.21.15.53:6839/570000540] conn(0x56381c2ca360 0x56381c2cb7d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.672+0000 7f0964564740 1 --2- 172.21.15.53:0/3620655860 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56381c2b3b40 0x56381c2bd890 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.672+0000 7f0964564740 1 --2- 172.21.15.53:0/3620655860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56381c2b3150 0x56381c2bd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.672+0000 7f0948c00640 0 client.25006 ms_handle_reset on v2:172.21.15.53:6838/570000540 2024-04-25T14:39:55.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.672+0000 7f0964564740 1 -- 172.21.15.53:0/3620655860 >> 172.21.15.53:0/3620655860 conn(0x56381c2ae7c0 msgr2=0x56381c2ada40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:55.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.673+0000 7f0964564740 1 -- 172.21.15.53:0/3620655860 shutdown_connections 2024-04-25T14:39:55.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.673+0000 7f0964564740 1 -- 172.21.15.53:0/3620655860 wait complete. 2024-04-25T14:39:55.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.673+0000 7f0964564740 20 client.25006 trim_cache size 0 max 16384 2024-04-25T14:39:55.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.693+0000 7f0964564740 1 Processor -- start 2024-04-25T14:39:55.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.693+0000 7f0964564740 1 -- start start 2024-04-25T14:39:55.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.693+0000 7f0964564740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56381c2b3150 0x56381c2c5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:55.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.693+0000 7f0964564740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56381c2b3b40 0x56381c2c4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:55.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.693+0000 7f0964564740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56381c2b8cb0 0x56381c2b83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:55.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.693+0000 7f0964564740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56381c2c9b10 con 0x56381c2b8cb0 2024-04-25T14:39:55.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.693+0000 7f0964564740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56381c2ca1e0 con 0x56381c2b3b40 2024-04-25T14:39:55.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.693+0000 7f0964564740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56381c2ca8b0 con 0x56381c2b3150 2024-04-25T14:39:55.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.694+0000 7f095b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56381c2b8cb0 0x56381c2b83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:55.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.694+0000 7f095a000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56381c2b3b40 0x56381c2c4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:55.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.694+0000 7f095b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56381c2b8cb0 0x56381c2b83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59680/0 (socket says 172.21.15.53:59680) 2024-04-25T14:39:55.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.694+0000 7f095a000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56381c2b3b40 0x56381c2c4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.53:60742/0 (socket says 172.21.15.53:60742) 2024-04-25T14:39:55.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.694+0000 7f095b400640 1 -- 172.21.15.53:0/1257075190 learned_addr learned my addr 172.21.15.53:0/1257075190 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:55.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.694+0000 7f095b400640 1 --2- 172.21.15.53:0/1257075190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56381c2b3150 0x56381c2c5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:55.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.694+0000 7f095a000640 1 -- 172.21.15.53:0/1257075190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56381c2b3150 msgr2=0x56381c2c5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:39:55.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.694+0000 7f095a000640 1 --2- 172.21.15.53:0/1257075190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56381c2b3150 0x56381c2c5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.694+0000 7f095a000640 1 -- 172.21.15.53:0/1257075190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56381c2b8cb0 msgr2=0x56381c2b83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:39:55.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.694+0000 7f095a000640 1 --2- 172.21.15.53:0/1257075190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56381c2b8cb0 0x56381c2b83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.694+0000 7f095a000640 1 -- 172.21.15.53:0/1257075190 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56381c2a8ef0 con 0x56381c2b3b40 2024-04-25T14:39:55.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.694+0000 7f095b400640 1 --2- 172.21.15.53:0/1257075190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56381c2b3150 0x56381c2c5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:39:55.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.694+0000 7f095a000640 1 --2- 172.21.15.53:0/1257075190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56381c2b3b40 0x56381c2c4890 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f094c014000 tx=0x7f094c003b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a9ddf619af32c4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:55.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.695+0000 7f094b400640 1 -- 172.21.15.53:0/1257075190 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c0095b0 con 0x56381c2b3b40 2024-04-25T14:39:55.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.695+0000 7f094b400640 1 -- 172.21.15.53:0/1257075190 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f094c00f790 con 0x56381c2b3b40 2024-04-25T14:39:55.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.695+0000 7f094b400640 1 -- 172.21.15.53:0/1257075190 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c0041a0 con 0x56381c2b3b40 2024-04-25T14:39:55.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.695+0000 7f0964564740 1 -- 172.21.15.53:0/1257075190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56381c2b3b40 msgr2=0x56381c2c4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:55.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.695+0000 7f0964564740 1 --2- 172.21.15.53:0/1257075190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56381c2b3b40 0x56381c2c4890 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f094c014000 tx=0x7f094c003b80 comp rx=0 tx=0).stop 2024-04-25T14:39:55.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.695+0000 7f0964564740 1 -- 172.21.15.53:0/1257075190 shutdown_connections 2024-04-25T14:39:55.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.695+0000 7f0964564740 1 --2- 172.21.15.53:0/1257075190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56381c2b8cb0 0x56381c2b83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.695+0000 7f0964564740 1 --2- 172.21.15.53:0/1257075190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56381c2b3b40 0x56381c2c4890 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.695+0000 7f0964564740 1 --2- 172.21.15.53:0/1257075190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56381c2b3150 0x56381c2c5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.695+0000 7f0964564740 1 -- 172.21.15.53:0/1257075190 >> 172.21.15.53:0/1257075190 conn(0x56381c2ae7c0 msgr2=0x56381c2aeba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:55.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.696+0000 7f0964564740 1 -- 172.21.15.53:0/1257075190 shutdown_connections 2024-04-25T14:39:55.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.696+0000 7f0964564740 1 -- 172.21.15.53:0/1257075190 wait complete. 2024-04-25T14:39:55.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.696+0000 7f0964564740 1 -- start start 2024-04-25T14:39:55.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.696+0000 7f0964564740 1 Processor -- start 2024-04-25T14:39:55.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.697+0000 7f0964564740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56381c2b3150 0x56381c2c2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:55.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.697+0000 7f0964564740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56381c2b3b40 0x56381c32c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:55.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.697+0000 7f0964564740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56381c2b8cb0 0x56381c32e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:55.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.697+0000 7f0964564740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56381c2cb650 con 0x56381c2b3b40 2024-04-25T14:39:55.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.697+0000 7f0964564740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56381c2caf80 con 0x56381c2b8cb0 2024-04-25T14:39:55.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.697+0000 7f0964564740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56381c2cbd20 con 0x56381c2b3150 2024-04-25T14:39:55.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.697+0000 7f0964564740 10 client.0 _ll_register_callbacks cb 0x7f094c012590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:39:55.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.697+0000 7f095a000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56381c2b3b40 0x56381c32c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:55.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.697+0000 7f095b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56381c2b3150 0x56381c2c2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:55.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.697+0000 7f095b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56381c2b3150 0x56381c2c2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:35388/0 (socket says 172.21.15.53:35388) 2024-04-25T14:39:55.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.697+0000 7f095b400640 1 -- 172.21.15.53:0/3118408189 learned_addr learned my addr 172.21.15.53:0/3118408189 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:55.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.697+0000 7f095b400640 1 --2- 172.21.15.53:0/3118408189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56381c2b8cb0 0x56381c32e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:55.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.697+0000 7f095a000640 1 -- 172.21.15.53:0/3118408189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56381c2b3150 msgr2=0x56381c2c2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:39:55.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.697+0000 7f095a000640 1 --2- 172.21.15.53:0/3118408189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56381c2b3150 0x56381c2c2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.697+0000 7f095a000640 1 -- 172.21.15.53:0/3118408189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56381c2b8cb0 msgr2=0x56381c32e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:39:55.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.697+0000 7f095a000640 1 --2- 172.21.15.53:0/3118408189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56381c2b8cb0 0x56381c32e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.697+0000 7f095a000640 1 -- 172.21.15.53:0/3118408189 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56381c32ee50 con 0x56381c2b3b40 2024-04-25T14:39:55.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.698+0000 7f095a000640 1 --2- 172.21.15.53:0/3118408189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56381c2b3b40 0x56381c32c2f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f094c000f80 tx=0x7f094c009690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f41ec9e05244581 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:55.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.698+0000 7f0948c00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-04-25T14:39:55.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.698+0000 7f0948c00640 1 -- 172.21.15.53:0/3118408189 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c00fbd0 con 0x56381c2b3b40 2024-04-25T14:39:55.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.698+0000 7f0948c00640 1 -- 172.21.15.53:0/3118408189 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f094c00fd70 con 0x56381c2b3b40 2024-04-25T14:39:55.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.698+0000 7f0964564740 1 -- 172.21.15.53:0/3118408189 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f094c00f790 con 0x56381c2b3b40 2024-04-25T14:39:55.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.698+0000 7f0948c00640 1 -- 172.21.15.53:0/3118408189 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f094c003f20 con 0x56381c2b3b40 2024-04-25T14:39:55.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.698+0000 7f0964564740 10 client.14880 fetch_fsmap learned FSMap version 21 2024-04-25T14:39:55.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.698+0000 7f0964564740 1 -- 172.21.15.53:0/3118408189 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56381c330150 con 0x56381c2b3b40 2024-04-25T14:39:55.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.699+0000 7f0948c00640 1 -- 172.21.15.53:0/3118408189 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f094c00f790 con 0x56381c2b3b40 2024-04-25T14:39:55.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.699+0000 7f0948c00640 1 -- 172.21.15.53:0/3118408189 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f094c043050 con 0x56381c2b3b40 2024-04-25T14:39:55.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.699+0000 7f0964564740 10 client.14880 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T14:39:55.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.699+0000 7f0964564740 10 client.14880 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-25T14:39:55.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.699+0000 7f0964564740 20 client.14880 populate_metadata read hostname 'smithi053' 2024-04-25T14:39:55.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.699+0000 7f0964564740 1 --2- 172.21.15.53:0/3118408189 >> [v2:172.21.15.53:6838/570000540,v1:172.21.15.53:6839/570000540] conn(0x56381c334b50 0x56381c333860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:55.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.699+0000 7f0964564740 4 client.14880 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T14:39:55.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.699+0000 7f0964564740 1 -- 172.21.15.53:0/3118408189 --> [v2:172.21.15.53:6838/570000540,v1:172.21.15.53:6839/570000540] -- command(tid 0: {"prefix": "status"}) -- 0x56381c21f3a0 con 0x56381c334b50 2024-04-25T14:39:55.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.699+0000 7f095b400640 1 --2- 172.21.15.53:0/3118408189 >> [v2:172.21.15.53:6838/570000540,v1:172.21.15.53:6839/570000540] conn(0x56381c334b50 0x56381c333860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:55.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.700+0000 7f095b400640 1 --2- 172.21.15.53:0/3118408189 >> [v2:172.21.15.53:6838/570000540,v1:172.21.15.53:6839/570000540] conn(0x56381c334b50 0x56381c333860 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8666b8c96a42dc27 server_cookie=ca5af24b884a0c61 in_seq=0 out_seq=0 2024-04-25T14:39:55.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.700+0000 7f0948c00640 10 client.14880 ms_handle_connect on v2:172.21.15.53:6838/570000540 2024-04-25T14:39:55.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.700+0000 7f0948c00640 1 -- 172.21.15.53:0/3118408189 <== mds.2 v2:172.21.15.53:6838/570000540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x56381c21f3a0 con 0x56381c334b50 2024-04-25T14:39:55.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.700+0000 7f0948c00640 10 client.14880 handle_command_reply: tid=0 2024-04-25T14:39:55.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.700+0000 7f0964564740 1 client.14880 shutdown 2024-04-25T14:39:55.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.700+0000 7f0964564740 1 -- 172.21.15.53:0/3118408189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56381c2b3b40 msgr2=0x56381c32c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:55.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.700+0000 7f0964564740 1 --2- 172.21.15.53:0/3118408189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56381c2b3b40 0x56381c32c2f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f094c000f80 tx=0x7f094c009690 comp rx=0 tx=0).stop 2024-04-25T14:39:55.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.700+0000 7f0964564740 1 -- 172.21.15.53:0/3118408189 shutdown_connections 2024-04-25T14:39:55.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.700+0000 7f0964564740 1 --2- 172.21.15.53:0/3118408189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56381c2b8cb0 0x56381c32e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.700+0000 7f0964564740 1 --2- 172.21.15.53:0/3118408189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56381c2b3b40 0x56381c32c2f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.700+0000 7f0964564740 1 --2- 172.21.15.53:0/3118408189 >> [v2:172.21.15.53:6838/570000540,v1:172.21.15.53:6839/570000540] conn(0x56381c334b50 0x56381c333860 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.700+0000 7f0964564740 1 --2- 172.21.15.53:0/3118408189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56381c2b3150 0x56381c2c2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.700+0000 7f0964564740 1 -- 172.21.15.53:0/3118408189 >> 172.21.15.53:0/3118408189 conn(0x56381c2ae7c0 msgr2=0x56381c2b7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:55.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.700+0000 7f0948c00640 0 client.14880 ms_handle_reset on v2:172.21.15.53:6838/570000540 2024-04-25T14:39:55.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.701+0000 7f0964564740 1 -- 172.21.15.53:0/3118408189 shutdown_connections 2024-04-25T14:39:55.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.701+0000 7f0964564740 1 -- 172.21.15.53:0/3118408189 wait complete. 2024-04-25T14:39:55.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.701+0000 7f0964564740 20 client.14880 trim_cache size 0 max 16384 2024-04-25T14:39:55.705 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-04-25T14:39:55.706 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "fd30e6ce-030f-11ef-bc93-c7b262605968", 2024-04-25T14:39:55.706 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 2, 2024-04-25T14:39:55.706 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 24560, 2024-04-25T14:39:55.706 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2024-04-25T14:39:55.706 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2024-04-25T14:39:55.706 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2024-04-25T14:39:55.706 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 100.688738628, 2024-04-25T14:39:55.706 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 21, 2024-04-25T14:39:55.706 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 90, 2024-04-25T14:39:55.706 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 88, 2024-04-25T14:39:55.706 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 117.533340224 2024-04-25T14:39:55.706 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-04-25T14:39:55.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.701+0000 7f095be00640 1 -- 172.21.15.53:0/774735561 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f09400a3bb0 msgr2=0x7f09400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:55.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.701+0000 7f095be00640 1 --2- 172.21.15.53:0/774735561 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f09400a3bb0 0x7f09400a6070 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f09440ee6a0 tx=0x7f09440f4650 comp rx=0 tx=0).stop 2024-04-25T14:39:55.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.701+0000 7f095be00640 1 -- 172.21.15.53:0/774735561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f095c104090 msgr2=0x7f095c19a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:55.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.701+0000 7f095be00640 1 --2- 172.21.15.53:0/774735561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f095c104090 0x7f095c19a320 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0944031500 tx=0x7f0944003c10 comp rx=0 tx=0).stop 2024-04-25T14:39:55.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.702+0000 7f095be00640 1 -- 172.21.15.53:0/774735561 shutdown_connections 2024-04-25T14:39:55.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.702+0000 7f095be00640 1 --2- 172.21.15.53:0/774735561 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f09400a3bb0 0x7f09400a6070 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.702+0000 7f095be00640 1 --2- 172.21.15.53:0/774735561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f095c105c40 0x7f095c1a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.702+0000 7f095be00640 1 --2- 172.21.15.53:0/774735561 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f095c105280 0x7f095c19a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.702+0000 7f095be00640 1 --2- 172.21.15.53:0/774735561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f095c104090 0x7f095c19a320 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.702+0000 7f095be00640 1 -- 172.21.15.53:0/774735561 >> 172.21.15.53:0/774735561 conn(0x7f095c0ff600 msgr2=0x7f095c108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:55.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.702+0000 7f095be00640 1 -- 172.21.15.53:0/774735561 shutdown_connections 2024-04-25T14:39:55.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.702+0000 7f095be00640 1 -- 172.21.15.53:0/774735561 wait complete. 2024-04-25T14:39:55.721 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-25T14:39:55.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.915+0000 7fce95a00640 1 -- 172.21.15.53:0/259809871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce90105270 msgr2=0x7fce90107660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:55.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.915+0000 7fce95a00640 1 --2- 172.21.15.53:0/259809871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce90105270 0x7fce90107660 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fce80009ca0 tx=0x7fce8002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.915+0000 7fce95a00640 1 -- 172.21.15.53:0/259809871 shutdown_connections 2024-04-25T14:39:55.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.915+0000 7fce95a00640 1 --2- 172.21.15.53:0/259809871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce90107ba0 0x7fce90109f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.915+0000 7fce95a00640 1 --2- 172.21.15.53:0/259809871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce90105270 0x7fce90107660 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.915+0000 7fce95a00640 1 --2- 172.21.15.53:0/259809871 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce90102940 0x7fce90104d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.915+0000 7fce95a00640 1 -- 172.21.15.53:0/259809871 >> 172.21.15.53:0/259809871 conn(0x7fce900fc2f0 msgr2=0x7fce900fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:55.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.915+0000 7fce95a00640 1 -- 172.21.15.53:0/259809871 shutdown_connections 2024-04-25T14:39:55.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.916+0000 7fce95a00640 1 -- 172.21.15.53:0/259809871 wait complete. 2024-04-25T14:39:55.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.917+0000 7fce95a00640 1 Processor -- start 2024-04-25T14:39:55.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.917+0000 7fce95a00640 1 -- start start 2024-04-25T14:39:55.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.917+0000 7fce95a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce90102940 0x7fce90198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:55.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.917+0000 7fce95a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce90105270 0x7fce901985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:55.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.917+0000 7fce95a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce90107ba0 0x7fce9019f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:55.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.917+0000 7fce95a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fce9010d9b0 con 0x7fce90102940 2024-04-25T14:39:55.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.917+0000 7fce95a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fce9010d830 con 0x7fce90105270 2024-04-25T14:39:55.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.917+0000 7fce95a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fce9010db30 con 0x7fce90107ba0 2024-04-25T14:39:55.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.918+0000 7fce8fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce90102940 0x7fce90198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:55.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.918+0000 7fce8fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce90102940 0x7fce90198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59714/0 (socket says 172.21.15.53:59714) 2024-04-25T14:39:55.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.918+0000 7fce8fe00640 1 -- 172.21.15.53:0/1091734320 learned_addr learned my addr 172.21.15.53:0/1091734320 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:55.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.918+0000 7fce8f400640 1 --2- 172.21.15.53:0/1091734320 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce90105270 0x7fce901985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:55.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.918+0000 7fce95000640 1 --2- 172.21.15.53:0/1091734320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce90107ba0 0x7fce9019f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:55.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.918+0000 7fce8fe00640 1 -- 172.21.15.53:0/1091734320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce90107ba0 msgr2=0x7fce9019f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:55.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.918+0000 7fce8fe00640 1 --2- 172.21.15.53:0/1091734320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce90107ba0 0x7fce9019f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.918+0000 7fce8fe00640 1 -- 172.21.15.53:0/1091734320 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce90105270 msgr2=0x7fce901985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:55.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.918+0000 7fce8fe00640 1 --2- 172.21.15.53:0/1091734320 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce90105270 0x7fce901985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.918+0000 7fce8fe00640 1 -- 172.21.15.53:0/1091734320 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce9019fca0 con 0x7fce90102940 2024-04-25T14:39:55.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.918+0000 7fce8fe00640 1 --2- 172.21.15.53:0/1091734320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce90102940 0x7fce90198080 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fce7800d950 tx=0x7fce7800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:55.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.919+0000 7fce8cc00640 1 -- 172.21.15.53:0/1091734320 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce78017030 con 0x7fce90102940 2024-04-25T14:39:55.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.919+0000 7fce8cc00640 1 -- 172.21.15.53:0/1091734320 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce78004020 con 0x7fce90102940 2024-04-25T14:39:55.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.919+0000 7fce95a00640 1 -- 172.21.15.53:0/1091734320 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce9019ff90 con 0x7fce90102940 2024-04-25T14:39:55.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.919+0000 7fce8cc00640 1 -- 172.21.15.53:0/1091734320 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce780047a0 con 0x7fce90102940 2024-04-25T14:39:55.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.919+0000 7fce95a00640 1 -- 172.21.15.53:0/1091734320 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce901a03f0 con 0x7fce90102940 2024-04-25T14:39:55.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.923+0000 7fce99fd8740 1 Processor -- start 2024-04-25T14:39:55.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.923+0000 7fce99fd8740 1 -- start start 2024-04-25T14:39:55.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.924+0000 7fce99fd8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef174c560 0x558ef174c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:55.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.924+0000 7fce99fd8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ef174cf50 0x558ef1751a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:55.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.924+0000 7fce99fd8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ef17520c0 0x558ef1754550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:55.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.924+0000 7fce99fd8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558ef16acb10 con 0x558ef17520c0 2024-04-25T14:39:55.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.924+0000 7fce99fd8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558ef16532b0 con 0x558ef174cf50 2024-04-25T14:39:55.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.924+0000 7fce99fd8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558ef16a8ad0 con 0x558ef174c560 2024-04-25T14:39:55.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.924+0000 7fce8f400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ef17520c0 0x558ef1754550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:55.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.924+0000 7fce95000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef174c560 0x558ef174c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:55.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.924+0000 7fce8f400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ef17520c0 0x558ef1754550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59730/0 (socket says 172.21.15.53:59730) 2024-04-25T14:39:55.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.924+0000 7fce8f400640 1 -- 172.21.15.53:0/3898724039 learned_addr learned my addr 172.21.15.53:0/3898724039 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:55.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.924+0000 7fce8cc00640 1 -- 172.21.15.53:0/1091734320 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fce78009a30 con 0x7fce90102940 2024-04-25T14:39:55.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.925+0000 7fce95000640 1 -- 172.21.15.53:0/3898724039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ef174cf50 msgr2=0x558ef1751a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:39:55.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.925+0000 7fce95000640 1 --2- 172.21.15.53:0/3898724039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ef174cf50 0x558ef1751a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.925+0000 7fce95000640 1 -- 172.21.15.53:0/3898724039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ef17520c0 msgr2=0x558ef1754550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:39:55.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.925+0000 7fce95000640 1 --2- 172.21.15.53:0/3898724039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ef17520c0 0x558ef1754550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.925+0000 7fce95000640 1 -- 172.21.15.53:0/3898724039 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ef13bf2d0 con 0x558ef174c560 2024-04-25T14:39:55.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.925+0000 7fce95000640 1 --2- 172.21.15.53:0/3898724039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef174c560 0x558ef174c980 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fce880045c0 tx=0x7fce8800ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39a22de6a65f384 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:55.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.925+0000 7fce85600640 1 -- 172.21.15.53:0/3898724039 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce88003a70 con 0x558ef174c560 2024-04-25T14:39:55.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.925+0000 7fce85600640 1 -- 172.21.15.53:0/3898724039 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce88015070 con 0x558ef174c560 2024-04-25T14:39:55.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.925+0000 7fce85600640 1 -- 172.21.15.53:0/3898724039 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce880116a0 con 0x558ef174c560 2024-04-25T14:39:55.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.926+0000 7fce99fd8740 1 -- 172.21.15.53:0/3898724039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef174c560 msgr2=0x558ef174c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:55.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.926+0000 7fce99fd8740 1 --2- 172.21.15.53:0/3898724039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef174c560 0x558ef174c980 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fce880045c0 tx=0x7fce8800ba90 comp rx=0 tx=0).stop 2024-04-25T14:39:55.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.926+0000 7fce99fd8740 1 -- 172.21.15.53:0/3898724039 shutdown_connections 2024-04-25T14:39:55.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.926+0000 7fce99fd8740 1 --2- 172.21.15.53:0/3898724039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ef17520c0 0x558ef1754550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.926+0000 7fce99fd8740 1 --2- 172.21.15.53:0/3898724039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ef174cf50 0x558ef1751a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.926+0000 7fce99fd8740 1 --2- 172.21.15.53:0/3898724039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef174c560 0x558ef174c980 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.926+0000 7fce99fd8740 1 -- 172.21.15.53:0/3898724039 >> 172.21.15.53:0/3898724039 conn(0x558ef17437c0 msgr2=0x558ef1743bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:55.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.926+0000 7fce8cc00640 1 --2- 172.21.15.53:0/1091734320 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fce740a3bb0 0x7fce740a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:55.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.926+0000 7fce99fd8740 1 -- 172.21.15.53:0/3898724039 shutdown_connections 2024-04-25T14:39:55.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.926+0000 7fce99fd8740 1 -- 172.21.15.53:0/3898724039 wait complete. 2024-04-25T14:39:55.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.927+0000 7fce8cc00640 1 -- 172.21.15.53:0/1091734320 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fce780cfe80 con 0x7fce90102940 2024-04-25T14:39:55.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.927+0000 7fce8f400640 1 --2- 172.21.15.53:0/1091734320 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fce740a3bb0 0x7fce740a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:55.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.927+0000 7fce99fd8740 1 -- start start 2024-04-25T14:39:55.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.927+0000 7fce99fd8740 1 Processor -- start 2024-04-25T14:39:55.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.928+0000 7fce8f400640 1 --2- 172.21.15.53:0/1091734320 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fce740a3bb0 0x7fce740a6070 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x558ef1744690 tx=0x7fce80000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:55.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.928+0000 7fce99fd8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef174c560 0x558ef1756820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:55.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.928+0000 7fce99fd8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ef174cf50 0x558ef1756d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:55.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.928+0000 7fce99fd8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ef17520c0 0x558ef175dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:55.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.928+0000 7fce99fd8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558ef1755070 con 0x558ef17520c0 2024-04-25T14:39:55.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.928+0000 7fce99fd8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558ef1666570 con 0x558ef174cf50 2024-04-25T14:39:55.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.928+0000 7fce99fd8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558ef17551f0 con 0x558ef174c560 2024-04-25T14:39:55.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.928+0000 7fce99fd8740 10 client.0 _ll_register_callbacks cb 0x558ef1634fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:39:55.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.928+0000 7fce95000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ef17520c0 0x558ef175dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:55.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.928+0000 7fce8f400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ef174cf50 0x558ef1756d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:55.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.928+0000 7fce8fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef174c560 0x558ef1756820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:55.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.928+0000 7fce95000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ef17520c0 0x558ef175dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59734/0 (socket says 172.21.15.53:59734) 2024-04-25T14:39:55.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.928+0000 7fce95000640 1 -- 172.21.15.53:0/749212030 learned_addr learned my addr 172.21.15.53:0/749212030 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:55.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.928+0000 7fce8fe00640 1 -- 172.21.15.53:0/749212030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ef174cf50 msgr2=0x558ef1756d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:39:55.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.928+0000 7fce8fe00640 1 --2- 172.21.15.53:0/749212030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ef174cf50 0x558ef1756d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.928+0000 7fce8fe00640 1 -- 172.21.15.53:0/749212030 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ef17520c0 msgr2=0x558ef175dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:39:55.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.928+0000 7fce8fe00640 1 --2- 172.21.15.53:0/749212030 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ef17520c0 0x558ef175dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.929+0000 7fce8fe00640 1 -- 172.21.15.53:0/749212030 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ef14896b0 con 0x558ef174c560 2024-04-25T14:39:55.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.929+0000 7fce95000640 1 --2- 172.21.15.53:0/749212030 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ef17520c0 0x558ef175dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:39:55.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.929+0000 7fce8fe00640 1 --2- 172.21.15.53:0/749212030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef174c560 0x558ef1756820 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x558ef174d9c0 tx=0x7fce7809c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=678eaacff0a78298 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:55.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.929+0000 7fce7e000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T14:39:55.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.929+0000 7fce7e000640 1 -- 172.21.15.53:0/749212030 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce780a4d60 con 0x558ef174c560 2024-04-25T14:39:55.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.929+0000 7fce7e000640 1 -- 172.21.15.53:0/749212030 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fce780041d0 con 0x558ef174c560 2024-04-25T14:39:55.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.929+0000 7fce99fd8740 1 -- 172.21.15.53:0/749212030 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce88015070 con 0x558ef174c560 2024-04-25T14:39:55.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.929+0000 7fce7e000640 1 -- 172.21.15.53:0/749212030 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fce780a0120 con 0x558ef174c560 2024-04-25T14:39:55.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.929+0000 7fce99fd8740 10 client.25024 fetch_fsmap learned FSMap version 21 2024-04-25T14:39:55.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.929+0000 7fce99fd8740 1 -- 172.21.15.53:0/749212030 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x558ef1761180 con 0x558ef174c560 2024-04-25T14:39:55.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.930+0000 7fce7e000640 1 -- 172.21.15.53:0/749212030 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce88015070 con 0x558ef174c560 2024-04-25T14:39:55.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.930+0000 7fce7e000640 1 -- 172.21.15.53:0/749212030 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fce78097e40 con 0x558ef174c560 2024-04-25T14:39:55.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.930+0000 7fce99fd8740 10 client.25024 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T14:39:55.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.930+0000 7fce99fd8740 10 client.25024 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T14:39:55.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.930+0000 7fce99fd8740 20 client.25024 populate_metadata read hostname 'smithi053' 2024-04-25T14:39:55.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.930+0000 7fce99fd8740 1 --2- 172.21.15.53:0/749212030 >> [v2:172.21.15.183:6838/1554258915,v1:172.21.15.183:6839/1554258915] conn(0x558ef1763740 0x558ef1764bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:55.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.930+0000 7fce99fd8740 4 client.25024 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:39:55.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.930+0000 7fce99fd8740 1 -- 172.21.15.53:0/749212030 --> [v2:172.21.15.183:6838/1554258915,v1:172.21.15.183:6839/1554258915] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ef16ba530 con 0x558ef1763740 2024-04-25T14:39:55.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.930+0000 7fce8f400640 1 --2- 172.21.15.53:0/749212030 >> [v2:172.21.15.183:6838/1554258915,v1:172.21.15.183:6839/1554258915] conn(0x558ef1763740 0x558ef1764bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:55.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.931+0000 7fce8f400640 1 --2- 172.21.15.53:0/749212030 >> [v2:172.21.15.183:6838/1554258915,v1:172.21.15.183:6839/1554258915] conn(0x558ef1763740 0x558ef1764bb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b2955046073fca9b server_cookie=cba9d871c8de06b4 in_seq=0 out_seq=0 2024-04-25T14:39:55.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.931+0000 7fce7e000640 10 client.25024 ms_handle_connect on v2:172.21.15.183:6838/1554258915 2024-04-25T14:39:55.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.932+0000 7fce7e000640 1 -- 172.21.15.53:0/749212030 <== mds.1 v2:172.21.15.183:6838/1554258915 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558ef16ba530 con 0x558ef1763740 2024-04-25T14:39:55.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.932+0000 7fce7e000640 10 client.25024 handle_command_reply: tid=0 2024-04-25T14:39:55.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.932+0000 7fce99fd8740 1 client.25024 shutdown 2024-04-25T14:39:55.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.932+0000 7fce99fd8740 1 -- 172.21.15.53:0/749212030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef174c560 msgr2=0x558ef1756820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:55.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.932+0000 7fce99fd8740 1 --2- 172.21.15.53:0/749212030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef174c560 0x558ef1756820 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x558ef174d9c0 tx=0x7fce7809c8b0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.932+0000 7fce99fd8740 1 -- 172.21.15.53:0/749212030 shutdown_connections 2024-04-25T14:39:55.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.932+0000 7fce99fd8740 1 --2- 172.21.15.53:0/749212030 >> [v2:172.21.15.183:6838/1554258915,v1:172.21.15.183:6839/1554258915] conn(0x558ef1763740 0x558ef1764bb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.932+0000 7fce99fd8740 1 --2- 172.21.15.53:0/749212030 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ef17520c0 0x558ef175dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.932+0000 7fce99fd8740 1 --2- 172.21.15.53:0/749212030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ef174cf50 0x558ef1756d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.932+0000 7fce7e000640 0 client.25024 ms_handle_reset on v2:172.21.15.183:6838/1554258915 2024-04-25T14:39:55.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.932+0000 7fce99fd8740 1 --2- 172.21.15.53:0/749212030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef174c560 0x558ef1756820 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.932+0000 7fce99fd8740 1 -- 172.21.15.53:0/749212030 >> 172.21.15.53:0/749212030 conn(0x558ef17437c0 msgr2=0x558ef1742a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:55.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.932+0000 7fce99fd8740 1 -- 172.21.15.53:0/749212030 shutdown_connections 2024-04-25T14:39:55.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.933+0000 7fce99fd8740 1 -- 172.21.15.53:0/749212030 wait complete. 2024-04-25T14:39:55.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.933+0000 7fce99fd8740 20 client.25024 trim_cache size 0 max 16384 2024-04-25T14:39:55.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.953+0000 7fce99fd8740 1 Processor -- start 2024-04-25T14:39:55.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.953+0000 7fce99fd8740 1 -- start start 2024-04-25T14:39:55.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.953+0000 7fce99fd8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef174c560 0x558ef17644e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:55.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.953+0000 7fce99fd8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ef174cf50 0x558ef175dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:55.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.953+0000 7fce99fd8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ef17520c0 0x558ef1756430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:55.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.953+0000 7fce99fd8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558ef1762ef0 con 0x558ef17520c0 2024-04-25T14:39:55.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.953+0000 7fce99fd8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558ef17635c0 con 0x558ef174cf50 2024-04-25T14:39:55.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.953+0000 7fce99fd8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558ef1763c90 con 0x558ef174c560 2024-04-25T14:39:55.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.953+0000 7fce8fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef174c560 0x558ef17644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:55.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.953+0000 7fce8fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef174c560 0x558ef17644e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:35432/0 (socket says 172.21.15.53:35432) 2024-04-25T14:39:55.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.953+0000 7fce8fe00640 1 -- 172.21.15.53:0/1641733379 learned_addr learned my addr 172.21.15.53:0/1641733379 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:55.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.953+0000 7fce95000640 1 --2- 172.21.15.53:0/1641733379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ef17520c0 0x558ef1756430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:55.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.953+0000 7fce8f400640 1 --2- 172.21.15.53:0/1641733379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ef174cf50 0x558ef175dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:55.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.954+0000 7fce8fe00640 1 -- 172.21.15.53:0/1641733379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ef174cf50 msgr2=0x558ef175dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:39:55.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.954+0000 7fce8fe00640 1 --2- 172.21.15.53:0/1641733379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ef174cf50 0x558ef175dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.954+0000 7fce8fe00640 1 -- 172.21.15.53:0/1641733379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ef17520c0 msgr2=0x558ef1756430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:39:55.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.954+0000 7fce8fe00640 1 --2- 172.21.15.53:0/1641733379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ef17520c0 0x558ef1756430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.954+0000 7fce8fe00640 1 -- 172.21.15.53:0/1641733379 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ef1744490 con 0x558ef174c560 2024-04-25T14:39:55.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.954+0000 7fce8f400640 1 --2- 172.21.15.53:0/1641733379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ef174cf50 0x558ef175dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:39:55.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.954+0000 7fce8fe00640 1 --2- 172.21.15.53:0/1641733379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef174c560 0x558ef17644e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fce780a4830 tx=0x7fce780a0620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8288767e90174d81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:55.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.954+0000 7fce84c00640 1 -- 172.21.15.53:0/1641733379 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce780a4d60 con 0x558ef174c560 2024-04-25T14:39:55.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.954+0000 7fce84c00640 1 -- 172.21.15.53:0/1641733379 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce78009d20 con 0x558ef174c560 2024-04-25T14:39:55.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.954+0000 7fce84c00640 1 -- 172.21.15.53:0/1641733379 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce780b0df0 con 0x558ef174c560 2024-04-25T14:39:55.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.955+0000 7fce99fd8740 1 -- 172.21.15.53:0/1641733379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef174c560 msgr2=0x558ef17644e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:55.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.955+0000 7fce99fd8740 1 --2- 172.21.15.53:0/1641733379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef174c560 0x558ef17644e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fce780a4830 tx=0x7fce780a0620 comp rx=0 tx=0).stop 2024-04-25T14:39:55.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.955+0000 7fce99fd8740 1 -- 172.21.15.53:0/1641733379 shutdown_connections 2024-04-25T14:39:55.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.955+0000 7fce99fd8740 1 --2- 172.21.15.53:0/1641733379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ef17520c0 0x558ef1756430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.955+0000 7fce99fd8740 1 --2- 172.21.15.53:0/1641733379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ef174cf50 0x558ef175dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.955+0000 7fce99fd8740 1 --2- 172.21.15.53:0/1641733379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef174c560 0x558ef17644e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.955+0000 7fce99fd8740 1 -- 172.21.15.53:0/1641733379 >> 172.21.15.53:0/1641733379 conn(0x558ef17437c0 msgr2=0x558ef1751810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:55.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.955+0000 7fce99fd8740 1 -- 172.21.15.53:0/1641733379 shutdown_connections 2024-04-25T14:39:55.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.955+0000 7fce99fd8740 1 -- 172.21.15.53:0/1641733379 wait complete. 2024-04-25T14:39:55.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.955+0000 7fce99fd8740 1 -- start start 2024-04-25T14:39:55.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.956+0000 7fce99fd8740 1 Processor -- start 2024-04-25T14:39:55.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.956+0000 7fce99fd8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ef174c560 0x558ef175bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:55.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.956+0000 7fce99fd8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef174cf50 0x558ef175c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:55.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.956+0000 7fce99fd8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ef17520c0 0x558ef175c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:55.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.956+0000 7fce99fd8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558ef1764a30 con 0x558ef17520c0 2024-04-25T14:39:55.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.956+0000 7fce99fd8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558ef1764360 con 0x558ef174c560 2024-04-25T14:39:55.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.956+0000 7fce99fd8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558ef1765100 con 0x558ef174cf50 2024-04-25T14:39:55.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.956+0000 7fce99fd8740 10 client.0 _ll_register_callbacks cb 0x7fce7809ba50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:39:55.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.956+0000 7fce8fe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ef174c560 0x558ef175bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:55.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.956+0000 7fce95000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ef17520c0 0x558ef175c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:55.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.956+0000 7fce8fe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ef174c560 0x558ef175bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.53:60820/0 (socket says 172.21.15.53:60820) 2024-04-25T14:39:55.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.956+0000 7fce8fe00640 1 -- 172.21.15.53:0/1785936127 learned_addr learned my addr 172.21.15.53:0/1785936127 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:55.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.956+0000 7fce8f400640 1 --2- 172.21.15.53:0/1785936127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef174cf50 0x558ef175c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:55.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.957+0000 7fce8fe00640 1 -- 172.21.15.53:0/1785936127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef174cf50 msgr2=0x558ef175c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:39:55.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.957+0000 7fce8fe00640 1 --2- 172.21.15.53:0/1785936127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef174cf50 0x558ef175c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.957+0000 7fce8fe00640 1 -- 172.21.15.53:0/1785936127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ef17520c0 msgr2=0x558ef175c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:39:55.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.957+0000 7fce8fe00640 1 --2- 172.21.15.53:0/1785936127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ef17520c0 0x558ef175c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.957+0000 7fce8fe00640 1 -- 172.21.15.53:0/1785936127 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ef1755920 con 0x558ef174c560 2024-04-25T14:39:55.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.957+0000 7fce8fe00640 1 --2- 172.21.15.53:0/1785936127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ef174c560 0x558ef175bf00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fce7809a3a0 tx=0x7fce78000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b2820fc427bd088 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:55.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.957+0000 7fce7e000640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-25T14:39:55.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.957+0000 7fce7e000640 1 -- 172.21.15.53:0/1785936127 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce780a4d60 con 0x558ef174c560 2024-04-25T14:39:55.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.957+0000 7fce7e000640 1 -- 172.21.15.53:0/1785936127 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fce780b0da0 con 0x558ef174c560 2024-04-25T14:39:55.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.957+0000 7fce99fd8740 1 -- 172.21.15.53:0/1785936127 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce78009d20 con 0x558ef174c560 2024-04-25T14:39:55.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.957+0000 7fce7e000640 1 -- 172.21.15.53:0/1785936127 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fce78085360 con 0x558ef174c560 2024-04-25T14:39:55.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.957+0000 7fce99fd8740 10 client.25013 fetch_fsmap learned FSMap version 21 2024-04-25T14:39:55.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.958+0000 7fce99fd8740 1 -- 172.21.15.53:0/1785936127 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x558ef17c96f0 con 0x558ef174c560 2024-04-25T14:39:55.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.958+0000 7fce7e000640 1 -- 172.21.15.53:0/1785936127 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce78009d20 con 0x558ef174c560 2024-04-25T14:39:55.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.958+0000 7fce7e000640 1 -- 172.21.15.53:0/1785936127 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fce780a1630 con 0x558ef174c560 2024-04-25T14:39:55.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.958+0000 7fce99fd8740 10 client.25013 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T14:39:55.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.958+0000 7fce99fd8740 10 client.25013 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T14:39:55.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.958+0000 7fce99fd8740 20 client.25013 populate_metadata read hostname 'smithi053' 2024-04-25T14:39:55.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.958+0000 7fce99fd8740 1 --2- 172.21.15.53:0/1785936127 >> [v2:172.21.15.183:6838/1554258915,v1:172.21.15.183:6839/1554258915] conn(0x558ef17cdd50 0x558ef17cca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:55.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.958+0000 7fce99fd8740 4 client.25013 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T14:39:55.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.958+0000 7fce99fd8740 1 -- 172.21.15.53:0/1785936127 --> [v2:172.21.15.183:6838/1554258915,v1:172.21.15.183:6839/1554258915] -- command(tid 0: {"prefix": "status"}) -- 0x558ef16b43a0 con 0x558ef17cdd50 2024-04-25T14:39:55.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.958+0000 7fce8f400640 1 --2- 172.21.15.53:0/1785936127 >> [v2:172.21.15.183:6838/1554258915,v1:172.21.15.183:6839/1554258915] conn(0x558ef17cdd50 0x558ef17cca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:55.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.959+0000 7fce8f400640 1 --2- 172.21.15.53:0/1785936127 >> [v2:172.21.15.183:6838/1554258915,v1:172.21.15.183:6839/1554258915] conn(0x558ef17cdd50 0x558ef17cca60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4bdbf79afd4b3837 server_cookie=c7013c9f28643f94 in_seq=0 out_seq=0 2024-04-25T14:39:55.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.959+0000 7fce7e000640 10 client.25013 ms_handle_connect on v2:172.21.15.183:6838/1554258915 2024-04-25T14:39:55.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.959+0000 7fce7e000640 1 -- 172.21.15.53:0/1785936127 <== mds.1 v2:172.21.15.183:6838/1554258915 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x558ef16b43a0 con 0x558ef17cdd50 2024-04-25T14:39:55.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.959+0000 7fce7e000640 10 client.25013 handle_command_reply: tid=0 2024-04-25T14:39:55.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.959+0000 7fce99fd8740 1 client.25013 shutdown 2024-04-25T14:39:55.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.959+0000 7fce99fd8740 1 -- 172.21.15.53:0/1785936127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ef174c560 msgr2=0x558ef175bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:55.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.959+0000 7fce99fd8740 1 --2- 172.21.15.53:0/1785936127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ef174c560 0x558ef175bf00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fce7809a3a0 tx=0x7fce78000bc0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.959+0000 7fce99fd8740 1 -- 172.21.15.53:0/1785936127 shutdown_connections 2024-04-25T14:39:55.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.959+0000 7fce99fd8740 1 --2- 172.21.15.53:0/1785936127 >> [v2:172.21.15.183:6838/1554258915,v1:172.21.15.183:6839/1554258915] conn(0x558ef17cdd50 0x558ef17cca60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.959+0000 7fce99fd8740 1 --2- 172.21.15.53:0/1785936127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ef17520c0 0x558ef175c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.959+0000 7fce99fd8740 1 --2- 172.21.15.53:0/1785936127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef174cf50 0x558ef175c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.959+0000 7fce7e000640 0 client.25013 ms_handle_reset on v2:172.21.15.183:6838/1554258915 2024-04-25T14:39:55.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.959+0000 7fce99fd8740 1 --2- 172.21.15.53:0/1785936127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ef174c560 0x558ef175bf00 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.959+0000 7fce99fd8740 1 -- 172.21.15.53:0/1785936127 >> 172.21.15.53:0/1785936127 conn(0x558ef17437c0 msgr2=0x558ef1756430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:55.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.960+0000 7fce99fd8740 1 -- 172.21.15.53:0/1785936127 shutdown_connections 2024-04-25T14:39:55.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.960+0000 7fce99fd8740 1 -- 172.21.15.53:0/1785936127 wait complete. 2024-04-25T14:39:55.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.960+0000 7fce99fd8740 20 client.25013 trim_cache size 0 max 16384 2024-04-25T14:39:55.970 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-04-25T14:39:55.970 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "fd30e6ce-030f-11ef-bc93-c7b262605968", 2024-04-25T14:39:55.970 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 1, 2024-04-25T14:39:55.970 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 24565, 2024-04-25T14:39:55.970 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2024-04-25T14:39:55.971 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2024-04-25T14:39:55.971 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2024-04-25T14:39:55.971 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 101.965320648, 2024-04-25T14:39:55.971 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 21, 2024-04-25T14:39:55.971 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 90, 2024-04-25T14:39:55.971 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 88, 2024-04-25T14:39:55.971 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 121.617468019 2024-04-25T14:39:55.971 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-04-25T14:39:55.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.960+0000 7fce95a00640 1 -- 172.21.15.53:0/1091734320 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fce740a3bb0 msgr2=0x7fce740a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:55.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.960+0000 7fce95a00640 1 --2- 172.21.15.53:0/1091734320 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fce740a3bb0 0x7fce740a6070 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x558ef1744690 tx=0x7fce80000f40 comp rx=0 tx=0).stop 2024-04-25T14:39:55.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.960+0000 7fce95a00640 1 -- 172.21.15.53:0/1091734320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce90102940 msgr2=0x7fce90198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:55.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.960+0000 7fce95a00640 1 --2- 172.21.15.53:0/1091734320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce90102940 0x7fce90198080 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fce7800d950 tx=0x7fce7800de10 comp rx=0 tx=0).stop 2024-04-25T14:39:55.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.960+0000 7fce95a00640 1 -- 172.21.15.53:0/1091734320 shutdown_connections 2024-04-25T14:39:55.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.960+0000 7fce95a00640 1 --2- 172.21.15.53:0/1091734320 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fce740a3bb0 0x7fce740a6070 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.960+0000 7fce95a00640 1 --2- 172.21.15.53:0/1091734320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce90107ba0 0x7fce9019f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.960+0000 7fce95a00640 1 --2- 172.21.15.53:0/1091734320 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce90105270 0x7fce901985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.960+0000 7fce95a00640 1 --2- 172.21.15.53:0/1091734320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce90102940 0x7fce90198080 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:55.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.960+0000 7fce95a00640 1 -- 172.21.15.53:0/1091734320 >> 172.21.15.53:0/1091734320 conn(0x7fce900fc2f0 msgr2=0x7fce901031e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:55.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.961+0000 7fce95a00640 1 -- 172.21.15.53:0/1091734320 shutdown_connections 2024-04-25T14:39:55.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:55.961+0000 7fce95a00640 1 -- 172.21.15.53:0/1091734320 wait complete. 2024-04-25T14:39:55.980 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T14:39:56.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.196+0000 7fa3e7400640 1 -- 172.21.15.53:0/4044533172 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3e0101b60 msgr2=0x7fa3e0101fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:56.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.196+0000 7fa3e7400640 1 --2- 172.21.15.53:0/4044533172 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3e0101b60 0x7fa3e0101fe0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8009ca0 tx=0x7fa3d802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:39:56.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.196+0000 7fa3e7400640 1 -- 172.21.15.53:0/4044533172 shutdown_connections 2024-04-25T14:39:56.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.196+0000 7fa3e7400640 1 --2- 172.21.15.53:0/4044533172 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3e0101b60 0x7fa3e0101fe0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:56.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.196+0000 7fa3e7400640 1 --2- 172.21.15.53:0/4044533172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3e01011c0 0x7fa3e0101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:56.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.196+0000 7fa3e7400640 1 --2- 172.21.15.53:0/4044533172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3e0100860 0x7fa3e0100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:56.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.196+0000 7fa3e7400640 1 -- 172.21.15.53:0/4044533172 >> 172.21.15.53:0/4044533172 conn(0x7fa3e00fc400 msgr2=0x7fa3e00fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:56.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.196+0000 7fa3e7400640 1 -- 172.21.15.53:0/4044533172 shutdown_connections 2024-04-25T14:39:56.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.196+0000 7fa3e7400640 1 -- 172.21.15.53:0/4044533172 wait complete. 2024-04-25T14:39:56.198 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.197+0000 7fa3e7400640 1 Processor -- start 2024-04-25T14:39:56.198 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.198+0000 7fa3e7400640 1 -- start start 2024-04-25T14:39:56.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.198+0000 7fa3e7400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3e0100860 0x7fa3e019a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:56.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.198+0000 7fa3e7400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3e01011c0 0x7fa3e019a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:56.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.198+0000 7fa3e7400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3e0101b60 0x7fa3e01a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:56.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.198+0000 7fa3e7400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e010fd30 con 0x7fa3e0100860 2024-04-25T14:39:56.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.198+0000 7fa3e7400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e010fbb0 con 0x7fa3e0101b60 2024-04-25T14:39:56.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.198+0000 7fa3e7400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e010feb0 con 0x7fa3e01011c0 2024-04-25T14:39:56.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.198+0000 7fa3e4a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3e0100860 0x7fa3e019a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:56.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.198+0000 7fa3e4a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3e0100860 0x7fa3e019a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59766/0 (socket says 172.21.15.53:59766) 2024-04-25T14:39:56.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.198+0000 7fa3e4a00640 1 -- 172.21.15.53:0/479919450 learned_addr learned my addr 172.21.15.53:0/479919450 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:56.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.198+0000 7fa3e5400640 1 --2- 172.21.15.53:0/479919450 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3e0101b60 0x7fa3e01a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:56.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.198+0000 7fa3dfe00640 1 --2- 172.21.15.53:0/479919450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3e01011c0 0x7fa3e019a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:56.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.199+0000 7fa3e4a00640 1 -- 172.21.15.53:0/479919450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3e01011c0 msgr2=0x7fa3e019a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:56.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.199+0000 7fa3e4a00640 1 --2- 172.21.15.53:0/479919450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3e01011c0 0x7fa3e019a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:56.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.199+0000 7fa3e4a00640 1 -- 172.21.15.53:0/479919450 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3e0101b60 msgr2=0x7fa3e01a1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:56.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.199+0000 7fa3e4a00640 1 --2- 172.21.15.53:0/479919450 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3e0101b60 0x7fa3e01a1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:56.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.199+0000 7fa3e4a00640 1 -- 172.21.15.53:0/479919450 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e01a1fc0 con 0x7fa3e0100860 2024-04-25T14:39:56.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.199+0000 7fa3e4a00640 1 --2- 172.21.15.53:0/479919450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3e0100860 0x7fa3e019a3a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa3d400ea60 tx=0x7fa3d400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:56.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.199+0000 7fa3dd600640 1 -- 172.21.15.53:0/479919450 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d400cde0 con 0x7fa3e0100860 2024-04-25T14:39:56.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.199+0000 7fa3dd600640 1 -- 172.21.15.53:0/479919450 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3d40035f0 con 0x7fa3e0100860 2024-04-25T14:39:56.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.199+0000 7fa3e7400640 1 -- 172.21.15.53:0/479919450 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3e01a22b0 con 0x7fa3e0100860 2024-04-25T14:39:56.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.199+0000 7fa3dd600640 1 -- 172.21.15.53:0/479919450 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d4010690 con 0x7fa3e0100860 2024-04-25T14:39:56.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.200+0000 7fa3e7400640 1 -- 172.21.15.53:0/479919450 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3e01a2800 con 0x7fa3e0100860 2024-04-25T14:39:56.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.203+0000 7fa3ea3b7740 1 Processor -- start 2024-04-25T14:39:56.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.203+0000 7fa3ea3b7740 1 -- start start 2024-04-25T14:39:56.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.203+0000 7fa3ea3b7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556f7119bf60 0x556f7119c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:56.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.203+0000 7fa3dd600640 1 -- 172.21.15.53:0/479919450 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa3d40037a0 con 0x7fa3e0100860 2024-04-25T14:39:56.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.203+0000 7fa3ea3b7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f7119c950 0x556f711a1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:56.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.203+0000 7fa3ea3b7740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556f711a1ac0 0x556f711a3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:56.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.203+0000 7fa3ea3b7740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x556f71100b10 con 0x556f711a1ac0 2024-04-25T14:39:56.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.203+0000 7fa3ea3b7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556f710a72b0 con 0x556f7119bf60 2024-04-25T14:39:56.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.203+0000 7fa3ea3b7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556f710fcad0 con 0x556f7119c950 2024-04-25T14:39:56.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.204+0000 7fa3e5400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556f7119bf60 0x556f7119c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:56.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.204+0000 7fa3e5400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556f7119bf60 0x556f7119c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.53:60844/0 (socket says 172.21.15.53:60844) 2024-04-25T14:39:56.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.204+0000 7fa3e5400640 1 -- 172.21.15.53:0/2650920589 learned_addr learned my addr 172.21.15.53:0/2650920589 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:56.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.204+0000 7fa3e4a00640 1 --2- 172.21.15.53:0/2650920589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f7119c950 0x556f711a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:56.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.204+0000 7fa3dfe00640 1 --2- 172.21.15.53:0/2650920589 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556f711a1ac0 0x556f711a3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:56.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.204+0000 7fa3e5400640 1 -- 172.21.15.53:0/2650920589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f7119c950 msgr2=0x556f711a1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:39:56.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.204+0000 7fa3e5400640 1 --2- 172.21.15.53:0/2650920589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f7119c950 0x556f711a1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:56.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.204+0000 7fa3e5400640 1 -- 172.21.15.53:0/2650920589 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556f711a1ac0 msgr2=0x556f711a3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:39:56.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.204+0000 7fa3e5400640 1 --2- 172.21.15.53:0/2650920589 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556f711a1ac0 0x556f711a3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:56.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.204+0000 7fa3e5400640 1 -- 172.21.15.53:0/2650920589 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f70e132d0 con 0x556f7119bf60 2024-04-25T14:39:56.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.204+0000 7fa3e5400640 1 --2- 172.21.15.53:0/2650920589 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556f7119bf60 0x556f7119c380 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8009ca0 tx=0x7fa3d80093c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de7391e57116e3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:56.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.205+0000 7fa3d1c00640 1 -- 172.21.15.53:0/2650920589 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d8031650 con 0x556f7119bf60 2024-04-25T14:39:56.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.205+0000 7fa3d1c00640 1 -- 172.21.15.53:0/2650920589 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3d8036030 con 0x556f7119bf60 2024-04-25T14:39:56.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.205+0000 7fa3d1c00640 1 -- 172.21.15.53:0/2650920589 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d8004660 con 0x556f7119bf60 2024-04-25T14:39:56.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.206+0000 7fa3ea3b7740 1 -- 172.21.15.53:0/2650920589 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556f7119bf60 msgr2=0x556f7119c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:56.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.206+0000 7fa3ea3b7740 1 --2- 172.21.15.53:0/2650920589 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556f7119bf60 0x556f7119c380 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8009ca0 tx=0x7fa3d80093c0 comp rx=0 tx=0).stop 2024-04-25T14:39:56.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.206+0000 7fa3ea3b7740 1 -- 172.21.15.53:0/2650920589 shutdown_connections 2024-04-25T14:39:56.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.206+0000 7fa3ea3b7740 1 --2- 172.21.15.53:0/2650920589 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556f711a1ac0 0x556f711a3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:56.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.206+0000 7fa3ea3b7740 1 --2- 172.21.15.53:0/2650920589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f7119c950 0x556f711a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:56.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.206+0000 7fa3ea3b7740 1 --2- 172.21.15.53:0/2650920589 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556f7119bf60 0x556f7119c380 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:56.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.206+0000 7fa3ea3b7740 1 -- 172.21.15.53:0/2650920589 >> 172.21.15.53:0/2650920589 conn(0x556f711975d0 msgr2=0x556f711979d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:56.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.206+0000 7fa3ea3b7740 1 -- 172.21.15.53:0/2650920589 shutdown_connections 2024-04-25T14:39:56.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.206+0000 7fa3ea3b7740 1 -- 172.21.15.53:0/2650920589 wait complete. 2024-04-25T14:39:56.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.206+0000 7fa3dd600640 1 --2- 172.21.15.53:0/479919450 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fa3c80a3ac0 0x7fa3c80a5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:56.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.206+0000 7fa3dd600640 1 -- 172.21.15.53:0/479919450 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fa3d40d06e0 con 0x7fa3e0100860 2024-04-25T14:39:56.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.206+0000 7fa3dfe00640 1 --2- 172.21.15.53:0/479919450 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fa3c80a3ac0 0x7fa3c80a5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:56.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.206+0000 7fa3ea3b7740 1 -- start start 2024-04-25T14:39:56.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.207+0000 7fa3ea3b7740 1 Processor -- start 2024-04-25T14:39:56.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.207+0000 7fa3dfe00640 1 --2- 172.21.15.53:0/479919450 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fa3c80a3ac0 0x7fa3c80a5f80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x556f711984a0 tx=0x7fa3cc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:56.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.207+0000 7fa3ea3b7740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556f7119bf60 0x556f711a5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:56.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.207+0000 7fa3ea3b7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556f7119c950 0x556f711a6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:56.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.207+0000 7fa3ea3b7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f711a1ac0 0x556f711ad550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:56.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.207+0000 7fa3ea3b7740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x556f711a4890 con 0x556f7119bf60 2024-04-25T14:39:56.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.207+0000 7fa3ea3b7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556f710ba570 con 0x556f7119c950 2024-04-25T14:39:56.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.207+0000 7fa3ea3b7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556f711a4a10 con 0x556f711a1ac0 2024-04-25T14:39:56.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.207+0000 7fa3ea3b7740 10 client.0 _ll_register_callbacks cb 0x556f71088fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:39:56.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.207+0000 7fa3e4a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556f7119bf60 0x556f711a5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:56.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.207+0000 7fa3e4a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556f7119bf60 0x556f711a5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59794/0 (socket says 172.21.15.53:59794) 2024-04-25T14:39:56.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.207+0000 7fa3e4a00640 1 -- 172.21.15.53:0/400114323 learned_addr learned my addr 172.21.15.53:0/400114323 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:56.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.207+0000 7fa3dfe00640 1 --2- 172.21.15.53:0/400114323 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556f7119c950 0x556f711a6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:56.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.208+0000 7fa3e5400640 1 --2- 172.21.15.53:0/400114323 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f711a1ac0 0x556f711ad550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:56.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.208+0000 7fa3e4a00640 1 -- 172.21.15.53:0/400114323 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f711a1ac0 msgr2=0x556f711ad550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:39:56.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.208+0000 7fa3e4a00640 1 --2- 172.21.15.53:0/400114323 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f711a1ac0 0x556f711ad550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:56.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.208+0000 7fa3e4a00640 1 -- 172.21.15.53:0/400114323 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556f7119c950 msgr2=0x556f711a6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:39:56.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.208+0000 7fa3e4a00640 1 --2- 172.21.15.53:0/400114323 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556f7119c950 0x556f711a6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:56.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.208+0000 7fa3e4a00640 1 -- 172.21.15.53:0/400114323 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f70edd6b0 con 0x556f7119bf60 2024-04-25T14:39:56.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.208+0000 7fa3e5400640 1 --2- 172.21.15.53:0/400114323 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f711a1ac0 0x556f711ad550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:39:56.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.208+0000 7fa3e4a00640 1 --2- 172.21.15.53:0/400114323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556f7119bf60 0x556f711a5fd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4099320 tx=0x7fa3d40045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d60056f8089f01a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:56.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.208+0000 7fa3c6000640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-04-25T14:39:56.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.208+0000 7fa3c6000640 1 -- 172.21.15.53:0/400114323 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d409c720 con 0x556f7119bf60 2024-04-25T14:39:56.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.209+0000 7fa3c6000640 1 -- 172.21.15.53:0/400114323 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fa3d4098fc0 con 0x556f7119bf60 2024-04-25T14:39:56.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.209+0000 7fa3ea3b7740 1 -- 172.21.15.53:0/400114323 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3d8036030 con 0x556f7119bf60 2024-04-25T14:39:56.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.209+0000 7fa3c6000640 1 -- 172.21.15.53:0/400114323 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa3d409c2b0 con 0x556f7119bf60 2024-04-25T14:39:56.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.209+0000 7fa3ea3b7740 10 client.14910 fetch_fsmap learned FSMap version 21 2024-04-25T14:39:56.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.209+0000 7fa3ea3b7740 1 -- 172.21.15.53:0/400114323 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556f711b0930 con 0x556f7119bf60 2024-04-25T14:39:56.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.209+0000 7fa3c6000640 1 -- 172.21.15.53:0/400114323 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3d8036030 con 0x556f7119bf60 2024-04-25T14:39:56.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.209+0000 7fa3c6000640 1 -- 172.21.15.53:0/400114323 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa3d40870b0 con 0x556f7119bf60 2024-04-25T14:39:56.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.209+0000 7fa3ea3b7740 10 client.14910 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T14:39:56.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.209+0000 7fa3ea3b7740 10 client.14910 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T14:39:56.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.209+0000 7fa3ea3b7740 20 client.14910 populate_metadata read hostname 'smithi053' 2024-04-25T14:39:56.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.209+0000 7fa3ea3b7740 1 --2- 172.21.15.53:0/400114323 >> [v2:172.21.15.150:6838/149573616,v1:172.21.15.150:6839/149573616] conn(0x556f711b2f80 0x556f711b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:56.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.209+0000 7fa3ea3b7740 4 client.14910 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:39:56.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.209+0000 7fa3ea3b7740 1 -- 172.21.15.53:0/400114323 --> [v2:172.21.15.150:6838/149573616,v1:172.21.15.150:6839/149573616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f7110e530 con 0x556f711b2f80 2024-04-25T14:39:56.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.210+0000 7fa3dfe00640 1 --2- 172.21.15.53:0/400114323 >> [v2:172.21.15.150:6838/149573616,v1:172.21.15.150:6839/149573616] conn(0x556f711b2f80 0x556f711b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:56.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.210+0000 7fa3dfe00640 1 --2- 172.21.15.53:0/400114323 >> [v2:172.21.15.150:6838/149573616,v1:172.21.15.150:6839/149573616] conn(0x556f711b2f80 0x556f711b43f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c471a11d6e4e0505 server_cookie=a3c96f8a6c509115 in_seq=0 out_seq=0 2024-04-25T14:39:56.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.210+0000 7fa3c6000640 10 client.14910 ms_handle_connect on v2:172.21.15.150:6838/149573616 2024-04-25T14:39:56.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.211+0000 7fa3c6000640 1 -- 172.21.15.53:0/400114323 <== mds.0 v2:172.21.15.150:6838/149573616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556f7110e530 con 0x556f711b2f80 2024-04-25T14:39:56.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.211+0000 7fa3c6000640 10 client.14910 handle_command_reply: tid=0 2024-04-25T14:39:56.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.211+0000 7fa3ea3b7740 1 client.14910 shutdown 2024-04-25T14:39:56.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.212+0000 7fa3ea3b7740 1 -- 172.21.15.53:0/400114323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556f7119bf60 msgr2=0x556f711a5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:56.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.212+0000 7fa3ea3b7740 1 --2- 172.21.15.53:0/400114323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556f7119bf60 0x556f711a5fd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4099320 tx=0x7fa3d40045d0 comp rx=0 tx=0).stop 2024-04-25T14:39:56.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.212+0000 7fa3ea3b7740 1 -- 172.21.15.53:0/400114323 shutdown_connections 2024-04-25T14:39:56.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.212+0000 7fa3ea3b7740 1 --2- 172.21.15.53:0/400114323 >> [v2:172.21.15.150:6838/149573616,v1:172.21.15.150:6839/149573616] conn(0x556f711b2f80 0x556f711b43f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:56.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.212+0000 7fa3ea3b7740 1 --2- 172.21.15.53:0/400114323 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f711a1ac0 0x556f711ad550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:56.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.212+0000 7fa3ea3b7740 1 --2- 172.21.15.53:0/400114323 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556f7119c950 0x556f711a6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:56.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.212+0000 7fa3c6000640 0 client.14910 ms_handle_reset on v2:172.21.15.150:6838/149573616 2024-04-25T14:39:56.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.212+0000 7fa3ea3b7740 1 --2- 172.21.15.53:0/400114323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556f7119bf60 0x556f711a5fd0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:56.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.212+0000 7fa3ea3b7740 1 -- 172.21.15.53:0/400114323 >> 172.21.15.53:0/400114323 conn(0x556f711975d0 msgr2=0x556f71196730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:56.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.212+0000 7fa3ea3b7740 1 -- 172.21.15.53:0/400114323 shutdown_connections 2024-04-25T14:39:56.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.212+0000 7fa3ea3b7740 1 -- 172.21.15.53:0/400114323 wait complete. 2024-04-25T14:39:56.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.212+0000 7fa3ea3b7740 20 client.14910 trim_cache size 0 max 16384 2024-04-25T14:39:56.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.232+0000 7fa3ea3b7740 1 Processor -- start 2024-04-25T14:39:56.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.232+0000 7fa3ea3b7740 1 -- start start 2024-04-25T14:39:56.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.233+0000 7fa3ea3b7740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556f7119bf60 0x556f71197070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:56.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.233+0000 7fa3ea3b7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556f7119c950 0x556f711ad510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:56.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.233+0000 7fa3ea3b7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f711a1ac0 0x556f7119dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:56.234 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.233+0000 7fa3ea3b7740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x556f711b2730 con 0x556f7119bf60 2024-04-25T14:39:56.234 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.233+0000 7fa3ea3b7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556f711b2e00 con 0x556f7119c950 2024-04-25T14:39:56.234 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.233+0000 7fa3ea3b7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556f711b34d0 con 0x556f711a1ac0 2024-04-25T14:39:56.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.233+0000 7fa3e4a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556f7119bf60 0x556f71197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:56.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.233+0000 7fa3e4a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556f7119bf60 0x556f71197070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59810/0 (socket says 172.21.15.53:59810) 2024-04-25T14:39:56.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.233+0000 7fa3e4a00640 1 -- 172.21.15.53:0/2410321486 learned_addr learned my addr 172.21.15.53:0/2410321486 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:56.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.233+0000 7fa3e5400640 1 --2- 172.21.15.53:0/2410321486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f711a1ac0 0x556f7119dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:56.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.233+0000 7fa3dfe00640 1 --2- 172.21.15.53:0/2410321486 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556f7119c950 0x556f711ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:56.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.233+0000 7fa3e4a00640 1 -- 172.21.15.53:0/2410321486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f711a1ac0 msgr2=0x556f7119dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:39:56.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.233+0000 7fa3e4a00640 1 --2- 172.21.15.53:0/2410321486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f711a1ac0 0x556f7119dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:56.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.233+0000 7fa3e4a00640 1 -- 172.21.15.53:0/2410321486 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556f7119c950 msgr2=0x556f711ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:39:56.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.234+0000 7fa3e4a00640 1 --2- 172.21.15.53:0/2410321486 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556f7119c950 0x556f711ad510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:56.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.234+0000 7fa3e4a00640 1 -- 172.21.15.53:0/2410321486 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f7119cd30 con 0x556f7119bf60 2024-04-25T14:39:56.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.234+0000 7fa3e5400640 1 --2- 172.21.15.53:0/2410321486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f711a1ac0 0x556f7119dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:39:56.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.234+0000 7fa3e4a00640 1 --2- 172.21.15.53:0/2410321486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556f7119bf60 0x556f71197070 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa3d400c760 tx=0x7fa3d409ae60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b33580c22e636ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:56.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.234+0000 7fa3d1200640 1 -- 172.21.15.53:0/2410321486 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d409c490 con 0x556f7119bf60 2024-04-25T14:39:56.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.234+0000 7fa3d1200640 1 -- 172.21.15.53:0/2410321486 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3d409d050 con 0x556f7119bf60 2024-04-25T14:39:56.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.234+0000 7fa3d1200640 1 -- 172.21.15.53:0/2410321486 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d4098fc0 con 0x556f7119bf60 2024-04-25T14:39:56.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.235+0000 7fa3ea3b7740 1 -- 172.21.15.53:0/2410321486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556f7119bf60 msgr2=0x556f71197070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:56.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.235+0000 7fa3ea3b7740 1 --2- 172.21.15.53:0/2410321486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556f7119bf60 0x556f71197070 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa3d400c760 tx=0x7fa3d409ae60 comp rx=0 tx=0).stop 2024-04-25T14:39:56.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.235+0000 7fa3ea3b7740 1 -- 172.21.15.53:0/2410321486 shutdown_connections 2024-04-25T14:39:56.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.235+0000 7fa3ea3b7740 1 --2- 172.21.15.53:0/2410321486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f711a1ac0 0x556f7119dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:56.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.235+0000 7fa3ea3b7740 1 --2- 172.21.15.53:0/2410321486 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556f7119c950 0x556f711ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:56.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.235+0000 7fa3ea3b7740 1 --2- 172.21.15.53:0/2410321486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556f7119bf60 0x556f71197070 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:56.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.235+0000 7fa3ea3b7740 1 -- 172.21.15.53:0/2410321486 >> 172.21.15.53:0/2410321486 conn(0x556f711975d0 msgr2=0x556f711a11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:56.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.235+0000 7fa3ea3b7740 1 -- 172.21.15.53:0/2410321486 shutdown_connections 2024-04-25T14:39:56.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.235+0000 7fa3ea3b7740 1 -- 172.21.15.53:0/2410321486 wait complete. 2024-04-25T14:39:56.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.235+0000 7fa3ea3b7740 1 -- start start 2024-04-25T14:39:56.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.236+0000 7fa3ea3b7740 1 Processor -- start 2024-04-25T14:39:56.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.236+0000 7fa3ea3b7740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556f7119bf60 0x556f711a9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:56.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.236+0000 7fa3ea3b7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f7119c950 0x556f711a9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:56.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.236+0000 7fa3ea3b7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556f711a1ac0 0x556f711ac5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:56.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.236+0000 7fa3ea3b7740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x556f711b4270 con 0x556f7119bf60 2024-04-25T14:39:56.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.236+0000 7fa3ea3b7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556f711b3ba0 con 0x556f711a1ac0 2024-04-25T14:39:56.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.236+0000 7fa3ea3b7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556f711b4940 con 0x556f7119c950 2024-04-25T14:39:56.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.236+0000 7fa3ea3b7740 10 client.0 _ll_register_callbacks cb 0x7fa3d4003d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:39:56.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.236+0000 7fa3e4a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556f7119bf60 0x556f711a9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:56.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.236+0000 7fa3e4a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556f7119bf60 0x556f711a9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59818/0 (socket says 172.21.15.53:59818) 2024-04-25T14:39:56.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.236+0000 7fa3e4a00640 1 -- 172.21.15.53:0/1623122485 learned_addr learned my addr 172.21.15.53:0/1623122485 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:56.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.236+0000 7fa3dfe00640 1 --2- 172.21.15.53:0/1623122485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f7119c950 0x556f711a9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:56.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.236+0000 7fa3e5400640 1 --2- 172.21.15.53:0/1623122485 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556f711a1ac0 0x556f711ac5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:56.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.237+0000 7fa3e4a00640 1 -- 172.21.15.53:0/1623122485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f7119c950 msgr2=0x556f711a9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:39:56.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.237+0000 7fa3e4a00640 1 --2- 172.21.15.53:0/1623122485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f7119c950 0x556f711a9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:56.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.237+0000 7fa3e4a00640 1 -- 172.21.15.53:0/1623122485 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556f711a1ac0 msgr2=0x556f711ac5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:39:56.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.237+0000 7fa3e4a00640 1 --2- 172.21.15.53:0/1623122485 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556f711a1ac0 0x556f711ac5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:56.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.237+0000 7fa3e4a00640 1 -- 172.21.15.53:0/1623122485 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f711acb60 con 0x556f7119bf60 2024-04-25T14:39:56.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.237+0000 7fa3e4a00640 1 --2- 172.21.15.53:0/1623122485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556f7119bf60 0x556f711a9740 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4087510 tx=0x7fa3d40d6190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb5a8c4b7df9fdb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:56.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.237+0000 7fa3c6000640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-04-25T14:39:56.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.237+0000 7fa3c6000640 1 -- 172.21.15.53:0/1623122485 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d409c490 con 0x556f7119bf60 2024-04-25T14:39:56.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.237+0000 7fa3c6000640 1 -- 172.21.15.53:0/1623122485 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fa3d409bda0 con 0x556f7119bf60 2024-04-25T14:39:56.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.237+0000 7fa3ea3b7740 1 -- 172.21.15.53:0/1623122485 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3d409d050 con 0x556f7119bf60 2024-04-25T14:39:56.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.237+0000 7fa3c6000640 1 -- 172.21.15.53:0/1623122485 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa3d409c950 con 0x556f7119bf60 2024-04-25T14:39:56.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.237+0000 7fa3ea3b7740 10 client.14916 fetch_fsmap learned FSMap version 21 2024-04-25T14:39:56.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.237+0000 7fa3ea3b7740 1 -- 172.21.15.53:0/1623122485 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556f71219250 con 0x556f7119bf60 2024-04-25T14:39:56.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.238+0000 7fa3c6000640 1 -- 172.21.15.53:0/1623122485 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3d409d050 con 0x556f7119bf60 2024-04-25T14:39:56.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.238+0000 7fa3c6000640 1 -- 172.21.15.53:0/1623122485 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa3d409b810 con 0x556f7119bf60 2024-04-25T14:39:56.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.238+0000 7fa3ea3b7740 10 client.14916 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T14:39:56.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.238+0000 7fa3ea3b7740 10 client.14916 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T14:39:56.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.238+0000 7fa3ea3b7740 20 client.14916 populate_metadata read hostname 'smithi053' 2024-04-25T14:39:56.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.238+0000 7fa3ea3b7740 1 --2- 172.21.15.53:0/1623122485 >> [v2:172.21.15.150:6838/149573616,v1:172.21.15.150:6839/149573616] conn(0x556f7121d9b0 0x556f7121c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:56.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.238+0000 7fa3ea3b7740 4 client.14916 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T14:39:56.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.238+0000 7fa3ea3b7740 1 -- 172.21.15.53:0/1623122485 --> [v2:172.21.15.150:6838/149573616,v1:172.21.15.150:6839/149573616] -- command(tid 0: {"prefix": "status"}) -- 0x556f711083a0 con 0x556f7121d9b0 2024-04-25T14:39:56.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.238+0000 7fa3dfe00640 1 --2- 172.21.15.53:0/1623122485 >> [v2:172.21.15.150:6838/149573616,v1:172.21.15.150:6839/149573616] conn(0x556f7121d9b0 0x556f7121c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:56.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.239+0000 7fa3dfe00640 1 --2- 172.21.15.53:0/1623122485 >> [v2:172.21.15.150:6838/149573616,v1:172.21.15.150:6839/149573616] conn(0x556f7121d9b0 0x556f7121c6c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d683920555c4729e server_cookie=26f7123231bb83b8 in_seq=0 out_seq=0 2024-04-25T14:39:56.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.239+0000 7fa3c6000640 10 client.14916 ms_handle_connect on v2:172.21.15.150:6838/149573616 2024-04-25T14:39:56.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.239+0000 7fa3c6000640 1 -- 172.21.15.53:0/1623122485 <== mds.0 v2:172.21.15.150:6838/149573616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x556f711083a0 con 0x556f7121d9b0 2024-04-25T14:39:56.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.239+0000 7fa3c6000640 10 client.14916 handle_command_reply: tid=0 2024-04-25T14:39:56.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.239+0000 7fa3ea3b7740 1 client.14916 shutdown 2024-04-25T14:39:56.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.239+0000 7fa3ea3b7740 1 -- 172.21.15.53:0/1623122485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556f7119bf60 msgr2=0x556f711a9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:56.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.239+0000 7fa3ea3b7740 1 --2- 172.21.15.53:0/1623122485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556f7119bf60 0x556f711a9740 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4087510 tx=0x7fa3d40d6190 comp rx=0 tx=0).stop 2024-04-25T14:39:56.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.239+0000 7fa3ea3b7740 1 -- 172.21.15.53:0/1623122485 shutdown_connections 2024-04-25T14:39:56.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.239+0000 7fa3ea3b7740 1 --2- 172.21.15.53:0/1623122485 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556f711a1ac0 0x556f711ac5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:56.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.239+0000 7fa3ea3b7740 1 --2- 172.21.15.53:0/1623122485 >> [v2:172.21.15.150:6838/149573616,v1:172.21.15.150:6839/149573616] conn(0x556f7121d9b0 0x556f7121c6c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:56.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.239+0000 7fa3ea3b7740 1 --2- 172.21.15.53:0/1623122485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f7119c950 0x556f711a9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:56.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.239+0000 7fa3ea3b7740 1 --2- 172.21.15.53:0/1623122485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556f7119bf60 0x556f711a9740 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:56.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.239+0000 7fa3c6000640 0 client.14916 ms_handle_reset on v2:172.21.15.150:6838/149573616 2024-04-25T14:39:56.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.239+0000 7fa3ea3b7740 1 -- 172.21.15.53:0/1623122485 >> 172.21.15.53:0/1623122485 conn(0x556f711975d0 msgr2=0x556f7119dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:56.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.240+0000 7fa3ea3b7740 1 -- 172.21.15.53:0/1623122485 shutdown_connections 2024-04-25T14:39:56.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.240+0000 7fa3ea3b7740 1 -- 172.21.15.53:0/1623122485 wait complete. 2024-04-25T14:39:56.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.240+0000 7fa3ea3b7740 20 client.14916 trim_cache size 0 max 16384 2024-04-25T14:39:56.245 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-04-25T14:39:56.245 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "fd30e6ce-030f-11ef-bc93-c7b262605968", 2024-04-25T14:39:56.245 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2024-04-25T14:39:56.245 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 24584, 2024-04-25T14:39:56.245 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2024-04-25T14:39:56.245 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2024-04-25T14:39:56.245 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2024-04-25T14:39:56.245 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 105.706295146, 2024-04-25T14:39:56.245 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 21, 2024-04-25T14:39:56.245 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 90, 2024-04-25T14:39:56.245 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 88, 2024-04-25T14:39:56.245 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 111.02398805599999 2024-04-25T14:39:56.245 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-04-25T14:39:56.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.240+0000 7fa3e7400640 1 -- 172.21.15.53:0/479919450 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fa3c80a3ac0 msgr2=0x7fa3c80a5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:56.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.240+0000 7fa3e7400640 1 --2- 172.21.15.53:0/479919450 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fa3c80a3ac0 0x7fa3c80a5f80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x556f711984a0 tx=0x7fa3cc005f90 comp rx=0 tx=0).stop 2024-04-25T14:39:56.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.240+0000 7fa3e7400640 1 -- 172.21.15.53:0/479919450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3e0100860 msgr2=0x7fa3e019a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:56.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.240+0000 7fa3e7400640 1 --2- 172.21.15.53:0/479919450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3e0100860 0x7fa3e019a3a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa3d400ea60 tx=0x7fa3d400ef20 comp rx=0 tx=0).stop 2024-04-25T14:39:56.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.240+0000 7fa3e7400640 1 -- 172.21.15.53:0/479919450 shutdown_connections 2024-04-25T14:39:56.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.240+0000 7fa3e7400640 1 --2- 172.21.15.53:0/479919450 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fa3c80a3ac0 0x7fa3c80a5f80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:56.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.240+0000 7fa3e7400640 1 --2- 172.21.15.53:0/479919450 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3e0101b60 0x7fa3e01a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:56.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.240+0000 7fa3e7400640 1 --2- 172.21.15.53:0/479919450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3e01011c0 0x7fa3e019a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:56.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.240+0000 7fa3e7400640 1 --2- 172.21.15.53:0/479919450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3e0100860 0x7fa3e019a3a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:56.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.241+0000 7fa3e7400640 1 -- 172.21.15.53:0/479919450 >> 172.21.15.53:0/479919450 conn(0x7fa3e00fc400 msgr2=0x7fa3e00fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:56.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.241+0000 7fa3e7400640 1 -- 172.21.15.53:0/479919450 shutdown_connections 2024-04-25T14:39:56.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:56.241+0000 7fa3e7400640 1 -- 172.21.15.53:0/479919450 wait complete. 2024-04-25T14:39:56.257 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:56 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1199173754' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:39:56.261 INFO:tasks.cephadm:Setup complete, yielding 2024-04-25T14:39:56.261 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T14:39:56.272 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi053.front.sepia.ceph.com 2024-04-25T14:39:56.272 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-25T14:39:56.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:56 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/1199173754' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:39:56.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:56 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/1199173754' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:39:56.736 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:39:57.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:57 smithi183 ceph-mon[32584]: pgmap v371: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T14:39:57.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:57 smithi150 ceph-mon[32504]: pgmap v371: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T14:39:57.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:57 smithi053 ceph-mon[27773]: pgmap v371: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T14:39:59.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.057+0000 7fd730c00700 1 -- 172.21.15.53:0/1943506536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd72c108790 msgr2=0x7fd72c10ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:59.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.057+0000 7fd730c00700 1 --2- 172.21.15.53:0/1943506536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd72c108790 0x7fd72c10ac00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd72400b0a0 tx=0x7fd72400b3b0 comp rx=0 tx=0).stop 2024-04-25T14:39:59.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.058+0000 7fd730c00700 1 -- 172.21.15.53:0/1943506536 shutdown_connections 2024-04-25T14:39:59.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.058+0000 7fd730c00700 1 --2- 172.21.15.53:0/1943506536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd72c108790 0x7fd72c10ac00 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:59.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.058+0000 7fd730c00700 1 --2- 172.21.15.53:0/1943506536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd72c103530 0x7fd72c105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:59.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.058+0000 7fd730c00700 1 --2- 172.21.15.53:0/1943506536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd72c105e60 0x7fd72c108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:59.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.058+0000 7fd730c00700 1 -- 172.21.15.53:0/1943506536 >> 172.21.15.53:0/1943506536 conn(0x7fd72c0fcf40 msgr2=0x7fd72c0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:59.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.058+0000 7fd730c00700 1 -- 172.21.15.53:0/1943506536 shutdown_connections 2024-04-25T14:39:59.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.058+0000 7fd730c00700 1 -- 172.21.15.53:0/1943506536 wait complete. 2024-04-25T14:39:59.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.059+0000 7fd730c00700 1 Processor -- start 2024-04-25T14:39:59.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.059+0000 7fd730c00700 1 -- start start 2024-04-25T14:39:59.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.059+0000 7fd730c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd72c103530 0x7fd72c19adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:59.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.059+0000 7fd730c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd72c105e60 0x7fd72c19b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:59.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.059+0000 7fd730c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd72c108790 0x7fd72c1a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:59.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.059+0000 7fd730c00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c10e500 con 0x7fd72c105e60 2024-04-25T14:39:59.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.059+0000 7fd730c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c10e380 con 0x7fd72c103530 2024-04-25T14:39:59.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.059+0000 7fd730c00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c10e680 con 0x7fd72c108790 2024-04-25T14:39:59.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.060+0000 7fd729e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd72c103530 0x7fd72c19adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:59.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.060+0000 7fd729400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd72c105e60 0x7fd72c19b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:59.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.060+0000 7fd729e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd72c103530 0x7fd72c19adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.53:60900/0 (socket says 172.21.15.53:60900) 2024-04-25T14:39:59.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.060+0000 7fd729e00700 1 -- 172.21.15.53:0/4269955798 learned_addr learned my addr 172.21.15.53:0/4269955798 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:39:59.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.060+0000 7fd72a800700 1 --2- 172.21.15.53:0/4269955798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd72c108790 0x7fd72c1a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:59.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.060+0000 7fd72a800700 1 -- 172.21.15.53:0/4269955798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd72c103530 msgr2=0x7fd72c19adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:59.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.060+0000 7fd72a800700 1 --2- 172.21.15.53:0/4269955798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd72c103530 0x7fd72c19adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:59.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.060+0000 7fd72a800700 1 -- 172.21.15.53:0/4269955798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd72c105e60 msgr2=0x7fd72c19b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:59.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.060+0000 7fd72a800700 1 --2- 172.21.15.53:0/4269955798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd72c105e60 0x7fd72c19b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:59.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.060+0000 7fd72a800700 1 -- 172.21.15.53:0/4269955798 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd72c1a2a50 con 0x7fd72c108790 2024-04-25T14:39:59.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.060+0000 7fd729e00700 1 --2- 172.21.15.53:0/4269955798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd72c103530 0x7fd72c19adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:39:59.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.060+0000 7fd729400700 1 --2- 172.21.15.53:0/4269955798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd72c105e60 0x7fd72c19b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:39:59.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.061+0000 7fd72a800700 1 --2- 172.21.15.53:0/4269955798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd72c108790 0x7fd72c1a23b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd72400d010 tx=0x7fd724004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:59.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.061+0000 7fd722a00700 1 -- 172.21.15.53:0/4269955798 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd724021070 con 0x7fd72c108790 2024-04-25T14:39:59.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.061+0000 7fd730c00700 1 -- 172.21.15.53:0/4269955798 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd72c1a2ce0 con 0x7fd72c108790 2024-04-25T14:39:59.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.061+0000 7fd722a00700 1 -- 172.21.15.53:0/4269955798 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd724007e30 con 0x7fd72c108790 2024-04-25T14:39:59.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.061+0000 7fd722a00700 1 -- 172.21.15.53:0/4269955798 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd724004730 con 0x7fd72c108790 2024-04-25T14:39:59.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.061+0000 7fd730c00700 1 -- 172.21.15.53:0/4269955798 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd72c1a3190 con 0x7fd72c108790 2024-04-25T14:39:59.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.062+0000 7fd730c00700 1 -- 172.21.15.53:0/4269955798 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd72c069730 con 0x7fd72c108790 2024-04-25T14:39:59.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.064+0000 7fd722a00700 1 -- 172.21.15.53:0/4269955798 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd724003e40 con 0x7fd72c108790 2024-04-25T14:39:59.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.066+0000 7fd722a00700 1 --2- 172.21.15.53:0/4269955798 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fd7100a3c40 0x7fd7100a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:39:59.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.066+0000 7fd722a00700 1 -- 172.21.15.53:0/4269955798 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fd7240d1660 con 0x7fd72c108790 2024-04-25T14:39:59.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.066+0000 7fd729e00700 1 --2- 172.21.15.53:0/4269955798 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fd7100a3c40 0x7fd7100a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:39:59.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.067+0000 7fd729e00700 1 --2- 172.21.15.53:0/4269955798 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fd7100a3c40 0x7fd7100a6100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd71c0104e0 tx=0x7fd71c007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:39:59.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.068+0000 7fd722a00700 1 -- 172.21.15.53:0/4269955798 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd72409e060 con 0x7fd72c108790 2024-04-25T14:39:59.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.320+0000 7fd730c00700 1 -- 172.21.15.53:0/4269955798 --> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fd72c062e80 con 0x7fd7100a3c40 2024-04-25T14:39:59.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.324+0000 7fd722a00700 1 -- 172.21.15.53:0/4269955798 <== mgr.14152 v2:172.21.15.53:6800/4154933108 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fd72c062e80 con 0x7fd7100a3c40 2024-04-25T14:39:59.325 INFO:teuthology.orchestra.run.smithi053.stdout:Backend: cephadm 2024-04-25T14:39:59.325 INFO:teuthology.orchestra.run.smithi053.stdout:Available: Yes 2024-04-25T14:39:59.325 INFO:teuthology.orchestra.run.smithi053.stdout:Paused: No 2024-04-25T14:39:59.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.327+0000 7fd730c00700 1 -- 172.21.15.53:0/4269955798 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fd7100a3c40 msgr2=0x7fd7100a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:59.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.327+0000 7fd730c00700 1 --2- 172.21.15.53:0/4269955798 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fd7100a3c40 0x7fd7100a6100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd71c0104e0 tx=0x7fd71c007480 comp rx=0 tx=0).stop 2024-04-25T14:39:59.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.327+0000 7fd730c00700 1 -- 172.21.15.53:0/4269955798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd72c108790 msgr2=0x7fd72c1a23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:39:59.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.327+0000 7fd730c00700 1 --2- 172.21.15.53:0/4269955798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd72c108790 0x7fd72c1a23b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd72400d010 tx=0x7fd724004d00 comp rx=0 tx=0).stop 2024-04-25T14:39:59.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.327+0000 7fd730c00700 1 -- 172.21.15.53:0/4269955798 shutdown_connections 2024-04-25T14:39:59.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.327+0000 7fd730c00700 1 --2- 172.21.15.53:0/4269955798 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fd7100a3c40 0x7fd7100a6100 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:59.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.327+0000 7fd730c00700 1 --2- 172.21.15.53:0/4269955798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd72c108790 0x7fd72c1a23b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:59.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.327+0000 7fd730c00700 1 --2- 172.21.15.53:0/4269955798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd72c103530 0x7fd72c19adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:59.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.327+0000 7fd730c00700 1 --2- 172.21.15.53:0/4269955798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd72c105e60 0x7fd72c19b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:39:59.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.327+0000 7fd730c00700 1 -- 172.21.15.53:0/4269955798 >> 172.21.15.53:0/4269955798 conn(0x7fd72c0fcf40 msgr2=0x7fd72c102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:39:59.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.327+0000 7fd730c00700 1 -- 172.21.15.53:0/4269955798 shutdown_connections 2024-04-25T14:39:59.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:39:59.327+0000 7fd730c00700 1 -- 172.21.15.53:0/4269955798 wait complete. 2024-04-25T14:39:59.689 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:39:59 smithi053 ceph-mon[27773]: pgmap v372: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T14:39:59.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:39:59 smithi183 ceph-mon[32584]: pgmap v372: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T14:39:59.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:39:59 smithi150 ceph-mon[32504]: pgmap v372: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T14:40:00.223 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-25T14:40:00.681 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:40:00.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:00 smithi183 ceph-mon[32584]: from='client.25054 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:40:00.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:00 smithi183 ceph-mon[32584]: overall HEALTH_OK 2024-04-25T14:40:00.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:00 smithi150 ceph-mon[32504]: from='client.25054 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:40:00.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:00 smithi150 ceph-mon[32504]: overall HEALTH_OK 2024-04-25T14:40:00.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:00 smithi053 ceph-mon[27773]: from='client.25054 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:40:00.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:00 smithi053 ceph-mon[27773]: overall HEALTH_OK 2024-04-25T14:40:01.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:01 smithi053 ceph-mon[27773]: pgmap v373: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-25T14:40:01.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:01 smithi183 ceph-mon[32584]: pgmap v373: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-25T14:40:01.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:01 smithi150 ceph-mon[32504]: pgmap v373: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-25T14:40:02.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:02.954+0000 7f817e400700 1 -- 172.21.15.53:0/3361791694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f817810a2e0 msgr2=0x7f817810a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:02.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:02.954+0000 7f817e400700 1 --2- 172.21.15.53:0/3361791694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f817810a2e0 0x7f817810a740 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8164009990 tx=0x7f8164009ca0 comp rx=0 tx=0).stop 2024-04-25T14:40:02.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:02.954+0000 7f817e400700 1 -- 172.21.15.53:0/3361791694 shutdown_connections 2024-04-25T14:40:02.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:02.954+0000 7f817e400700 1 --2- 172.21.15.53:0/3361791694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f817810ac80 0x7f8178111530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:02.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:02.954+0000 7f817e400700 1 --2- 172.21.15.53:0/3361791694 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81781090e0 0x7f8178109500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:02.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:02.954+0000 7f817e400700 1 --2- 172.21.15.53:0/3361791694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f817810a2e0 0x7f817810a740 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:02.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:02.954+0000 7f817e400700 1 -- 172.21.15.53:0/3361791694 >> 172.21.15.53:0/3361791694 conn(0x7f81780780b0 msgr2=0x7f817807a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:02.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:02.955+0000 7f817e400700 1 -- 172.21.15.53:0/3361791694 shutdown_connections 2024-04-25T14:40:02.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:02.955+0000 7f817e400700 1 -- 172.21.15.53:0/3361791694 wait complete. 2024-04-25T14:40:02.957 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:02.956+0000 7f817e400700 1 Processor -- start 2024-04-25T14:40:02.957 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:02.956+0000 7f817e400700 1 -- start start 2024-04-25T14:40:02.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:02.957+0000 7f817e400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81781090e0 0x7f817819f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:02.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:02.957+0000 7f817e400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f817810a2e0 0x7f817819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:02.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:02.957+0000 7f817e400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f817810ac80 0x7f81781a6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:02.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:02.957+0000 7f817e400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8178114b50 con 0x7f817810a2e0 2024-04-25T14:40:02.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:02.957+0000 7f817e400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f81781149d0 con 0x7f81781090e0 2024-04-25T14:40:02.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:02.957+0000 7f817e400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8178114cd0 con 0x7f817810ac80 2024-04-25T14:40:02.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:02.957+0000 7f8176a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f817810a2e0 0x7f817819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:02.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:02.957+0000 7f8177400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81781090e0 0x7f817819f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:02.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:02.957+0000 7f8177400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81781090e0 0x7f817819f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.53:45122/0 (socket says 172.21.15.53:45122) 2024-04-25T14:40:02.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:02.957+0000 7f8177400700 1 -- 172.21.15.53:0/1687488381 learned_addr learned my addr 172.21.15.53:0/1687488381 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:02.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:02.957+0000 7f8177e00700 1 --2- 172.21.15.53:0/1687488381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f817810ac80 0x7f81781a6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:02.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:02.958+0000 7f8176a00700 1 -- 172.21.15.53:0/1687488381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f817810ac80 msgr2=0x7f81781a6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:02.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:02.958+0000 7f8176a00700 1 --2- 172.21.15.53:0/1687488381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f817810ac80 0x7f81781a6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:02.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:02.958+0000 7f8176a00700 1 -- 172.21.15.53:0/1687488381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81781090e0 msgr2=0x7f817819f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:02.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:02.958+0000 7f8176a00700 1 --2- 172.21.15.53:0/1687488381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81781090e0 0x7f817819f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:02.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:02.958+0000 7f8176a00700 1 -- 172.21.15.53:0/1687488381 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81781a6e20 con 0x7f817810a2e0 2024-04-25T14:40:02.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:02.958+0000 7f8177e00700 1 --2- 172.21.15.53:0/1687488381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f817810ac80 0x7f81781a6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:40:02.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:02.958+0000 7f8176a00700 1 --2- 172.21.15.53:0/1687488381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f817810a2e0 0x7f817819f700 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8164000c00 tx=0x7f81640047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:02.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:02.958+0000 7f816fe00700 1 -- 172.21.15.53:0/1687488381 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816400fad0 con 0x7f817810a2e0 2024-04-25T14:40:02.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:02.959+0000 7f817e400700 1 -- 172.21.15.53:0/1687488381 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81781a70b0 con 0x7f817810a2e0 2024-04-25T14:40:02.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:02.959+0000 7f816fe00700 1 -- 172.21.15.53:0/1687488381 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f816400fc70 con 0x7f817810a2e0 2024-04-25T14:40:02.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:02.959+0000 7f816fe00700 1 -- 172.21.15.53:0/1687488381 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816400e640 con 0x7f817810a2e0 2024-04-25T14:40:02.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:02.959+0000 7f817e400700 1 -- 172.21.15.53:0/1687488381 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81781a74a0 con 0x7f817810a2e0 2024-04-25T14:40:02.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:02.960+0000 7f817e400700 1 -- 172.21.15.53:0/1687488381 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8178069730 con 0x7f817810a2e0 2024-04-25T14:40:02.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:02.963+0000 7f816fe00700 1 -- 172.21.15.53:0/1687488381 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8164029060 con 0x7f817810a2e0 2024-04-25T14:40:02.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:02.965+0000 7f816fe00700 1 --2- 172.21.15.53:0/1687488381 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f815c0a3cc0 0x7f815c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:02.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:02.965+0000 7f816fe00700 1 -- 172.21.15.53:0/1687488381 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f81640d12e0 con 0x7f817810a2e0 2024-04-25T14:40:02.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:02.965+0000 7f8177400700 1 --2- 172.21.15.53:0/1687488381 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f815c0a3cc0 0x7f815c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:02.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:02.965+0000 7f8177400700 1 --2- 172.21.15.53:0/1687488381 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f815c0a3cc0 0x7f815c0a6180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8168005950 tx=0x7f8168009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:02.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:02.969+0000 7f816fe00700 1 -- 172.21.15.53:0/1687488381 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8164013070 con 0x7f817810a2e0 2024-04-25T14:40:03.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:03.217+0000 7f817e400700 1 -- 172.21.15.53:0/1687488381 --> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f8178062e80 con 0x7f815c0a3cc0 2024-04-25T14:40:03.259 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:03.259+0000 7f816fe00700 1 -- 172.21.15.53:0/1687488381 <== mgr.14152 v2:172.21.15.53:6800/4154933108 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f8178062e80 con 0x7f815c0a3cc0 2024-04-25T14:40:03.260 INFO:teuthology.orchestra.run.smithi053.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T14:40:03.260 INFO:teuthology.orchestra.run.smithi053.stdout:mds.a smithi053 running (2m) 25s ago 2m 12.3M - 19.0.0-3156-gd69f13d8 da4b004c09e8 39903d2c0c72 2024-04-25T14:40:03.260 INFO:teuthology.orchestra.run.smithi053.stdout:mds.b smithi150 running (118s) 49s ago 117s 25.1M - 19.0.0-3156-gd69f13d8 da4b004c09e8 835db8e020fa 2024-04-25T14:40:03.260 INFO:teuthology.orchestra.run.smithi053.stdout:mds.c smithi183 running (2m) 49s ago 2m 20.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 bd74bae21e32 2024-04-25T14:40:03.260 INFO:teuthology.orchestra.run.smithi053.stdout:mds.d smithi053 running (2m) 25s ago 2m 12.5M - 19.0.0-3156-gd69f13d8 da4b004c09e8 a35a0e9c6b51 2024-04-25T14:40:03.260 INFO:teuthology.orchestra.run.smithi053.stdout:mds.e smithi150 running (2m) 49s ago 2m 15.5M - 19.0.0-3156-gd69f13d8 da4b004c09e8 b270a6af510a 2024-04-25T14:40:03.260 INFO:teuthology.orchestra.run.smithi053.stdout:mds.f smithi183 running (2m) 49s ago 2m 19.1M - 19.0.0-3156-gd69f13d8 da4b004c09e8 63624d8b533c 2024-04-25T14:40:03.261 INFO:teuthology.orchestra.run.smithi053.stdout:mds.g smithi053 running (2m) 25s ago 2m 17.0M - 19.0.0-3156-gd69f13d8 da4b004c09e8 67aef1549e18 2024-04-25T14:40:03.261 INFO:teuthology.orchestra.run.smithi053.stdout:mds.h smithi150 running (2m) 49s ago 2m 16.0M - 19.0.0-3156-gd69f13d8 da4b004c09e8 803c34c5a1c6 2024-04-25T14:40:03.261 INFO:teuthology.orchestra.run.smithi053.stdout:mds.i smithi183 running (2m) 49s ago 2m 20.4M - 19.0.0-3156-gd69f13d8 da4b004c09e8 90c0d1203a2f 2024-04-25T14:40:03.261 INFO:teuthology.orchestra.run.smithi053.stdout:mds.j smithi053 running (2m) 25s ago 2m 12.6M - 19.0.0-3156-gd69f13d8 da4b004c09e8 5baff6f1ec70 2024-04-25T14:40:03.261 INFO:teuthology.orchestra.run.smithi053.stdout:mds.k smithi150 running (2m) 49s ago 2m 13.9M - 19.0.0-3156-gd69f13d8 da4b004c09e8 21b3427b8c01 2024-04-25T14:40:03.261 INFO:teuthology.orchestra.run.smithi053.stdout:mds.l smithi183 running (2m) 49s ago 2m 14.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 de96912deee8 2024-04-25T14:40:03.261 INFO:teuthology.orchestra.run.smithi053.stdout:mgr.x smithi053 *:9283,8765 running (10m) 25s ago 10m 514M - 19.0.0-3156-gd69f13d8 da4b004c09e8 a15886d6773d 2024-04-25T14:40:03.261 INFO:teuthology.orchestra.run.smithi053.stdout:mgr.y smithi150 *:8443,8765 running (8m) 49s ago 8m 454M - 19.0.0-3156-gd69f13d8 da4b004c09e8 637d815f7da0 2024-04-25T14:40:03.261 INFO:teuthology.orchestra.run.smithi053.stdout:mgr.z smithi183 *:8443,8765 running (8m) 49s ago 8m 451M - 19.0.0-3156-gd69f13d8 da4b004c09e8 71b0b1274392 2024-04-25T14:40:03.261 INFO:teuthology.orchestra.run.smithi053.stdout:mon.a smithi053 running (10m) 25s ago 11m 48.7M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 fc4336fa1ec1 2024-04-25T14:40:03.261 INFO:teuthology.orchestra.run.smithi053.stdout:mon.b smithi150 running (8m) 49s ago 8m 42.7M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 807bc911e52d 2024-04-25T14:40:03.261 INFO:teuthology.orchestra.run.smithi053.stdout:mon.c smithi183 running (8m) 49s ago 8m 40.1M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 d39383c27b47 2024-04-25T14:40:03.261 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0 smithi053 running (7m) 25s ago 7m 53.1M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 46b6b22d17bd 2024-04-25T14:40:03.261 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1 smithi150 running (7m) 49s ago 7m 47.9M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 c82f8eed2e4f 2024-04-25T14:40:03.261 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2 smithi183 running (7m) 49s ago 7m 57.4M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 4ad7b8c31b09 2024-04-25T14:40:03.261 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3 smithi053 running (6m) 25s ago 6m 45.4M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 5867d9706c06 2024-04-25T14:40:03.262 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4 smithi150 running (6m) 49s ago 6m 53.5M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 f89a6f1a1d24 2024-04-25T14:40:03.262 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5 smithi183 running (5m) 49s ago 5m 57.3M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 9498b3c73c9e 2024-04-25T14:40:03.262 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6 smithi053 running (5m) 25s ago 5m 43.6M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 9fde94d2e826 2024-04-25T14:40:03.262 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7 smithi150 running (4m) 49s ago 4m 46.7M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 873cf903b4e5 2024-04-25T14:40:03.262 INFO:teuthology.orchestra.run.smithi053.stdout:osd.8 smithi183 running (4m) 49s ago 4m 54.9M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 caf9cd7c0459 2024-04-25T14:40:03.262 INFO:teuthology.orchestra.run.smithi053.stdout:osd.9 smithi053 running (3m) 25s ago 3m 43.6M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 6d5d402ef1e2 2024-04-25T14:40:03.262 INFO:teuthology.orchestra.run.smithi053.stdout:osd.10 smithi150 running (3m) 49s ago 3m 45.9M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 23336e851497 2024-04-25T14:40:03.262 INFO:teuthology.orchestra.run.smithi053.stdout:osd.11 smithi183 running (3m) 49s ago 3m 54.7M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 a940eabdc2d7 2024-04-25T14:40:03.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:03.262+0000 7f817e400700 1 -- 172.21.15.53:0/1687488381 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f815c0a3cc0 msgr2=0x7f815c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:03.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:03.262+0000 7f817e400700 1 --2- 172.21.15.53:0/1687488381 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f815c0a3cc0 0x7f815c0a6180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8168005950 tx=0x7f8168009260 comp rx=0 tx=0).stop 2024-04-25T14:40:03.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:03.262+0000 7f817e400700 1 -- 172.21.15.53:0/1687488381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f817810a2e0 msgr2=0x7f817819f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:03.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:03.262+0000 7f817e400700 1 --2- 172.21.15.53:0/1687488381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f817810a2e0 0x7f817819f700 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8164000c00 tx=0x7f81640047b0 comp rx=0 tx=0).stop 2024-04-25T14:40:03.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:03.262+0000 7f817e400700 1 -- 172.21.15.53:0/1687488381 shutdown_connections 2024-04-25T14:40:03.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:03.262+0000 7f817e400700 1 --2- 172.21.15.53:0/1687488381 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f815c0a3cc0 0x7f815c0a6180 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:03.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:03.262+0000 7f817e400700 1 --2- 172.21.15.53:0/1687488381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f817810ac80 0x7f81781a6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:03.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:03.262+0000 7f817e400700 1 --2- 172.21.15.53:0/1687488381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81781090e0 0x7f817819f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:03.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:03.262+0000 7f817e400700 1 --2- 172.21.15.53:0/1687488381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f817810a2e0 0x7f817819f700 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:03.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:03.262+0000 7f817e400700 1 -- 172.21.15.53:0/1687488381 >> 172.21.15.53:0/1687488381 conn(0x7f81780780b0 msgr2=0x7f817810d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:03.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:03.262+0000 7f817e400700 1 -- 172.21.15.53:0/1687488381 shutdown_connections 2024-04-25T14:40:03.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:03.263+0000 7f817e400700 1 -- 172.21.15.53:0/1687488381 wait complete. 2024-04-25T14:40:03.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:03 smithi053 ceph-mon[27773]: pgmap v374: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T14:40:03.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:03 smithi183 ceph-mon[32584]: pgmap v374: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T14:40:03.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:03 smithi150 ceph-mon[32504]: pgmap v374: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T14:40:04.059 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-25T14:40:04.525 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:40:04.682 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:04 smithi053 ceph-mon[27773]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:40:04.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:04 smithi183 ceph-mon[32584]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:40:04.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:04 smithi150 ceph-mon[32504]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:40:05.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:05 smithi183 ceph-mon[32584]: pgmap v375: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T14:40:05.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:05 smithi150 ceph-mon[32504]: pgmap v375: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T14:40:05.719 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:05 smithi053 ceph-mon[27773]: pgmap v375: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T14:40:06.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:06.777+0000 7f32da000700 1 -- 172.21.15.53:0/2769586737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32d4076a10 msgr2=0x7f32d4074e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:06.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:06.777+0000 7f32da000700 1 --2- 172.21.15.53:0/2769586737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32d4076a10 0x7f32d4074e70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f32c4009990 tx=0x7f32c4009ca0 comp rx=0 tx=0).stop 2024-04-25T14:40:06.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:06.777+0000 7f32da000700 1 -- 172.21.15.53:0/2769586737 shutdown_connections 2024-04-25T14:40:06.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:06.777+0000 7f32da000700 1 --2- 172.21.15.53:0/2769586737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32d410a9d0 0x7f32d410ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:06.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:06.777+0000 7f32da000700 1 --2- 172.21.15.53:0/2769586737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32d4076a10 0x7f32d4074e70 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:06.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:06.777+0000 7f32da000700 1 --2- 172.21.15.53:0/2769586737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32d4075440 0x7f32d40758c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:06.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:06.777+0000 7f32da000700 1 -- 172.21.15.53:0/2769586737 >> 172.21.15.53:0/2769586737 conn(0x7f32d40fe090 msgr2=0x7f32d41004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:06.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:06.777+0000 7f32da000700 1 -- 172.21.15.53:0/2769586737 shutdown_connections 2024-04-25T14:40:06.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:06.777+0000 7f32da000700 1 -- 172.21.15.53:0/2769586737 wait complete. 2024-04-25T14:40:06.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:06.778+0000 7f32da000700 1 Processor -- start 2024-04-25T14:40:06.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:06.778+0000 7f32da000700 1 -- start start 2024-04-25T14:40:06.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:06.778+0000 7f32da000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32d4075440 0x7f32d419adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:06.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:06.778+0000 7f32da000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32d4076a10 0x7f32d419b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:06.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:06.778+0000 7f32da000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32d410a9d0 0x7f32d41a2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:06.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:06.778+0000 7f32da000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f32d4110790 con 0x7f32d4076a10 2024-04-25T14:40:06.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:06.778+0000 7f32da000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f32d4110610 con 0x7f32d410a9d0 2024-04-25T14:40:06.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:06.778+0000 7f32da000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f32d4110910 con 0x7f32d4075440 2024-04-25T14:40:06.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:06.779+0000 7f32d3400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32d4075440 0x7f32d419adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:06.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:06.779+0000 7f32d3400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32d4075440 0x7f32d419adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:51070/0 (socket says 172.21.15.53:51070) 2024-04-25T14:40:06.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:06.779+0000 7f32d3400700 1 -- 172.21.15.53:0/2091138134 learned_addr learned my addr 172.21.15.53:0/2091138134 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:06.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:06.779+0000 7f32cbe00700 1 --2- 172.21.15.53:0/2091138134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32d4076a10 0x7f32d419b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:06.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:06.779+0000 7f32d3e00700 1 --2- 172.21.15.53:0/2091138134 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32d410a9d0 0x7f32d41a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:06.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:06.779+0000 7f32d3400700 1 -- 172.21.15.53:0/2091138134 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32d410a9d0 msgr2=0x7f32d41a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:06.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:06.779+0000 7f32d3400700 1 --2- 172.21.15.53:0/2091138134 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32d410a9d0 0x7f32d41a2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:06.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:06.779+0000 7f32d3400700 1 -- 172.21.15.53:0/2091138134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32d4076a10 msgr2=0x7f32d419b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:06.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:06.779+0000 7f32d3400700 1 --2- 172.21.15.53:0/2091138134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32d4076a10 0x7f32d419b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:06.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:06.779+0000 7f32d3400700 1 -- 172.21.15.53:0/2091138134 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32d41a2a10 con 0x7f32d4075440 2024-04-25T14:40:06.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:06.780+0000 7f32d3400700 1 --2- 172.21.15.53:0/2091138134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32d4075440 0x7f32d419adb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f32c400b760 tx=0x7f32c40057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:06.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:06.780+0000 7f32d0c00700 1 -- 172.21.15.53:0/2091138134 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c4027070 con 0x7f32d4075440 2024-04-25T14:40:06.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:06.780+0000 7f32da000700 1 -- 172.21.15.53:0/2091138134 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32d41a2ca0 con 0x7f32d4075440 2024-04-25T14:40:06.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:06.780+0000 7f32d0c00700 1 -- 172.21.15.53:0/2091138134 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32c4022470 con 0x7f32d4075440 2024-04-25T14:40:06.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:06.780+0000 7f32d0c00700 1 -- 172.21.15.53:0/2091138134 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c4017650 con 0x7f32d4075440 2024-04-25T14:40:06.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:06.780+0000 7f32da000700 1 -- 172.21.15.53:0/2091138134 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32d41a3150 con 0x7f32d4075440 2024-04-25T14:40:06.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:06.781+0000 7f32da000700 1 -- 172.21.15.53:0/2091138134 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32d4069730 con 0x7f32d4075440 2024-04-25T14:40:06.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:06.783+0000 7f32d0c00700 1 -- 172.21.15.53:0/2091138134 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f32c402a070 con 0x7f32d4075440 2024-04-25T14:40:06.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:06.784+0000 7f32d0c00700 1 --2- 172.21.15.53:0/2091138134 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f32b80a3cc0 0x7f32b80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:06.785 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:06.785+0000 7f32d0c00700 1 -- 172.21.15.53:0/2091138134 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f32c40d11b0 con 0x7f32d4075440 2024-04-25T14:40:06.785 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:06.785+0000 7f32cbe00700 1 --2- 172.21.15.53:0/2091138134 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f32b80a3cc0 0x7f32b80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:06.785 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:06.785+0000 7f32cbe00700 1 --2- 172.21.15.53:0/2091138134 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f32b80a3cc0 0x7f32b80a6180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f32bc009680 tx=0x7f32bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:06.789 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:06.789+0000 7f32d0c00700 1 -- 172.21.15.53:0/2091138134 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f32c4013070 con 0x7f32d4075440 2024-04-25T14:40:07.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:07.030+0000 7f32da000700 1 -- 172.21.15.53:0/2091138134 --> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f32d4062e80 con 0x7f32b80a3cc0 2024-04-25T14:40:07.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:07.050+0000 7f32d0c00700 1 -- 172.21.15.53:0/2091138134 <== mgr.14152 v2:172.21.15.53:6800/4154933108 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f32d4062e80 con 0x7f32b80a3cc0 2024-04-25T14:40:07.051 INFO:teuthology.orchestra.run.smithi053.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-25T14:40:07.051 INFO:teuthology.orchestra.run.smithi053.stdout:mds.all 12/12 53s ago 2m smithi053=a;smithi053=d;smithi053=g;smithi053=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l;count:12 2024-04-25T14:40:07.051 INFO:teuthology.orchestra.run.smithi053.stdout:mgr 3/3 53s ago 8m smithi053=x;smithi150=y;smithi183=z;count:3 2024-04-25T14:40:07.051 INFO:teuthology.orchestra.run.smithi053.stdout:mon 3/3 53s ago 9m smithi053:172.21.15.53=a;smithi150:172.21.15.150=b;smithi183:172.21.15.183=c;count:3 2024-04-25T14:40:07.051 INFO:teuthology.orchestra.run.smithi053.stdout:osd 12 53s ago - 2024-04-25T14:40:07.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:07.054+0000 7f32da000700 1 -- 172.21.15.53:0/2091138134 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f32b80a3cc0 msgr2=0x7f32b80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:07.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:07.054+0000 7f32da000700 1 --2- 172.21.15.53:0/2091138134 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f32b80a3cc0 0x7f32b80a6180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f32bc009680 tx=0x7f32bc009260 comp rx=0 tx=0).stop 2024-04-25T14:40:07.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:07.054+0000 7f32da000700 1 -- 172.21.15.53:0/2091138134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32d4075440 msgr2=0x7f32d419adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:07.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:07.054+0000 7f32da000700 1 --2- 172.21.15.53:0/2091138134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32d4075440 0x7f32d419adb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f32c400b760 tx=0x7f32c40057c0 comp rx=0 tx=0).stop 2024-04-25T14:40:07.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:07.054+0000 7f32da000700 1 -- 172.21.15.53:0/2091138134 shutdown_connections 2024-04-25T14:40:07.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:07.054+0000 7f32da000700 1 --2- 172.21.15.53:0/2091138134 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f32b80a3cc0 0x7f32b80a6180 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:07.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:07.054+0000 7f32da000700 1 --2- 172.21.15.53:0/2091138134 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32d410a9d0 0x7f32d41a2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:07.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:07.054+0000 7f32da000700 1 --2- 172.21.15.53:0/2091138134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32d4075440 0x7f32d419adb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:07.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:07.054+0000 7f32da000700 1 --2- 172.21.15.53:0/2091138134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32d4076a10 0x7f32d419b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:07.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:07.054+0000 7f32da000700 1 -- 172.21.15.53:0/2091138134 >> 172.21.15.53:0/2091138134 conn(0x7f32d40fe090 msgr2=0x7f32d4106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:07.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:07.055+0000 7f32da000700 1 -- 172.21.15.53:0/2091138134 shutdown_connections 2024-04-25T14:40:07.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:07.055+0000 7f32da000700 1 -- 172.21.15.53:0/2091138134 wait complete. 2024-04-25T14:40:07.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:07 smithi183 ceph-mon[32584]: pgmap v376: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T14:40:07.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:07 smithi183 ceph-mon[32584]: from='client.25066 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:40:07.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:07 smithi150 ceph-mon[32504]: pgmap v376: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T14:40:07.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:07 smithi150 ceph-mon[32504]: from='client.25066 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:40:07.762 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:07 smithi053 ceph-mon[27773]: pgmap v376: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T14:40:07.762 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:07 smithi053 ceph-mon[27773]: from='client.25066 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:40:07.814 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-25T14:40:08.286 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:40:09.654 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:09 smithi053 ceph-mon[27773]: pgmap v377: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T14:40:09.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:09 smithi183 ceph-mon[32584]: pgmap v377: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T14:40:09.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:09 smithi150 ceph-mon[32504]: pgmap v377: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T14:40:10.565 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.565+0000 7f85dcc00700 1 -- 172.21.15.53:0/4234555235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85d8106780 msgr2=0x7f85d810d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:10.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.565+0000 7f85dcc00700 1 --2- 172.21.15.53:0/4234555235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85d8106780 0x7f85d810d030 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f85d0009fc0 tx=0x7f85d000b330 comp rx=0 tx=0).stop 2024-04-25T14:40:10.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.565+0000 7f85dcc00700 1 -- 172.21.15.53:0/4234555235 shutdown_connections 2024-04-25T14:40:10.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.565+0000 7f85dcc00700 1 --2- 172.21.15.53:0/4234555235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85d8106780 0x7f85d810d030 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:10.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.565+0000 7f85dcc00700 1 --2- 172.21.15.53:0/4234555235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85d8104be0 0x7f85d8105000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:10.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.565+0000 7f85dcc00700 1 --2- 172.21.15.53:0/4234555235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85d8105de0 0x7f85d8106240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:10.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.565+0000 7f85dcc00700 1 -- 172.21.15.53:0/4234555235 >> 172.21.15.53:0/4234555235 conn(0x7f85d8100180 msgr2=0x7f85d81025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:10.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.565+0000 7f85dcc00700 1 -- 172.21.15.53:0/4234555235 shutdown_connections 2024-04-25T14:40:10.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.565+0000 7f85dcc00700 1 -- 172.21.15.53:0/4234555235 wait complete. 2024-04-25T14:40:10.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.566+0000 7f85dcc00700 1 Processor -- start 2024-04-25T14:40:10.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.566+0000 7f85dcc00700 1 -- start start 2024-04-25T14:40:10.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.567+0000 7f85dcc00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85d8104be0 0x7f85d819ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:10.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.567+0000 7f85dcc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85d8105de0 0x7f85d819b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:10.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.567+0000 7f85dcc00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85d8106780 0x7f85d81a2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:10.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.567+0000 7f85dcc00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f85d81106b0 con 0x7f85d8105de0 2024-04-25T14:40:10.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.567+0000 7f85dcc00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f85d8110530 con 0x7f85d8104be0 2024-04-25T14:40:10.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.567+0000 7f85dcc00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f85d8110830 con 0x7f85d8106780 2024-04-25T14:40:10.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.567+0000 7f85d7400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85d8104be0 0x7f85d819ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:10.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.567+0000 7f85d7400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85d8104be0 0x7f85d819ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.53:45154/0 (socket says 172.21.15.53:45154) 2024-04-25T14:40:10.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.567+0000 7f85d7400700 1 -- 172.21.15.53:0/2922209456 learned_addr learned my addr 172.21.15.53:0/2922209456 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:10.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.567+0000 7f85cea00700 1 --2- 172.21.15.53:0/2922209456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85d8105de0 0x7f85d819b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:10.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.567+0000 7f85d7e00700 1 --2- 172.21.15.53:0/2922209456 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85d8106780 0x7f85d81a2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:10.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.568+0000 7f85d7400700 1 -- 172.21.15.53:0/2922209456 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85d8106780 msgr2=0x7f85d81a2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:10.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.568+0000 7f85d7400700 1 --2- 172.21.15.53:0/2922209456 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85d8106780 0x7f85d81a2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:10.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.568+0000 7f85d7400700 1 -- 172.21.15.53:0/2922209456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85d8105de0 msgr2=0x7f85d819b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:10.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.568+0000 7f85d7400700 1 --2- 172.21.15.53:0/2922209456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85d8105de0 0x7f85d819b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:10.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.568+0000 7f85d7400700 1 -- 172.21.15.53:0/2922209456 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d81a28f0 con 0x7f85d8104be0 2024-04-25T14:40:10.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.568+0000 7f85d7e00700 1 --2- 172.21.15.53:0/2922209456 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85d8106780 0x7f85d81a2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:40:10.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.568+0000 7f85d7400700 1 --2- 172.21.15.53:0/2922209456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85d8104be0 0x7f85d819ac70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f85c800e9f0 tx=0x7f85c800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:10.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.568+0000 7f85d4c00700 1 -- 172.21.15.53:0/2922209456 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c8004220 con 0x7f85d8104be0 2024-04-25T14:40:10.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.568+0000 7f85d4c00700 1 -- 172.21.15.53:0/2922209456 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85c80043c0 con 0x7f85d8104be0 2024-04-25T14:40:10.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.568+0000 7f85dcc00700 1 -- 172.21.15.53:0/2922209456 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85d81a2be0 con 0x7f85d8104be0 2024-04-25T14:40:10.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.568+0000 7f85d4c00700 1 -- 172.21.15.53:0/2922209456 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c8003ce0 con 0x7f85d8104be0 2024-04-25T14:40:10.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.569+0000 7f85dcc00700 1 -- 172.21.15.53:0/2922209456 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85d81a3170 con 0x7f85d8104be0 2024-04-25T14:40:10.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.570+0000 7f85dcc00700 1 -- 172.21.15.53:0/2922209456 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85d8069730 con 0x7f85d8104be0 2024-04-25T14:40:10.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.572+0000 7f85d4c00700 1 -- 172.21.15.53:0/2922209456 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f85c800c7c0 con 0x7f85d8104be0 2024-04-25T14:40:10.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.574+0000 7f85d4c00700 1 --2- 172.21.15.53:0/2922209456 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f85bc0a3cc0 0x7f85bc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:10.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.574+0000 7f85d4c00700 1 -- 172.21.15.53:0/2922209456 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f85c80d0a20 con 0x7f85d8104be0 2024-04-25T14:40:10.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.574+0000 7f85cea00700 1 --2- 172.21.15.53:0/2922209456 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f85bc0a3cc0 0x7f85bc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:10.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.575+0000 7f85cea00700 1 --2- 172.21.15.53:0/2922209456 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f85bc0a3cc0 0x7f85bc0a6180 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f85c0009680 tx=0x7f85c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:10.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.576+0000 7f85d4c00700 1 -- 172.21.15.53:0/2922209456 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f85c801f340 con 0x7f85d8104be0 2024-04-25T14:40:10.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.815+0000 7f85dcc00700 1 -- 172.21.15.53:0/2922209456 --> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f85d8062e80 con 0x7f85bc0a3cc0 2024-04-25T14:40:10.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.818+0000 7f85d4c00700 1 -- 172.21.15.53:0/2922209456 <== mgr.14152 v2:172.21.15.53:6800/4154933108 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f85d8062e80 con 0x7f85bc0a3cc0 2024-04-25T14:40:10.818 INFO:teuthology.orchestra.run.smithi053.stdout:HOST ADDR LABELS STATUS 2024-04-25T14:40:10.819 INFO:teuthology.orchestra.run.smithi053.stdout:smithi053 172.21.15.53 2024-04-25T14:40:10.819 INFO:teuthology.orchestra.run.smithi053.stdout:smithi150 172.21.15.150 2024-04-25T14:40:10.819 INFO:teuthology.orchestra.run.smithi053.stdout:smithi183 172.21.15.183 2024-04-25T14:40:10.819 INFO:teuthology.orchestra.run.smithi053.stdout:3 hosts in cluster 2024-04-25T14:40:10.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.821+0000 7f85dcc00700 1 -- 172.21.15.53:0/2922209456 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f85bc0a3cc0 msgr2=0x7f85bc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:10.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.821+0000 7f85dcc00700 1 --2- 172.21.15.53:0/2922209456 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f85bc0a3cc0 0x7f85bc0a6180 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f85c0009680 tx=0x7f85c0009260 comp rx=0 tx=0).stop 2024-04-25T14:40:10.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.821+0000 7f85dcc00700 1 -- 172.21.15.53:0/2922209456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85d8104be0 msgr2=0x7f85d819ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:10.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.821+0000 7f85dcc00700 1 --2- 172.21.15.53:0/2922209456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85d8104be0 0x7f85d819ac70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f85c800e9f0 tx=0x7f85c800ed00 comp rx=0 tx=0).stop 2024-04-25T14:40:10.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.822+0000 7f85dcc00700 1 -- 172.21.15.53:0/2922209456 shutdown_connections 2024-04-25T14:40:10.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.822+0000 7f85dcc00700 1 --2- 172.21.15.53:0/2922209456 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f85bc0a3cc0 0x7f85bc0a6180 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:10.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.822+0000 7f85dcc00700 1 --2- 172.21.15.53:0/2922209456 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85d8106780 0x7f85d81a2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:10.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.822+0000 7f85dcc00700 1 --2- 172.21.15.53:0/2922209456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85d8104be0 0x7f85d819ac70 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:10.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.822+0000 7f85dcc00700 1 --2- 172.21.15.53:0/2922209456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85d8105de0 0x7f85d819b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:10.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.822+0000 7f85dcc00700 1 -- 172.21.15.53:0/2922209456 >> 172.21.15.53:0/2922209456 conn(0x7f85d8100180 msgr2=0x7f85d8109160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:10.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.822+0000 7f85dcc00700 1 -- 172.21.15.53:0/2922209456 shutdown_connections 2024-04-25T14:40:10.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:10.822+0000 7f85dcc00700 1 -- 172.21.15.53:0/2922209456 wait complete. 2024-04-25T14:40:11.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:11 smithi053 ceph-mon[27773]: pgmap v378: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T14:40:11.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:11 smithi053 ceph-mon[27773]: from='client.25055 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:40:11.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:11 smithi183 ceph-mon[32584]: pgmap v378: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T14:40:11.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:11 smithi183 ceph-mon[32584]: from='client.25055 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:40:11.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:11 smithi150 ceph-mon[32504]: pgmap v378: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T14:40:11.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:11 smithi150 ceph-mon[32504]: from='client.25055 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:40:11.751 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-25T14:40:12.228 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:40:13.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:13 smithi053 ceph-mon[27773]: pgmap v379: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T14:40:13.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:13 smithi183 ceph-mon[32584]: pgmap v379: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T14:40:13.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:13 smithi150 ceph-mon[32504]: pgmap v379: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T14:40:14.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.502+0000 7fe972e00700 1 -- 172.21.15.53:0/3428153553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe96c101320 msgr2=0x7fe96c101740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:14.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.502+0000 7fe972e00700 1 --2- 172.21.15.53:0/3428153553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe96c101320 0x7fe96c101740 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe95c0098f0 tx=0x7fe95c009c00 comp rx=0 tx=0).stop 2024-04-25T14:40:14.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.502+0000 7fe972e00700 1 -- 172.21.15.53:0/3428153553 shutdown_connections 2024-04-25T14:40:14.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.502+0000 7fe972e00700 1 --2- 172.21.15.53:0/3428153553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe96c102640 0x7fe96c102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:14.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.502+0000 7fe972e00700 1 --2- 172.21.15.53:0/3428153553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe96c101320 0x7fe96c101740 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:14.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.502+0000 7fe972e00700 1 --2- 172.21.15.53:0/3428153553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe96c101c80 0x7fe96c102100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:14.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.502+0000 7fe972e00700 1 -- 172.21.15.53:0/3428153553 >> 172.21.15.53:0/3428153553 conn(0x7fe96c0fcf60 msgr2=0x7fe96c0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:14.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.502+0000 7fe972e00700 1 -- 172.21.15.53:0/3428153553 shutdown_connections 2024-04-25T14:40:14.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.503+0000 7fe972e00700 1 -- 172.21.15.53:0/3428153553 wait complete. 2024-04-25T14:40:14.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.504+0000 7fe972e00700 1 Processor -- start 2024-04-25T14:40:14.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.504+0000 7fe972e00700 1 -- start start 2024-04-25T14:40:14.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.504+0000 7fe972e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe96c101320 0x7fe96c19ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:14.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.505+0000 7fe972e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe96c101c80 0x7fe96c19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:14.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.505+0000 7fe972e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe96c102640 0x7fe96c1a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:14.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.505+0000 7fe972e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe96c1106f0 con 0x7fe96c102640 2024-04-25T14:40:14.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.505+0000 7fe972e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe96c110570 con 0x7fe96c101320 2024-04-25T14:40:14.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.505+0000 7fe972e00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe96c110870 con 0x7fe96c101c80 2024-04-25T14:40:14.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.505+0000 7fe96be00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe96c101320 0x7fe96c19ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:14.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.505+0000 7fe96b400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe96c101c80 0x7fe96c19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:14.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.505+0000 7fe970e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe96c102640 0x7fe96c1a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:14.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.505+0000 7fe96be00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe96c101320 0x7fe96c19ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.53:54826/0 (socket says 172.21.15.53:54826) 2024-04-25T14:40:14.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.505+0000 7fe96be00700 1 -- 172.21.15.53:0/273056368 learned_addr learned my addr 172.21.15.53:0/273056368 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:14.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.505+0000 7fe96be00700 1 -- 172.21.15.53:0/273056368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe96c101c80 msgr2=0x7fe96c19b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:14.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.505+0000 7fe96be00700 1 --2- 172.21.15.53:0/273056368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe96c101c80 0x7fe96c19b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:14.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.505+0000 7fe96be00700 1 -- 172.21.15.53:0/273056368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe96c102640 msgr2=0x7fe96c1a23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:14.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.505+0000 7fe96be00700 1 --2- 172.21.15.53:0/273056368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe96c102640 0x7fe96c1a23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:14.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.505+0000 7fe96be00700 1 -- 172.21.15.53:0/273056368 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe96c1a2a60 con 0x7fe96c101320 2024-04-25T14:40:14.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.505+0000 7fe96b400700 1 --2- 172.21.15.53:0/273056368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe96c101c80 0x7fe96c19b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:40:14.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.505+0000 7fe970e00700 1 --2- 172.21.15.53:0/273056368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe96c102640 0x7fe96c1a23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:40:14.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.506+0000 7fe96be00700 1 --2- 172.21.15.53:0/273056368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe96c101320 0x7fe96c19ae00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00fc20 tx=0x7fe95c005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:14.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.506+0000 7fe968c00700 1 -- 172.21.15.53:0/273056368 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95c005490 con 0x7fe96c101320 2024-04-25T14:40:14.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.506+0000 7fe968c00700 1 -- 172.21.15.53:0/273056368 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe95c005630 con 0x7fe96c101320 2024-04-25T14:40:14.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.506+0000 7fe968c00700 1 -- 172.21.15.53:0/273056368 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95c003b40 con 0x7fe96c101320 2024-04-25T14:40:14.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.506+0000 7fe972e00700 1 -- 172.21.15.53:0/273056368 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe96c1a2cf0 con 0x7fe96c101320 2024-04-25T14:40:14.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.506+0000 7fe972e00700 1 -- 172.21.15.53:0/273056368 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe96c1a30f0 con 0x7fe96c101320 2024-04-25T14:40:14.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.507+0000 7fe972e00700 1 -- 172.21.15.53:0/273056368 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe96c069770 con 0x7fe96c101320 2024-04-25T14:40:14.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.509+0000 7fe968c00700 1 -- 172.21.15.53:0/273056368 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe95c003ce0 con 0x7fe96c101320 2024-04-25T14:40:14.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.511+0000 7fe968c00700 1 --2- 172.21.15.53:0/273056368 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fe9500a3cc0 0x7fe9500a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:14.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.511+0000 7fe968c00700 1 -- 172.21.15.53:0/273056368 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fe95c0d13e0 con 0x7fe96c101320 2024-04-25T14:40:14.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.511+0000 7fe96b400700 1 --2- 172.21.15.53:0/273056368 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fe9500a3cc0 0x7fe9500a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:14.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.512+0000 7fe96b400700 1 --2- 172.21.15.53:0/273056368 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fe9500a3cc0 0x7fe9500a6180 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe958005fd0 tx=0x7fe958009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:14.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.515+0000 7fe968c00700 1 -- 172.21.15.53:0/273056368 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe95c013030 con 0x7fe96c101320 2024-04-25T14:40:14.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.756+0000 7fe972e00700 1 -- 172.21.15.53:0/273056368 --> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fe96c062e80 con 0x7fe9500a3cc0 2024-04-25T14:40:14.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.759+0000 7fe968c00700 1 -- 172.21.15.53:0/273056368 <== mgr.14152 v2:172.21.15.53:6800/4154933108 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fe96c062e80 con 0x7fe9500a3cc0 2024-04-25T14:40:14.759 INFO:teuthology.orchestra.run.smithi053.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-25T14:40:14.759 INFO:teuthology.orchestra.run.smithi053.stdout:smithi053 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300G8480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-04-25T14:40:14.759 INFO:teuthology.orchestra.run.smithi053.stdout:smithi150 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GX400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-25T14:40:14.759 INFO:teuthology.orchestra.run.smithi053.stdout:smithi183 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500QJ400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-25T14:40:14.762 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.762+0000 7fe972e00700 1 -- 172.21.15.53:0/273056368 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fe9500a3cc0 msgr2=0x7fe9500a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:14.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.762+0000 7fe972e00700 1 --2- 172.21.15.53:0/273056368 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fe9500a3cc0 0x7fe9500a6180 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe958005fd0 tx=0x7fe958009310 comp rx=0 tx=0).stop 2024-04-25T14:40:14.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.762+0000 7fe972e00700 1 -- 172.21.15.53:0/273056368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe96c101320 msgr2=0x7fe96c19ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:14.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.762+0000 7fe972e00700 1 --2- 172.21.15.53:0/273056368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe96c101320 0x7fe96c19ae00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00fc20 tx=0x7fe95c005270 comp rx=0 tx=0).stop 2024-04-25T14:40:14.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.762+0000 7fe972e00700 1 -- 172.21.15.53:0/273056368 shutdown_connections 2024-04-25T14:40:14.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.762+0000 7fe972e00700 1 --2- 172.21.15.53:0/273056368 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fe9500a3cc0 0x7fe9500a6180 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:14.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.762+0000 7fe972e00700 1 --2- 172.21.15.53:0/273056368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe96c102640 0x7fe96c1a23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:14.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.762+0000 7fe972e00700 1 --2- 172.21.15.53:0/273056368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe96c101320 0x7fe96c19ae00 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:14.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.762+0000 7fe972e00700 1 --2- 172.21.15.53:0/273056368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe96c101c80 0x7fe96c19b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:14.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.762+0000 7fe972e00700 1 -- 172.21.15.53:0/273056368 >> 172.21.15.53:0/273056368 conn(0x7fe96c0fcf60 msgr2=0x7fe96c0fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:14.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.763+0000 7fe972e00700 1 -- 172.21.15.53:0/273056368 shutdown_connections 2024-04-25T14:40:14.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:14.763+0000 7fe972e00700 1 -- 172.21.15.53:0/273056368 wait complete. 2024-04-25T14:40:15.521 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T14:40:15.530 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi053.front.sepia.ceph.com 2024-04-25T14:40:15.530 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-25T14:40:15.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:15 smithi150 ceph-mon[32504]: pgmap v380: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T14:40:15.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:15 smithi150 ceph-mon[32504]: from='client.25061 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:40:15.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:15 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-25T14:40:15.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:15 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [5, 7, 3, 9]}]: dispatch 2024-04-25T14:40:15.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:15 smithi053 ceph-mon[27773]: pgmap v380: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T14:40:15.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:15 smithi053 ceph-mon[27773]: from='client.25061 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:40:15.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:15 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-25T14:40:15.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:15 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [5, 7, 3, 9]}]: dispatch 2024-04-25T14:40:15.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:15 smithi183 ceph-mon[32584]: pgmap v380: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T14:40:15.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:15 smithi183 ceph-mon[32584]: from='client.25061 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:40:15.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:15 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-25T14:40:15.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:15 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [5, 7, 3, 9]}]: dispatch 2024-04-25T14:40:15.991 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:40:16.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:16 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-25T14:40:16.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:16 smithi053 ceph-mon[27773]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [5, 7, 3, 9]}]': finished 2024-04-25T14:40:16.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:16 smithi053 ceph-mon[27773]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T14:40:16.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:16 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-25T14:40:16.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:16 smithi183 ceph-mon[32584]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [5, 7, 3, 9]}]': finished 2024-04-25T14:40:16.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:16 smithi183 ceph-mon[32584]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T14:40:16.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:16 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-25T14:40:16.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:16 smithi150 ceph-mon[32504]: from='mgr.14152 172.21.15.53:0/4077751069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [5, 7, 3, 9]}]': finished 2024-04-25T14:40:16.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:16 smithi150 ceph-mon[32504]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T14:40:17.769 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:17 smithi053 ceph-mon[27773]: pgmap v382: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T14:40:17.769 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:17 smithi053 ceph-mon[27773]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T14:40:17.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:17 smithi183 ceph-mon[32584]: pgmap v382: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T14:40:17.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:17 smithi183 ceph-mon[32584]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T14:40:17.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:17 smithi150 ceph-mon[32504]: pgmap v382: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T14:40:17.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:17 smithi150 ceph-mon[32504]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T14:40:18.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.379+0000 7f5c55000700 1 -- 172.21.15.53:0/2623701484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c5010a980 msgr2=0x7f5c5010ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:18.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.379+0000 7f5c55000700 1 --2- 172.21.15.53:0/2623701484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c5010a980 0x7f5c5010ce10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5c4800b0a0 tx=0x7f5c4800b3b0 comp rx=0 tx=0).stop 2024-04-25T14:40:18.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.379+0000 7f5c55000700 1 -- 172.21.15.53:0/2623701484 shutdown_connections 2024-04-25T14:40:18.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.380+0000 7f5c55000700 1 --2- 172.21.15.53:0/2623701484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c5010a980 0x7f5c5010ce10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:18.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.380+0000 7f5c55000700 1 --2- 172.21.15.53:0/2623701484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c50076a10 0x7f5c50074e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:18.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.380+0000 7f5c55000700 1 --2- 172.21.15.53:0/2623701484 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c50075440 0x7f5c500758c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:18.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.380+0000 7f5c55000700 1 -- 172.21.15.53:0/2623701484 >> 172.21.15.53:0/2623701484 conn(0x7f5c500fe040 msgr2=0x7f5c50100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:18.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.380+0000 7f5c55000700 1 -- 172.21.15.53:0/2623701484 shutdown_connections 2024-04-25T14:40:18.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.380+0000 7f5c55000700 1 -- 172.21.15.53:0/2623701484 wait complete. 2024-04-25T14:40:18.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.381+0000 7f5c55000700 1 Processor -- start 2024-04-25T14:40:18.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.381+0000 7f5c55000700 1 -- start start 2024-04-25T14:40:18.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.381+0000 7f5c55000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c50075440 0x7f5c5019ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:18.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.381+0000 7f5c55000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c50076a10 0x7f5c5019b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:18.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.381+0000 7f5c55000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c5010a980 0x7f5c501a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:18.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.381+0000 7f5c55000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5c50110740 con 0x7f5c50076a10 2024-04-25T14:40:18.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.381+0000 7f5c55000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5c501105c0 con 0x7f5c5010a980 2024-04-25T14:40:18.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.381+0000 7f5c55000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5c501108c0 con 0x7f5c50075440 2024-04-25T14:40:18.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.382+0000 7f5c4d400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c50076a10 0x7f5c5019b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:18.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.382+0000 7f5c4de00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c50075440 0x7f5c5019ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:18.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.382+0000 7f5c4de00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c50075440 0x7f5c5019ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:44212/0 (socket says 172.21.15.53:44212) 2024-04-25T14:40:18.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.382+0000 7f5c4de00700 1 -- 172.21.15.53:0/1776671177 learned_addr learned my addr 172.21.15.53:0/1776671177 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:18.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.382+0000 7f5c4e800700 1 --2- 172.21.15.53:0/1776671177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c5010a980 0x7f5c501a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:18.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.382+0000 7f5c4de00700 1 -- 172.21.15.53:0/1776671177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c5010a980 msgr2=0x7f5c501a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:18.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.382+0000 7f5c4de00700 1 --2- 172.21.15.53:0/1776671177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c5010a980 0x7f5c501a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:18.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.382+0000 7f5c4de00700 1 -- 172.21.15.53:0/1776671177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c50076a10 msgr2=0x7f5c5019b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:18.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.382+0000 7f5c4de00700 1 --2- 172.21.15.53:0/1776671177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c50076a10 0x7f5c5019b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:18.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.382+0000 7f5c4de00700 1 -- 172.21.15.53:0/1776671177 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c501a29e0 con 0x7f5c50075440 2024-04-25T14:40:18.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.382+0000 7f5c4d400700 1 --2- 172.21.15.53:0/1776671177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c50076a10 0x7f5c5019b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:40:18.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.383+0000 7f5c4de00700 1 --2- 172.21.15.53:0/1776671177 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c50075440 0x7f5c5019ad90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5c4000e9a0 tx=0x7f5c4000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:18.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.383+0000 7f5c46a00700 1 -- 172.21.15.53:0/1776671177 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4000ca10 con 0x7f5c50075440 2024-04-25T14:40:18.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.383+0000 7f5c46a00700 1 -- 172.21.15.53:0/1776671177 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c4000cbb0 con 0x7f5c50075440 2024-04-25T14:40:18.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.383+0000 7f5c46a00700 1 -- 172.21.15.53:0/1776671177 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c40018680 con 0x7f5c50075440 2024-04-25T14:40:18.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.383+0000 7f5c55000700 1 -- 172.21.15.53:0/1776671177 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c501a2cd0 con 0x7f5c50075440 2024-04-25T14:40:18.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.383+0000 7f5c55000700 1 -- 172.21.15.53:0/1776671177 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c501a31c0 con 0x7f5c50075440 2024-04-25T14:40:18.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.384+0000 7f5c55000700 1 -- 172.21.15.53:0/1776671177 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c50069730 con 0x7f5c50075440 2024-04-25T14:40:18.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.387+0000 7f5c46a00700 1 -- 172.21.15.53:0/1776671177 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5c40010430 con 0x7f5c50075440 2024-04-25T14:40:18.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.388+0000 7f5c46a00700 1 --2- 172.21.15.53:0/1776671177 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5c340a3cc0 0x7f5c340a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:18.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.388+0000 7f5c46a00700 1 -- 172.21.15.53:0/1776671177 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8285+0+0 (secure 0 0 0) 0x7f5c400d08a0 con 0x7f5c50075440 2024-04-25T14:40:18.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.388+0000 7f5c4d400700 1 --2- 172.21.15.53:0/1776671177 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5c340a3cc0 0x7f5c340a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:18.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.389+0000 7f5c4d400700 1 --2- 172.21.15.53:0/1776671177 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5c340a3cc0 0x7f5c340a6180 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c011490 tx=0x7f5c3c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:18.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.390+0000 7f5c46a00700 1 -- 172.21.15.53:0/1776671177 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5c4009d1f0 con 0x7f5c50075440 2024-04-25T14:40:18.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.683+0000 7f5c55000700 1 -- 172.21.15.53:0/1776671177 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f5c50063650 con 0x7f5c50075440 2024-04-25T14:40:18.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.684+0000 7f5c46a00700 1 -- 172.21.15.53:0/1776671177 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) ==== 76+0+3044 (secure 0 0 0) 0x7f5c4009caf0 con 0x7f5c50075440 2024-04-25T14:40:18.685 INFO:teuthology.orchestra.run.smithi053.stdout:e21 2024-04-25T14:40:18.685 INFO:teuthology.orchestra.run.smithi053.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T14:40:18.685 INFO:teuthology.orchestra.run.smithi053.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T14:40:18.685 INFO:teuthology.orchestra.run.smithi053.stdout:legacy client fscid: 1 2024-04-25T14:40:18.685 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:40:18.685 INFO:teuthology.orchestra.run.smithi053.stdout:Filesystem 'cephfs' (1) 2024-04-25T14:40:18.685 INFO:teuthology.orchestra.run.smithi053.stdout:fs_name cephfs 2024-04-25T14:40:18.685 INFO:teuthology.orchestra.run.smithi053.stdout:epoch 21 2024-04-25T14:40:18.685 INFO:teuthology.orchestra.run.smithi053.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T14:40:18.685 INFO:teuthology.orchestra.run.smithi053.stdout:created 2024-04-25T14:38:10.037451+0000 2024-04-25T14:40:18.685 INFO:teuthology.orchestra.run.smithi053.stdout:modified 2024-04-25T14:38:15.629854+0000 2024-04-25T14:40:18.685 INFO:teuthology.orchestra.run.smithi053.stdout:tableserver 0 2024-04-25T14:40:18.685 INFO:teuthology.orchestra.run.smithi053.stdout:root 0 2024-04-25T14:40:18.686 INFO:teuthology.orchestra.run.smithi053.stdout:session_timeout 300 2024-04-25T14:40:18.686 INFO:teuthology.orchestra.run.smithi053.stdout:session_autoclose 300 2024-04-25T14:40:18.686 INFO:teuthology.orchestra.run.smithi053.stdout:max_file_size 1099511627776 2024-04-25T14:40:18.686 INFO:teuthology.orchestra.run.smithi053.stdout:max_xattr_size 65536 2024-04-25T14:40:18.686 INFO:teuthology.orchestra.run.smithi053.stdout:required_client_features {} 2024-04-25T14:40:18.686 INFO:teuthology.orchestra.run.smithi053.stdout:last_failure 0 2024-04-25T14:40:18.686 INFO:teuthology.orchestra.run.smithi053.stdout:last_failure_osd_epoch 0 2024-04-25T14:40:18.686 INFO:teuthology.orchestra.run.smithi053.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T14:40:18.686 INFO:teuthology.orchestra.run.smithi053.stdout:max_mds 3 2024-04-25T14:40:18.686 INFO:teuthology.orchestra.run.smithi053.stdout:in 0,1,2 2024-04-25T14:40:18.686 INFO:teuthology.orchestra.run.smithi053.stdout:up {0=24584,1=24565,2=24560} 2024-04-25T14:40:18.686 INFO:teuthology.orchestra.run.smithi053.stdout:failed 2024-04-25T14:40:18.686 INFO:teuthology.orchestra.run.smithi053.stdout:damaged 2024-04-25T14:40:18.686 INFO:teuthology.orchestra.run.smithi053.stdout:stopped 2024-04-25T14:40:18.686 INFO:teuthology.orchestra.run.smithi053.stdout:data_pools [3] 2024-04-25T14:40:18.687 INFO:teuthology.orchestra.run.smithi053.stdout:metadata_pool 2 2024-04-25T14:40:18.687 INFO:teuthology.orchestra.run.smithi053.stdout:inline_data disabled 2024-04-25T14:40:18.687 INFO:teuthology.orchestra.run.smithi053.stdout:balancer 2024-04-25T14:40:18.687 INFO:teuthology.orchestra.run.smithi053.stdout:bal_rank_mask -1 2024-04-25T14:40:18.687 INFO:teuthology.orchestra.run.smithi053.stdout:standby_count_wanted 0 2024-04-25T14:40:18.687 INFO:teuthology.orchestra.run.smithi053.stdout:qdb_cluster leader: 24584 members: 24565,24584,24560 2024-04-25T14:40:18.687 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.b{0:24584} state up:active seq 3 addr [v2:172.21.15.150:6838/149573616,v1:172.21.15.150:6839/149573616] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T14:40:18.687 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.c{0:24544} state up:standby-replay seq 1 addr [v2:172.21.15.183:6836/1163336384,v1:172.21.15.183:6837/1163336384] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T14:40:18.687 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.i{1:24565} state up:active seq 6 addr [v2:172.21.15.183:6838/1554258915,v1:172.21.15.183:6839/1554258915] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T14:40:18.687 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.h{1:24533} state up:standby-replay seq 1 addr [v2:172.21.15.150:6836/2344343627,v1:172.21.15.150:6837/2344343627] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T14:40:18.687 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.g{2:24560} state up:active seq 6 addr [v2:172.21.15.53:6838/570000540,v1:172.21.15.53:6839/570000540] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T14:40:18.687 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.f{2:24532} state up:standby-replay seq 1 addr [v2:172.21.15.183:6834/1189328602,v1:172.21.15.183:6835/1189328602] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T14:40:18.687 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:40:18.687 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:40:18.688 INFO:teuthology.orchestra.run.smithi053.stdout:Standby daemons: 2024-04-25T14:40:18.688 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:40:18.688 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.a{-1:14460} state up:standby seq 1 addr [v2:172.21.15.53:6840/41110727,v1:172.21.15.53:6841/41110727] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T14:40:18.688 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.k{-1:24452} state up:standby seq 1 addr [v2:172.21.15.150:6832/2966992472,v1:172.21.15.150:6833/2966992472] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T14:40:18.688 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.l{-1:24460} state up:standby seq 1 addr [v2:172.21.15.183:6832/94143643,v1:172.21.15.183:6833/94143643] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T14:40:18.688 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.j{-1:24470} state up:standby seq 1 addr [v2:172.21.15.53:6834/506682943,v1:172.21.15.53:6835/506682943] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T14:40:18.688 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.e{-1:24500} state up:standby seq 1 addr [v2:172.21.15.150:6834/1369488844,v1:172.21.15.150:6835/1369488844] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T14:40:18.688 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.d{-1:24502} state up:standby seq 1 addr [v2:172.21.15.53:6836/3786232682,v1:172.21.15.53:6837/3786232682] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T14:40:18.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.687+0000 7f5c55000700 1 -- 172.21.15.53:0/1776671177 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5c340a3cc0 msgr2=0x7f5c340a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:18.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.687+0000 7f5c55000700 1 --2- 172.21.15.53:0/1776671177 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5c340a3cc0 0x7f5c340a6180 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c011490 tx=0x7f5c3c009260 comp rx=0 tx=0).stop 2024-04-25T14:40:18.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.687+0000 7f5c55000700 1 -- 172.21.15.53:0/1776671177 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c50075440 msgr2=0x7f5c5019ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:18.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.687+0000 7f5c55000700 1 --2- 172.21.15.53:0/1776671177 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c50075440 0x7f5c5019ad90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5c4000e9a0 tx=0x7f5c4000ecb0 comp rx=0 tx=0).stop 2024-04-25T14:40:18.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.687+0000 7f5c55000700 1 -- 172.21.15.53:0/1776671177 shutdown_connections 2024-04-25T14:40:18.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.687+0000 7f5c55000700 1 --2- 172.21.15.53:0/1776671177 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f5c340a3cc0 0x7f5c340a6180 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:18.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.687+0000 7f5c55000700 1 --2- 172.21.15.53:0/1776671177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c5010a980 0x7f5c501a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:18.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.687+0000 7f5c55000700 1 --2- 172.21.15.53:0/1776671177 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c50075440 0x7f5c5019ad90 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:18.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.687+0000 7f5c55000700 1 --2- 172.21.15.53:0/1776671177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c50076a10 0x7f5c5019b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:18.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.687+0000 7f5c55000700 1 -- 172.21.15.53:0/1776671177 >> 172.21.15.53:0/1776671177 conn(0x7f5c500fe040 msgr2=0x7f5c501068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:18.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.687+0000 7f5c55000700 1 -- 172.21.15.53:0/1776671177 shutdown_connections 2024-04-25T14:40:18.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:18.687+0000 7f5c55000700 1 -- 172.21.15.53:0/1776671177 wait complete. 2024-04-25T14:40:18.690 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 21 2024-04-25T14:40:19.470 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-25T14:40:19.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:19 smithi053 ceph-mon[27773]: pgmap v384: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:40:19.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:19 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1776671177' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T14:40:19.925 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:40:19.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:19 smithi183 ceph-mon[32584]: pgmap v384: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:40:19.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:19 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/1776671177' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T14:40:19.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:19 smithi150 ceph-mon[32504]: pgmap v384: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:40:19.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:19 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/1776671177' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T14:40:21.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:21 smithi053 ceph-mon[27773]: pgmap v385: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-25T14:40:21.763 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:21 smithi150 ceph-mon[32504]: pgmap v385: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-25T14:40:21.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:21 smithi183 ceph-mon[32584]: pgmap v385: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-25T14:40:22.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.110+0000 7fd3d3400700 1 -- 172.21.15.53:0/483809664 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3cc106890 msgr2=0x7fd3cc10d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:22.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.110+0000 7fd3d3400700 1 --2- 172.21.15.53:0/483809664 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3cc106890 0x7fd3cc10d140 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd3c400b0d0 tx=0x7fd3c400b3e0 comp rx=0 tx=0).stop 2024-04-25T14:40:22.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.110+0000 7fd3d3400700 1 -- 172.21.15.53:0/483809664 shutdown_connections 2024-04-25T14:40:22.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.110+0000 7fd3d3400700 1 --2- 172.21.15.53:0/483809664 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3cc106890 0x7fd3cc10d140 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:22.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.110+0000 7fd3d3400700 1 --2- 172.21.15.53:0/483809664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3cc104cf0 0x7fd3cc105110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:22.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.110+0000 7fd3d3400700 1 --2- 172.21.15.53:0/483809664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3cc105ef0 0x7fd3cc106350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:22.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.110+0000 7fd3d3400700 1 -- 172.21.15.53:0/483809664 >> 172.21.15.53:0/483809664 conn(0x7fd3cc100290 msgr2=0x7fd3cc1026d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:22.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.110+0000 7fd3d3400700 1 -- 172.21.15.53:0/483809664 shutdown_connections 2024-04-25T14:40:22.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.110+0000 7fd3d3400700 1 -- 172.21.15.53:0/483809664 wait complete. 2024-04-25T14:40:22.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.111+0000 7fd3d3400700 1 Processor -- start 2024-04-25T14:40:22.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.111+0000 7fd3d3400700 1 -- start start 2024-04-25T14:40:22.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.111+0000 7fd3d3400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3cc104cf0 0x7fd3cc19adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:22.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.111+0000 7fd3d3400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3cc105ef0 0x7fd3cc19b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:22.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.111+0000 7fd3d3400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3cc106890 0x7fd3cc1a2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:22.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.111+0000 7fd3d3400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd3cc1107c0 con 0x7fd3cc106890 2024-04-25T14:40:22.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.111+0000 7fd3d3400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd3cc110640 con 0x7fd3cc104cf0 2024-04-25T14:40:22.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.111+0000 7fd3d3400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd3cc110940 con 0x7fd3cc105ef0 2024-04-25T14:40:22.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.112+0000 7fd3d1400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3cc106890 0x7fd3cc1a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:22.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.112+0000 7fd3d0a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3cc104cf0 0x7fd3cc19adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:22.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.112+0000 7fd3d1400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3cc106890 0x7fd3cc1a2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47486/0 (socket says 172.21.15.53:47486) 2024-04-25T14:40:22.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.112+0000 7fd3d1400700 1 -- 172.21.15.53:0/1266634361 learned_addr learned my addr 172.21.15.53:0/1266634361 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:22.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.112+0000 7fd3cbe00700 1 --2- 172.21.15.53:0/1266634361 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3cc105ef0 0x7fd3cc19b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:22.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.112+0000 7fd3d0a00700 1 -- 172.21.15.53:0/1266634361 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3cc105ef0 msgr2=0x7fd3cc19b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:22.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.112+0000 7fd3d0a00700 1 --2- 172.21.15.53:0/1266634361 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3cc105ef0 0x7fd3cc19b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:22.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.112+0000 7fd3d0a00700 1 -- 172.21.15.53:0/1266634361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3cc106890 msgr2=0x7fd3cc1a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:22.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.112+0000 7fd3d0a00700 1 --2- 172.21.15.53:0/1266634361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3cc106890 0x7fd3cc1a2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:22.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.112+0000 7fd3d0a00700 1 -- 172.21.15.53:0/1266634361 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3cc1a2a10 con 0x7fd3cc104cf0 2024-04-25T14:40:22.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.112+0000 7fd3d1400700 1 --2- 172.21.15.53:0/1266634361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3cc106890 0x7fd3cc1a2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:40:22.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.113+0000 7fd3d0a00700 1 --2- 172.21.15.53:0/1266634361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3cc104cf0 0x7fd3cc19adb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd3c000e9c0 tx=0x7fd3c000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:22.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.113+0000 7fd3c9600700 1 -- 172.21.15.53:0/1266634361 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c000cb70 con 0x7fd3cc104cf0 2024-04-25T14:40:22.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.113+0000 7fd3c9600700 1 -- 172.21.15.53:0/1266634361 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3c000ee60 con 0x7fd3cc104cf0 2024-04-25T14:40:22.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.113+0000 7fd3c9600700 1 -- 172.21.15.53:0/1266634361 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c0018680 con 0x7fd3cc104cf0 2024-04-25T14:40:22.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.113+0000 7fd3d3400700 1 -- 172.21.15.53:0/1266634361 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3cc1a2d00 con 0x7fd3cc104cf0 2024-04-25T14:40:22.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.113+0000 7fd3d3400700 1 -- 172.21.15.53:0/1266634361 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3cc1a31d0 con 0x7fd3cc104cf0 2024-04-25T14:40:22.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.114+0000 7fd3d3400700 1 -- 172.21.15.53:0/1266634361 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3cc069730 con 0x7fd3cc104cf0 2024-04-25T14:40:22.116 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.116+0000 7fd3c9600700 1 -- 172.21.15.53:0/1266634361 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd3c001f020 con 0x7fd3cc104cf0 2024-04-25T14:40:22.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.118+0000 7fd3c9600700 1 --2- 172.21.15.53:0/1266634361 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fd3b40a3cc0 0x7fd3b40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:22.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.118+0000 7fd3c9600700 1 -- 172.21.15.53:0/1266634361 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8285+0+0 (secure 0 0 0) 0x7fd3c00d0490 con 0x7fd3cc104cf0 2024-04-25T14:40:22.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.118+0000 7fd3cbe00700 1 --2- 172.21.15.53:0/1266634361 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fd3b40a3cc0 0x7fd3b40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:22.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.119+0000 7fd3cbe00700 1 --2- 172.21.15.53:0/1266634361 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fd3b40a3cc0 0x7fd3b40a6180 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8009c60 tx=0x7fd3b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:22.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.120+0000 7fd3c9600700 1 -- 172.21.15.53:0/1266634361 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd3c009cde0 con 0x7fd3cc104cf0 2024-04-25T14:40:22.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.361+0000 7fd3d3400700 1 -- 172.21.15.53:0/1266634361 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fd3cc063650 con 0x7fd3cc104cf0 2024-04-25T14:40:22.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.363+0000 7fd3c9600700 1 -- 172.21.15.53:0/1266634361 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v92) ==== 56+0+5325 (secure 0 0 0) 0x7fd3c009c6e0 con 0x7fd3cc104cf0 2024-04-25T14:40:22.364 INFO:teuthology.orchestra.run.smithi053.stdout:epoch 92 2024-04-25T14:40:22.364 INFO:teuthology.orchestra.run.smithi053.stdout:fsid fd30e6ce-030f-11ef-bc93-c7b262605968 2024-04-25T14:40:22.364 INFO:teuthology.orchestra.run.smithi053.stdout:created 2024-04-25T14:28:54.330326+0000 2024-04-25T14:40:22.364 INFO:teuthology.orchestra.run.smithi053.stdout:modified 2024-04-25T14:40:16.467090+0000 2024-04-25T14:40:22.364 INFO:teuthology.orchestra.run.smithi053.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-25T14:40:22.364 INFO:teuthology.orchestra.run.smithi053.stdout:crush_version 26 2024-04-25T14:40:22.364 INFO:teuthology.orchestra.run.smithi053.stdout:full_ratio 0.95 2024-04-25T14:40:22.364 INFO:teuthology.orchestra.run.smithi053.stdout:backfillfull_ratio 0.9 2024-04-25T14:40:22.364 INFO:teuthology.orchestra.run.smithi053.stdout:nearfull_ratio 0.85 2024-04-25T14:40:22.365 INFO:teuthology.orchestra.run.smithi053.stdout:require_min_compat_client luminous 2024-04-25T14:40:22.365 INFO:teuthology.orchestra.run.smithi053.stdout:min_compat_client luminous 2024-04-25T14:40:22.365 INFO:teuthology.orchestra.run.smithi053.stdout:require_osd_release squid 2024-04-25T14:40:22.365 INFO:teuthology.orchestra.run.smithi053.stdout:stretch_mode_enabled false 2024-04-25T14:40:22.365 INFO:teuthology.orchestra.run.smithi053.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-25T14:40:22.365 INFO:teuthology.orchestra.run.smithi053.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 87 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-25T14:40:22.365 INFO:teuthology.orchestra.run.smithi053.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 88 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-25T14:40:22.365 INFO:teuthology.orchestra.run.smithi053.stdout:max_osd 12 2024-04-25T14:40:22.365 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0 up in weight 1 up_from 8 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6802/464176015,v1:172.21.15.53:6803/464176015] [v2:172.21.15.53:6804/464176015,v1:172.21.15.53:6805/464176015] exists,up aecfbaf0-39da-4749-a49b-c7205dabd4d8 2024-04-25T14:40:22.365 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1 up in weight 1 up_from 13 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6800/1304621982,v1:172.21.15.150:6801/1304621982] [v2:172.21.15.150:6802/1304621982,v1:172.21.15.150:6803/1304621982] exists,up 2c4a21ae-c43e-4839-8a56-c3fc63aeb7e5 2024-04-25T14:40:22.365 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2 up in weight 1 up_from 18 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6800/574509311,v1:172.21.15.183:6801/574509311] [v2:172.21.15.183:6802/574509311,v1:172.21.15.183:6803/574509311] exists,up e8effd43-adc0-472a-9eed-a9273b06dc0e 2024-04-25T14:40:22.366 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3 up in weight 1 up_from 26 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6810/570672219,v1:172.21.15.53:6811/570672219] [v2:172.21.15.53:6812/570672219,v1:172.21.15.53:6813/570672219] exists,up e7cfced4-9b40-40de-a303-75ea8d5df4a1 2024-04-25T14:40:22.366 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4 up in weight 1 up_from 34 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6808/1526850103,v1:172.21.15.150:6809/1526850103] [v2:172.21.15.150:6810/1526850103,v1:172.21.15.150:6811/1526850103] exists,up 44a9e070-2282-43f1-9f05-b1f8ff81e540 2024-04-25T14:40:22.366 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5 up in weight 1 up_from 40 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6808/773726576,v1:172.21.15.183:6809/773726576] [v2:172.21.15.183:6810/773726576,v1:172.21.15.183:6811/773726576] exists,up 2c5619d2-c0f9-47d6-bf7e-2f4afd1b09f0 2024-04-25T14:40:22.366 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6 up in weight 1 up_from 48 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6818/355884319,v1:172.21.15.53:6819/355884319] [v2:172.21.15.53:6820/355884319,v1:172.21.15.53:6821/355884319] exists,up 1cf74d94-c118-4738-b6db-040af458a8eb 2024-04-25T14:40:22.366 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7 up in weight 1 up_from 53 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6816/229022562,v1:172.21.15.150:6817/229022562] [v2:172.21.15.150:6818/229022562,v1:172.21.15.150:6819/229022562] exists,up 0ce97c04-cd41-4080-bbbc-be9cf5fadc02 2024-04-25T14:40:22.366 INFO:teuthology.orchestra.run.smithi053.stdout:osd.8 up in weight 1 up_from 62 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6816/301003844,v1:172.21.15.183:6817/301003844] [v2:172.21.15.183:6818/301003844,v1:172.21.15.183:6819/301003844] exists,up 5d57787d-2a7c-4514-bd01-357ced413f68 2024-04-25T14:40:22.366 INFO:teuthology.orchestra.run.smithi053.stdout:osd.9 up in weight 1 up_from 70 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6826/3449260978,v1:172.21.15.53:6827/3449260978] [v2:172.21.15.53:6828/3449260978,v1:172.21.15.53:6829/3449260978] exists,up e11dd8eb-c55a-4d70-8c58-457540b7a50a 2024-04-25T14:40:22.366 INFO:teuthology.orchestra.run.smithi053.stdout:osd.10 up in weight 1 up_from 75 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6824/3435992308,v1:172.21.15.150:6825/3435992308] [v2:172.21.15.150:6826/3435992308,v1:172.21.15.150:6827/3435992308] exists,up 5de4803d-ad37-4a90-add5-0048f44fc4ca 2024-04-25T14:40:22.366 INFO:teuthology.orchestra.run.smithi053.stdout:osd.11 up in weight 1 up_from 83 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6824/1957842686,v1:172.21.15.183:6825/1957842686] [v2:172.21.15.183:6826/1957842686,v1:172.21.15.183:6827/1957842686] exists,up 0e819a68-ed56-477e-b9c0-81e8cc0bce11 2024-04-25T14:40:22.366 INFO:teuthology.orchestra.run.smithi053.stdout:pg_upmap_items 2.0 [8,4] 2024-04-25T14:40:22.366 INFO:teuthology.orchestra.run.smithi053.stdout:pg_upmap_items 2.4 [2,11] 2024-04-25T14:40:22.366 INFO:teuthology.orchestra.run.smithi053.stdout:pg_upmap_items 2.5 [8,11] 2024-04-25T14:40:22.366 INFO:teuthology.orchestra.run.smithi053.stdout:pg_upmap_items 2.1b [2,4] 2024-04-25T14:40:22.367 INFO:teuthology.orchestra.run.smithi053.stdout:pg_upmap_items 3.0 [4,6] 2024-04-25T14:40:22.367 INFO:teuthology.orchestra.run.smithi053.stdout:pg_upmap_items 3.1 [4,9] 2024-04-25T14:40:22.367 INFO:teuthology.orchestra.run.smithi053.stdout:pg_upmap_items 3.1b [4,8] 2024-04-25T14:40:22.367 INFO:teuthology.orchestra.run.smithi053.stdout:pg_upmap_items 3.1e [3,9,5,6] 2024-04-25T14:40:22.367 INFO:teuthology.orchestra.run.smithi053.stdout:pg_upmap_items 3.38 [5,7,3,9] 2024-04-25T14:40:22.367 INFO:teuthology.orchestra.run.smithi053.stdout:pg_upmap_items 3.3a [4,8] 2024-04-25T14:40:22.367 INFO:teuthology.orchestra.run.smithi053.stdout:blocklist 172.21.15.53:6800/3887954889 expires 2024-04-26T14:30:14.162383+0000 2024-04-25T14:40:22.367 INFO:teuthology.orchestra.run.smithi053.stdout:blocklist 172.21.15.53:0/1014501022 expires 2024-04-26T14:30:14.162383+0000 2024-04-25T14:40:22.367 INFO:teuthology.orchestra.run.smithi053.stdout:blocklist 172.21.15.53:0/2576091441 expires 2024-04-26T14:29:38.106242+0000 2024-04-25T14:40:22.367 INFO:teuthology.orchestra.run.smithi053.stdout:blocklist 172.21.15.53:0/1233527569 expires 2024-04-26T14:29:38.106242+0000 2024-04-25T14:40:22.367 INFO:teuthology.orchestra.run.smithi053.stdout:blocklist 172.21.15.53:6801/3887954889 expires 2024-04-26T14:30:14.162383+0000 2024-04-25T14:40:22.367 INFO:teuthology.orchestra.run.smithi053.stdout:blocklist 172.21.15.53:0/931889810 expires 2024-04-26T14:29:38.106242+0000 2024-04-25T14:40:22.367 INFO:teuthology.orchestra.run.smithi053.stdout:blocklist 172.21.15.53:0/177196141 expires 2024-04-26T14:30:14.162383+0000 2024-04-25T14:40:22.367 INFO:teuthology.orchestra.run.smithi053.stdout:blocklist 172.21.15.53:6800/1722497390 expires 2024-04-26T14:29:38.106242+0000 2024-04-25T14:40:22.367 INFO:teuthology.orchestra.run.smithi053.stdout:blocklist 172.21.15.53:6801/1722497390 expires 2024-04-26T14:29:38.106242+0000 2024-04-25T14:40:22.367 INFO:teuthology.orchestra.run.smithi053.stdout:blocklist 172.21.15.53:0/1927549198 expires 2024-04-26T14:30:14.162383+0000 2024-04-25T14:40:22.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.366+0000 7fd3d3400700 1 -- 172.21.15.53:0/1266634361 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fd3b40a3cc0 msgr2=0x7fd3b40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:22.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.366+0000 7fd3d3400700 1 --2- 172.21.15.53:0/1266634361 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fd3b40a3cc0 0x7fd3b40a6180 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8009c60 tx=0x7fd3b8009260 comp rx=0 tx=0).stop 2024-04-25T14:40:22.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.366+0000 7fd3d3400700 1 -- 172.21.15.53:0/1266634361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3cc104cf0 msgr2=0x7fd3cc19adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:22.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.366+0000 7fd3d3400700 1 --2- 172.21.15.53:0/1266634361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3cc104cf0 0x7fd3cc19adb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd3c000e9c0 tx=0x7fd3c000ecd0 comp rx=0 tx=0).stop 2024-04-25T14:40:22.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.366+0000 7fd3d3400700 1 -- 172.21.15.53:0/1266634361 shutdown_connections 2024-04-25T14:40:22.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.366+0000 7fd3d3400700 1 --2- 172.21.15.53:0/1266634361 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fd3b40a3cc0 0x7fd3b40a6180 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:22.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.366+0000 7fd3d3400700 1 --2- 172.21.15.53:0/1266634361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3cc106890 0x7fd3cc1a2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:22.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.366+0000 7fd3d3400700 1 --2- 172.21.15.53:0/1266634361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3cc104cf0 0x7fd3cc19adb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:22.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.366+0000 7fd3d3400700 1 --2- 172.21.15.53:0/1266634361 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3cc105ef0 0x7fd3cc19b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:22.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.366+0000 7fd3d3400700 1 -- 172.21.15.53:0/1266634361 >> 172.21.15.53:0/1266634361 conn(0x7fd3cc100290 msgr2=0x7fd3cc1091f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:22.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.366+0000 7fd3d3400700 1 -- 172.21.15.53:0/1266634361 shutdown_connections 2024-04-25T14:40:22.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:22.366+0000 7fd3d3400700 1 -- 172.21.15.53:0/1266634361 wait complete. 2024-04-25T14:40:22.734 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:22 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1266634361' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T14:40:22.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:22 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/1266634361' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T14:40:22.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:22 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/1266634361' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T14:40:23.125 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-25T14:40:23.137 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:40:23.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.365+0000 7f9cb3200640 1 -- 172.21.15.53:0/4074435512 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cb4077b50 msgr2=0x7f9cb4077fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:23.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.365+0000 7f9cb3200640 1 --2- 172.21.15.53:0/4074435512 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cb4077b50 0x7f9cb4077fb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c009860 tx=0x7f9c9c02a340 comp rx=0 tx=0).stop 2024-04-25T14:40:23.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.365+0000 7f9cb3200640 1 -- 172.21.15.53:0/4074435512 shutdown_connections 2024-04-25T14:40:23.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.365+0000 7f9cb3200640 1 --2- 172.21.15.53:0/4074435512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9cb40784f0 0x7f9cb4078d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:23.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.365+0000 7f9cb3200640 1 --2- 172.21.15.53:0/4074435512 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cb4077b50 0x7f9cb4077fb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:23.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.365+0000 7f9cb3200640 1 --2- 172.21.15.53:0/4074435512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9cb4076910 0x7f9cb4076d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:23.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.365+0000 7f9cb3200640 1 -- 172.21.15.53:0/4074435512 >> 172.21.15.53:0/4074435512 conn(0x7f9cb40ff560 msgr2=0x7f9cb4101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:23.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.365+0000 7f9cb3200640 1 -- 172.21.15.53:0/4074435512 shutdown_connections 2024-04-25T14:40:23.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.365+0000 7f9cb3200640 1 -- 172.21.15.53:0/4074435512 wait complete. 2024-04-25T14:40:23.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.366+0000 7f9cb3200640 1 Processor -- start 2024-04-25T14:40:23.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.366+0000 7f9cb3200640 1 -- start start 2024-04-25T14:40:23.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.367+0000 7f9cb3200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cb4076910 0x7f9cb419e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:23.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.367+0000 7f9cb3200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9cb4077b50 0x7f9cb419eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:23.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.367+0000 7f9cb3200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9cb40784f0 0x7f9cb41a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:23.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.367+0000 7f9cb3200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb4113f10 con 0x7f9cb40784f0 2024-04-25T14:40:23.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.367+0000 7f9cb3200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb4113d90 con 0x7f9cb4076910 2024-04-25T14:40:23.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.367+0000 7f9cb3200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb4114090 con 0x7f9cb4077b50 2024-04-25T14:40:23.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.367+0000 7f9cb2800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9cb40784f0 0x7f9cb41a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:23.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.367+0000 7f9cb1e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cb4076910 0x7f9cb419e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:23.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.367+0000 7f9cb2800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9cb40784f0 0x7f9cb41a5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36830/0 (socket says 172.21.15.53:36830) 2024-04-25T14:40:23.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.367+0000 7f9cb2800640 1 -- 172.21.15.53:0/1541807407 learned_addr learned my addr 172.21.15.53:0/1541807407 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:23.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.367+0000 7f9cb1e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cb4076910 0x7f9cb419e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.53:40522/0 (socket says 172.21.15.53:40522) 2024-04-25T14:40:23.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.367+0000 7f9ca9400640 1 --2- 172.21.15.53:0/1541807407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9cb4077b50 0x7f9cb419eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:23.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.367+0000 7f9cb1e00640 1 -- 172.21.15.53:0/1541807407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9cb4077b50 msgr2=0x7f9cb419eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:23.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.367+0000 7f9cb1e00640 1 --2- 172.21.15.53:0/1541807407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9cb4077b50 0x7f9cb419eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:23.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.367+0000 7f9cb1e00640 1 -- 172.21.15.53:0/1541807407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9cb40784f0 msgr2=0x7f9cb41a5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:23.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.367+0000 7f9cb1e00640 1 --2- 172.21.15.53:0/1541807407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9cb40784f0 0x7f9cb41a5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:23.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.367+0000 7f9cb1e00640 1 -- 172.21.15.53:0/1541807407 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb41a61f0 con 0x7f9cb4076910 2024-04-25T14:40:23.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.368+0000 7f9cb1e00640 1 --2- 172.21.15.53:0/1541807407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cb4076910 0x7f9cb419e5d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4005fd0 tx=0x7f9ca400ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:23.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.368+0000 7f9cab400640 1 -- 172.21.15.53:0/1541807407 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca400cd00 con 0x7f9cb4076910 2024-04-25T14:40:23.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.368+0000 7f9cab400640 1 -- 172.21.15.53:0/1541807407 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ca40035f0 con 0x7f9cb4076910 2024-04-25T14:40:23.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.368+0000 7f9cb3200640 1 -- 172.21.15.53:0/1541807407 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cb41a64e0 con 0x7f9cb4076910 2024-04-25T14:40:23.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.368+0000 7f9cab400640 1 -- 172.21.15.53:0/1541807407 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4010690 con 0x7f9cb4076910 2024-04-25T14:40:23.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.368+0000 7f9cb3200640 1 -- 172.21.15.53:0/1541807407 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cb41a6a00 con 0x7f9cb4076910 2024-04-25T14:40:23.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.369+0000 7f9cb3200640 1 -- 172.21.15.53:0/1541807407 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cb4110b70 con 0x7f9cb4076910 2024-04-25T14:40:23.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.372+0000 7f9cab400640 1 -- 172.21.15.53:0/1541807407 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9ca4003a80 con 0x7f9cb4076910 2024-04-25T14:40:23.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.373+0000 7f9cab400640 1 --2- 172.21.15.53:0/1541807407 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f9c980a3bb0 0x7f9c980a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:23.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.373+0000 7f9cab400640 1 -- 172.21.15.53:0/1541807407 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8285+0+0 (secure 0 0 0) 0x7f9ca40d0650 con 0x7f9cb4076910 2024-04-25T14:40:23.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.373+0000 7f9ca9400640 1 --2- 172.21.15.53:0/1541807407 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f9c980a3bb0 0x7f9c980a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:23.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.374+0000 7f9ca9400640 1 --2- 172.21.15.53:0/1541807407 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f9c980a3bb0 0x7f9c980a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c02a930 tx=0x7f9c9c030450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:23.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.377+0000 7f9cab400640 1 -- 172.21.15.53:0/1541807407 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9ca4009510 con 0x7f9cb4076910 2024-04-25T14:40:23.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.586+0000 7f9cb3200640 1 -- 172.21.15.53:0/1541807407 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9cb402d5c0 con 0x7f9cb4076910 2024-04-25T14:40:23.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.589+0000 7f9cab400640 1 -- 172.21.15.53:0/1541807407 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22732 (secure 0 0 0) 0x7f9ca409cfa0 con 0x7f9cb4076910 2024-04-25T14:40:23.589 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:40:23.589 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":92,"fsid":"fd30e6ce-030f-11ef-bc93-c7b262605968","created":"2024-04-25T14:28:54.330326+0000","modified":"2024-04-25T14:40:16.467090+0000","last_up_change":"2024-04-25T14:37:10.921148+0000","last_in_change":"2024-04-25T14:36:49.566459+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:33:02.338115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:38:07.517974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:38:08.994122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aecfbaf0-39da-4749-a49b-c7205dabd4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6803","nonce":464176015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6805","nonce":464176015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6809","nonce":464176015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6807","nonce":464176015}]},"public_addr":"172.21.15.53:6803/464176015","cluster_addr":"172.21.15.53:6805/464176015","heartbeat_back_addr":"172.21.15.53:6809/464176015","heartbeat_front_addr":"172.21.15.53:6807/464176015","state":["exists","up"]},{"osd":1,"uuid":"2c4a21ae-c43e-4839-8a56-c3fc63aeb7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6801","nonce":1304621982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6803","nonce":1304621982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6807","nonce":1304621982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6805","nonce":1304621982}]},"public_addr":"172.21.15.150:6801/1304621982","cluster_addr":"172.21.15.150:6803/1304621982","heartbeat_back_addr":"172.21.15.150:6807/1304621982","heartbeat_front_addr":"172.21.15.150:6805/1304621982","state":["exists","up"]},{"osd":2,"uuid":"e8effd43-adc0-472a-9eed-a9273b06dc0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6801","nonce":574509311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6803","nonce":574509311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6807","nonce":574509311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6805","nonce":574509311}]},"public_addr":"172.21.15.183:6801/574509311","cluster_addr":"172.21.15.183:6803/574509311","heartbeat_back_addr":"172.21.15.183:6807/574509311","heartbeat_front_addr":"172.21.15.183:6805/574509311","state":["exists","up"]},{"osd":3,"uuid":"e7cfced4-9b40-40de-a303-75ea8d5df4a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6811","nonce":570672219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6813","nonce":570672219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6817","nonce":570672219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6815","nonce":570672219}]},"public_addr":"172.21.15.53:6811/570672219","cluster_addr":"172.21.15.53:6813/570672219","heartbeat_back_addr":"172.21.15.53:6817/570672219","heartbeat_front_addr":"172.21.15.53:6815/570672219","state":["exists","up"]},{"osd":4,"uuid":"44a9e070-2282-43f1-9f05-b1f8ff81e540","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6809","nonce":1526850103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6811","nonce":1526850103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6815","nonce":1526850103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6813","nonce":1526850103}]},"public_addr":"172.21.15.150:6809/1526850103","cluster_addr":"172.21.15.150:6811/1526850103","heartbeat_back_addr":"172.21.15.150:6815/1526850103","heartbeat_front_addr":"172.21.15.150:6813/1526850103","state":["exists","up"]},{"osd":5,"uuid":"2c5619d2-c0f9-47d6-bf7e-2f4afd1b09f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6809","nonce":773726576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6811","nonce":773726576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6815","nonce":773726576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6813","nonce":773726576}]},"public_addr":"172.21.15.183:6809/773726576","cluster_addr":"172.21.15.183:6811/773726576","heartbeat_back_addr":"172.21.15.183:6815/773726576","heartbeat_front_addr":"172.21.15.183:6813/773726576","state":["exists","up"]},{"osd":6,"uuid":"1cf74d94-c118-4738-b6db-040af458a8eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6819","nonce":355884319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6821","nonce":355884319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6825","nonce":355884319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6823","nonce":355884319}]},"public_addr":"172.21.15.53:6819/355884319","cluster_addr":"172.21.15.53:6821/355884319","heartbeat_back_addr":"172.21.15.53:6825/355884319","heartbeat_front_addr":"172.21.15.53:6823/355884319","state":["exists","up"]},{"osd":7,"uuid":"0ce97c04-cd41-4080-bbbc-be9cf5fadc02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6817","nonce":229022562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6819","nonce":229022562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6823","nonce":229022562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6821","nonce":229022562}]},"public_addr":"172.21.15.150:6817/229022562","cluster_addr":"172.21.15.150:6819/229022562","heartbeat_back_addr":"172.21.15.150:6823/229022562","heartbeat_front_addr":"172.21.15.150:6821/229022562","state":["exists","up"]},{"osd":8,"uuid":"5d57787d-2a7c-4514-bd01-357ced413f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6817","nonce":301003844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6819","nonce":301003844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6823","nonce":301003844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6821","nonce":301003844}]},"public_addr":"172.21.15.183:6817/301003844","cluster_addr":"172.21.15.183:6819/301003844","heartbeat_back_addr":"172.21.15.183:6823/301003844","heartbeat_front_addr":"172.21.15.183:6821/301003844","state":["exists","up"]},{"osd":9,"uuid":"e11dd8eb-c55a-4d70-8c58-457540b7a50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6827","nonce":3449260978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6829","nonce":3449260978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6833","nonce":3449260978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6831","nonce":3449260978}]},"public_addr":"172.21.15.53:6827/3449260978","cluster_addr":"172.21.15.53:6829/3449260978","heartbeat_back_addr":"172.21.15.53:6833/3449260978","heartbeat_front_addr":"172.21.15.53:6831/3449260978","state":["exists","up"]},{"osd":10,"uuid":"5de4803d-ad37-4a90-add5-0048f44fc4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6825","nonce":3435992308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6827","nonce":3435992308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6831","nonce":3435992308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6829","nonce":3435992308}]},"public_addr":"172.21.15.150:6825/3435992308","cluster_addr":"172.21.15.150:6827/3435992308","heartbeat_back_addr":"172.21.15.150:6831/3435992308","heartbeat_front_addr":"172.21.15.150:6829/3435992308","state":["exists","up"]},{"osd":11,"uuid":"0e819a68-ed56-477e-b9c0-81e8cc0bce11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6825","nonce":1957842686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6827","nonce":1957842686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6831","nonce":1957842686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6829","nonce":1957842686}]},"public_addr":"172.21.15.183:6825/1957842686","cluster_addr":"172.21.15.183:6827/1957842686","heartbeat_back_addr":"172.21.15.183:6831/1957842686","heartbeat_front_addr":"172.21.15.183:6829/1957842686","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:09.840629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:34.769537+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:58.112969+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:33:29.656790+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:33:55.181287+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:34:19.772645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:34:53.833921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:35:19.076733+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:35:43.163390+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:36:16.568340+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:36:42.492768+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:37:07.944639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"3.38","mappings":[{"from":5,"to":7},{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6800/3887954889":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/1014501022":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/2576091441":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/1233527569":"2024-04-26T14:29:38.106242+0000","172.21.15.53:6801/3887954889":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/931889810":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/177196141":"2024-04-26T14:30:14.162383+0000","172.21.15.53:6800/1722497390":"2024-04-26T14:29:38.106242+0000","172.21.15.53:6801/1722497390":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/1927549198":"2024-04-26T14:30:14.162383+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:40:23.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.591+0000 7f9cb3200640 1 -- 172.21.15.53:0/1541807407 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f9c980a3bb0 msgr2=0x7f9c980a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:23.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.591+0000 7f9cb3200640 1 --2- 172.21.15.53:0/1541807407 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f9c980a3bb0 0x7f9c980a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c02a930 tx=0x7f9c9c030450 comp rx=0 tx=0).stop 2024-04-25T14:40:23.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.591+0000 7f9cb3200640 1 -- 172.21.15.53:0/1541807407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cb4076910 msgr2=0x7f9cb419e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:23.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.591+0000 7f9cb3200640 1 --2- 172.21.15.53:0/1541807407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cb4076910 0x7f9cb419e5d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4005fd0 tx=0x7f9ca400ee50 comp rx=0 tx=0).stop 2024-04-25T14:40:23.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.591+0000 7f9cb3200640 1 -- 172.21.15.53:0/1541807407 shutdown_connections 2024-04-25T14:40:23.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.591+0000 7f9cb3200640 1 --2- 172.21.15.53:0/1541807407 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f9c980a3bb0 0x7f9c980a6070 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:23.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.591+0000 7f9cb3200640 1 --2- 172.21.15.53:0/1541807407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9cb40784f0 0x7f9cb41a5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:23.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.591+0000 7f9cb3200640 1 --2- 172.21.15.53:0/1541807407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9cb4077b50 0x7f9cb419eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:23.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.591+0000 7f9cb3200640 1 --2- 172.21.15.53:0/1541807407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cb4076910 0x7f9cb419e5d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:23.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.591+0000 7f9cb3200640 1 -- 172.21.15.53:0/1541807407 >> 172.21.15.53:0/1541807407 conn(0x7f9cb40ff560 msgr2=0x7f9cb4100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:23.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.592+0000 7f9cb3200640 1 -- 172.21.15.53:0/1541807407 shutdown_connections 2024-04-25T14:40:23.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.592+0000 7f9cb3200640 1 -- 172.21.15.53:0/1541807407 wait complete. 2024-04-25T14:40:23.611 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:33:02.338115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:38:07.517974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:38:08.994122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:40:23.612 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:40:23.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:23 smithi053 ceph-mon[27773]: pgmap v386: 97 pgs: 97 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T14:40:23.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.822+0000 7fd017200640 1 -- 172.21.15.53:0/1874489667 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd018105280 msgr2=0x7fd018105700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:23.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.822+0000 7fd017200640 1 --2- 172.21.15.53:0/1874489667 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd018105280 0x7fd018105700 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd000009860 tx=0x7fd00002a340 comp rx=0 tx=0).stop 2024-04-25T14:40:23.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.822+0000 7fd017200640 1 -- 172.21.15.53:0/1874489667 shutdown_connections 2024-04-25T14:40:23.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.822+0000 7fd017200640 1 --2- 172.21.15.53:0/1874489667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd018105c40 0x7fd01810c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:23.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.822+0000 7fd017200640 1 --2- 172.21.15.53:0/1874489667 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd018105280 0x7fd018105700 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:23.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.822+0000 7fd017200640 1 --2- 172.21.15.53:0/1874489667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd018104090 0x7fd0181044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:23.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.822+0000 7fd017200640 1 -- 172.21.15.53:0/1874489667 >> 172.21.15.53:0/1874489667 conn(0x7fd0180ff600 msgr2=0x7fd018101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:23.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.823+0000 7fd017200640 1 -- 172.21.15.53:0/1874489667 shutdown_connections 2024-04-25T14:40:23.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.823+0000 7fd017200640 1 -- 172.21.15.53:0/1874489667 wait complete. 2024-04-25T14:40:23.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.823+0000 7fd017200640 1 Processor -- start 2024-04-25T14:40:23.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.824+0000 7fd017200640 1 -- start start 2024-04-25T14:40:23.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.824+0000 7fd017200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd018104090 0x7fd01819a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:23.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.824+0000 7fd017200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd018105280 0x7fd01819a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:23.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.824+0000 7fd017200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd018105c40 0x7fd0181a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:23.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.824+0000 7fd017200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd01810fbd0 con 0x7fd018105c40 2024-04-25T14:40:23.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.824+0000 7fd017200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd01810fa50 con 0x7fd018105280 2024-04-25T14:40:23.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.824+0000 7fd017200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd01810fd50 con 0x7fd018104090 2024-04-25T14:40:23.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.824+0000 7fd016800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd018105c40 0x7fd0181a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:23.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.824+0000 7fd015e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd018104090 0x7fd01819a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:23.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.824+0000 7fd016800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd018105c40 0x7fd0181a1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36842/0 (socket says 172.21.15.53:36842) 2024-04-25T14:40:23.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.824+0000 7fd016800640 1 -- 172.21.15.53:0/1112352685 learned_addr learned my addr 172.21.15.53:0/1112352685 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:23.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.824+0000 7fd00d400640 1 --2- 172.21.15.53:0/1112352685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd018105280 0x7fd01819a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:23.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.825+0000 7fd015e00640 1 -- 172.21.15.53:0/1112352685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd018105280 msgr2=0x7fd01819a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:23.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.825+0000 7fd015e00640 1 --2- 172.21.15.53:0/1112352685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd018105280 0x7fd01819a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:23.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.825+0000 7fd015e00640 1 -- 172.21.15.53:0/1112352685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd018105c40 msgr2=0x7fd0181a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:23.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.825+0000 7fd015e00640 1 --2- 172.21.15.53:0/1112352685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd018105c40 0x7fd0181a1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:23.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.825+0000 7fd015e00640 1 -- 172.21.15.53:0/1112352685 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0181a1e90 con 0x7fd018104090 2024-04-25T14:40:23.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.825+0000 7fd016800640 1 --2- 172.21.15.53:0/1112352685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd018105c40 0x7fd0181a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:40:23.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.825+0000 7fd015e00640 1 --2- 172.21.15.53:0/1112352685 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd018104090 0x7fd01819a2c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd0181050e0 tx=0x7fd00800ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:23.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.825+0000 7fd00f400640 1 -- 172.21.15.53:0/1112352685 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd008004440 con 0x7fd018104090 2024-04-25T14:40:23.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.825+0000 7fd017200640 1 -- 172.21.15.53:0/1112352685 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0181a2180 con 0x7fd018104090 2024-04-25T14:40:23.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.825+0000 7fd00f400640 1 -- 172.21.15.53:0/1112352685 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd00800cd70 con 0x7fd018104090 2024-04-25T14:40:23.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.825+0000 7fd017200640 1 -- 172.21.15.53:0/1112352685 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0181a26f0 con 0x7fd018104090 2024-04-25T14:40:23.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.826+0000 7fd00f400640 1 -- 172.21.15.53:0/1112352685 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd008019480 con 0x7fd018104090 2024-04-25T14:40:23.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.827+0000 7fd017200640 1 -- 172.21.15.53:0/1112352685 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd01810b3b0 con 0x7fd018104090 2024-04-25T14:40:23.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.829+0000 7fd00f400640 1 -- 172.21.15.53:0/1112352685 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd0080035f0 con 0x7fd018104090 2024-04-25T14:40:23.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.830+0000 7fd00f400640 1 --2- 172.21.15.53:0/1112352685 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fcffc0a3c30 0x7fcffc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:23.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.830+0000 7fd00f400640 1 -- 172.21.15.53:0/1112352685 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8285+0+0 (secure 0 0 0) 0x7fd0080d07b0 con 0x7fd018104090 2024-04-25T14:40:23.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.830+0000 7fd00d400640 1 --2- 172.21.15.53:0/1112352685 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fcffc0a3c30 0x7fcffc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:23.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.831+0000 7fd00d400640 1 --2- 172.21.15.53:0/1112352685 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fcffc0a3c30 0x7fcffc0a60f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd00002a930 tx=0x7fd000030450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:23.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:23.835+0000 7fd00f400640 1 -- 172.21.15.53:0/1112352685 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd008009510 con 0x7fd018104090 2024-04-25T14:40:23.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:23 smithi183 ceph-mon[32584]: pgmap v386: 97 pgs: 97 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T14:40:23.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:23 smithi150 ceph-mon[32504]: pgmap v386: 97 pgs: 97 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T14:40:24.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.041+0000 7fd017200640 1 -- 172.21.15.53:0/1112352685 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd018062b50 con 0x7fd018104090 2024-04-25T14:40:24.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.042+0000 7fd00f400640 1 -- 172.21.15.53:0/1112352685 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7fd00809d100 con 0x7fd018104090 2024-04-25T14:40:24.042 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2024-04-25T14:40:24.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.045+0000 7fd017200640 1 -- 172.21.15.53:0/1112352685 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fcffc0a3c30 msgr2=0x7fcffc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:24.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.045+0000 7fd017200640 1 --2- 172.21.15.53:0/1112352685 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fcffc0a3c30 0x7fcffc0a60f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd00002a930 tx=0x7fd000030450 comp rx=0 tx=0).stop 2024-04-25T14:40:24.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.045+0000 7fd017200640 1 -- 172.21.15.53:0/1112352685 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd018104090 msgr2=0x7fd01819a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:24.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.045+0000 7fd017200640 1 --2- 172.21.15.53:0/1112352685 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd018104090 0x7fd01819a2c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd0181050e0 tx=0x7fd00800ee50 comp rx=0 tx=0).stop 2024-04-25T14:40:24.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.045+0000 7fd017200640 1 -- 172.21.15.53:0/1112352685 shutdown_connections 2024-04-25T14:40:24.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.045+0000 7fd017200640 1 --2- 172.21.15.53:0/1112352685 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fcffc0a3c30 0x7fcffc0a60f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:24.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.045+0000 7fd017200640 1 --2- 172.21.15.53:0/1112352685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd018105c40 0x7fd0181a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:24.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.045+0000 7fd017200640 1 --2- 172.21.15.53:0/1112352685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd018105280 0x7fd01819a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:24.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.045+0000 7fd017200640 1 --2- 172.21.15.53:0/1112352685 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd018104090 0x7fd01819a2c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:24.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.045+0000 7fd017200640 1 -- 172.21.15.53:0/1112352685 >> 172.21.15.53:0/1112352685 conn(0x7fd0180ff600 msgr2=0x7fd018108580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:24.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.045+0000 7fd017200640 1 -- 172.21.15.53:0/1112352685 shutdown_connections 2024-04-25T14:40:24.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.045+0000 7fd017200640 1 -- 172.21.15.53:0/1112352685 wait complete. 2024-04-25T14:40:24.065 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:40:24.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.280+0000 7f9973200640 1 -- 172.21.15.53:0/2166902714 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f996c109680 msgr2=0x7f996c109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:24.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.280+0000 7f9973200640 1 --2- 172.21.15.53:0/2166902714 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f996c109680 0x7f996c109b00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9960009900 tx=0x7f996002a3f0 comp rx=0 tx=0).stop 2024-04-25T14:40:24.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.281+0000 7f9973200640 1 -- 172.21.15.53:0/2166902714 shutdown_connections 2024-04-25T14:40:24.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.281+0000 7f9973200640 1 --2- 172.21.15.53:0/2166902714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f996c10a040 0x7f996c1108d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:24.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.281+0000 7f9973200640 1 --2- 172.21.15.53:0/2166902714 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f996c109680 0x7f996c109b00 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:24.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.281+0000 7f9973200640 1 --2- 172.21.15.53:0/2166902714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f996c108490 0x7f996c1088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:24.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.281+0000 7f9973200640 1 -- 172.21.15.53:0/2166902714 >> 172.21.15.53:0/2166902714 conn(0x7f996c077420 msgr2=0x7f996c079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:24.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.281+0000 7f9973200640 1 -- 172.21.15.53:0/2166902714 shutdown_connections 2024-04-25T14:40:24.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.281+0000 7f9973200640 1 -- 172.21.15.53:0/2166902714 wait complete. 2024-04-25T14:40:24.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.282+0000 7f9973200640 1 Processor -- start 2024-04-25T14:40:24.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.282+0000 7f9973200640 1 -- start start 2024-04-25T14:40:24.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.282+0000 7f9973200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f996c108490 0x7f996c19e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:24.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.283+0000 7f9973200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f996c109680 0x7f996c19ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:24.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.283+0000 7f9973200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f996c10a040 0x7f996c1a5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:24.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.283+0000 7f9973200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f996c113fd0 con 0x7f996c109680 2024-04-25T14:40:24.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.283+0000 7f9973200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f996c113e50 con 0x7f996c10a040 2024-04-25T14:40:24.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.283+0000 7f9973200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f996c114150 con 0x7f996c108490 2024-04-25T14:40:24.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.283+0000 7f9971e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f996c108490 0x7f996c19e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:24.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.283+0000 7f9971e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f996c108490 0x7f996c19e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:44938/0 (socket says 172.21.15.53:44938) 2024-04-25T14:40:24.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.283+0000 7f9971e00640 1 -- 172.21.15.53:0/237180917 learned_addr learned my addr 172.21.15.53:0/237180917 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:24.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.283+0000 7f9971400640 1 --2- 172.21.15.53:0/237180917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f996c109680 0x7f996c19ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:24.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.283+0000 7f9972800640 1 --2- 172.21.15.53:0/237180917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f996c10a040 0x7f996c1a5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:24.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.283+0000 7f9971e00640 1 -- 172.21.15.53:0/237180917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f996c10a040 msgr2=0x7f996c1a5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:24.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.283+0000 7f9971e00640 1 --2- 172.21.15.53:0/237180917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f996c10a040 0x7f996c1a5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:24.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.283+0000 7f9971e00640 1 -- 172.21.15.53:0/237180917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f996c109680 msgr2=0x7f996c19ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:24.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.283+0000 7f9971e00640 1 --2- 172.21.15.53:0/237180917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f996c109680 0x7f996c19ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:24.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.283+0000 7f9971e00640 1 -- 172.21.15.53:0/237180917 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f996c1a62d0 con 0x7f996c108490 2024-04-25T14:40:24.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.283+0000 7f9971400640 1 --2- 172.21.15.53:0/237180917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f996c109680 0x7f996c19ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:40:24.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.284+0000 7f9971e00640 1 --2- 172.21.15.53:0/237180917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f996c108490 0x7f996c19e6b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f995c00e9c0 tx=0x7f995c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:24.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.284+0000 7f9966a00640 1 -- 172.21.15.53:0/237180917 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c00cd80 con 0x7f996c108490 2024-04-25T14:40:24.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.284+0000 7f9966a00640 1 -- 172.21.15.53:0/237180917 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f995c0035f0 con 0x7f996c108490 2024-04-25T14:40:24.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.284+0000 7f9966a00640 1 -- 172.21.15.53:0/237180917 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c010690 con 0x7f996c108490 2024-04-25T14:40:24.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.284+0000 7f9973200640 1 -- 172.21.15.53:0/237180917 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f996c1a65c0 con 0x7f996c108490 2024-04-25T14:40:24.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.284+0000 7f9973200640 1 -- 172.21.15.53:0/237180917 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f996c1a6ac0 con 0x7f996c108490 2024-04-25T14:40:24.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.285+0000 7f9973200640 1 -- 172.21.15.53:0/237180917 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f996c10f8d0 con 0x7f996c108490 2024-04-25T14:40:24.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.288+0000 7f9966a00640 1 -- 172.21.15.53:0/237180917 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f995c004000 con 0x7f996c108490 2024-04-25T14:40:24.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.290+0000 7f9966a00640 1 --2- 172.21.15.53:0/237180917 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f99540a3bb0 0x7f99540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:24.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.290+0000 7f9966a00640 1 -- 172.21.15.53:0/237180917 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8285+0+0 (secure 0 0 0) 0x7f995c0d0c70 con 0x7f996c108490 2024-04-25T14:40:24.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.290+0000 7f9971400640 1 --2- 172.21.15.53:0/237180917 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f99540a3bb0 0x7f99540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:24.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.291+0000 7f9971400640 1 --2- 172.21.15.53:0/237180917 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f99540a3bb0 0x7f99540a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f996c19fbc0 tx=0x7f99600314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:24.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.291+0000 7f9966a00640 1 -- 172.21.15.53:0/237180917 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f995c09d5c0 con 0x7f996c108490 2024-04-25T14:40:24.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.490+0000 7f9973200640 1 -- 172.21.15.53:0/237180917 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f996c062b50 con 0x7f996c108490 2024-04-25T14:40:24.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.491+0000 7f9966a00640 1 -- 172.21.15.53:0/237180917 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f995c09cec0 con 0x7f996c108490 2024-04-25T14:40:24.490 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2024-04-25T14:40:24.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.493+0000 7f9973200640 1 -- 172.21.15.53:0/237180917 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f99540a3bb0 msgr2=0x7f99540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:24.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.493+0000 7f9973200640 1 --2- 172.21.15.53:0/237180917 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f99540a3bb0 0x7f99540a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f996c19fbc0 tx=0x7f99600314a0 comp rx=0 tx=0).stop 2024-04-25T14:40:24.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.493+0000 7f9973200640 1 -- 172.21.15.53:0/237180917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f996c108490 msgr2=0x7f996c19e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:24.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.493+0000 7f9973200640 1 --2- 172.21.15.53:0/237180917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f996c108490 0x7f996c19e6b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f995c00e9c0 tx=0x7f995c00ee80 comp rx=0 tx=0).stop 2024-04-25T14:40:24.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.493+0000 7f9973200640 1 -- 172.21.15.53:0/237180917 shutdown_connections 2024-04-25T14:40:24.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.493+0000 7f9973200640 1 --2- 172.21.15.53:0/237180917 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f99540a3bb0 0x7f99540a6070 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:24.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.493+0000 7f9973200640 1 --2- 172.21.15.53:0/237180917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f996c10a040 0x7f996c1a5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:24.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.493+0000 7f9973200640 1 --2- 172.21.15.53:0/237180917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f996c109680 0x7f996c19ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:24.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.493+0000 7f9973200640 1 --2- 172.21.15.53:0/237180917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f996c108490 0x7f996c19e6b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:24.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.493+0000 7f9973200640 1 -- 172.21.15.53:0/237180917 >> 172.21.15.53:0/237180917 conn(0x7f996c077420 msgr2=0x7f996c10caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:24.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.493+0000 7f9973200640 1 -- 172.21.15.53:0/237180917 shutdown_connections 2024-04-25T14:40:24.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.493+0000 7f9973200640 1 -- 172.21.15.53:0/237180917 wait complete. 2024-04-25T14:40:24.511 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:40:24.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.721+0000 7fdf0be00640 1 -- 172.21.15.53:0/3263907537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf0c105c40 msgr2=0x7fdf0c10c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:24.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.721+0000 7fdf0be00640 1 --2- 172.21.15.53:0/3263907537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf0c105c40 0x7fdf0c10c4d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdf04009ca0 tx=0x7fdf0402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:40:24.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.721+0000 7fdf0be00640 1 -- 172.21.15.53:0/3263907537 shutdown_connections 2024-04-25T14:40:24.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.721+0000 7fdf0be00640 1 --2- 172.21.15.53:0/3263907537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf0c105c40 0x7fdf0c10c4d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:24.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.721+0000 7fdf0be00640 1 --2- 172.21.15.53:0/3263907537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf0c105280 0x7fdf0c105700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:24.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.721+0000 7fdf0be00640 1 --2- 172.21.15.53:0/3263907537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf0c104090 0x7fdf0c1044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:24.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.721+0000 7fdf0be00640 1 -- 172.21.15.53:0/3263907537 >> 172.21.15.53:0/3263907537 conn(0x7fdf0c0ff600 msgr2=0x7fdf0c101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:24.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.721+0000 7fdf0be00640 1 -- 172.21.15.53:0/3263907537 shutdown_connections 2024-04-25T14:40:24.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.721+0000 7fdf0be00640 1 -- 172.21.15.53:0/3263907537 wait complete. 2024-04-25T14:40:24.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.722+0000 7fdf0be00640 1 Processor -- start 2024-04-25T14:40:24.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.722+0000 7fdf0be00640 1 -- start start 2024-04-25T14:40:24.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.723+0000 7fdf0be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf0c104090 0x7fdf0c19a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:24.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.723+0000 7fdf0be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf0c105280 0x7fdf0c19a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:24.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.723+0000 7fdf0be00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf0c105c40 0x7fdf0c1a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:24.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.723+0000 7fdf0be00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0c10fbd0 con 0x7fdf0c104090 2024-04-25T14:40:24.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.723+0000 7fdf0be00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0c10fa50 con 0x7fdf0c105c40 2024-04-25T14:40:24.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.723+0000 7fdf0be00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0c10fd50 con 0x7fdf0c105280 2024-04-25T14:40:24.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.723+0000 7fdf0aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf0c104090 0x7fdf0c19a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:24.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.723+0000 7fdf0aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf0c104090 0x7fdf0c19a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36882/0 (socket says 172.21.15.53:36882) 2024-04-25T14:40:24.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.723+0000 7fdf0aa00640 1 -- 172.21.15.53:0/3861226700 learned_addr learned my addr 172.21.15.53:0/3861226700 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:24.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.723+0000 7fdf0a000640 1 --2- 172.21.15.53:0/3861226700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf0c105280 0x7fdf0c19a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:24.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.723+0000 7fdf0b400640 1 --2- 172.21.15.53:0/3861226700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf0c105c40 0x7fdf0c1a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:24.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.723+0000 7fdf0aa00640 1 -- 172.21.15.53:0/3861226700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf0c105280 msgr2=0x7fdf0c19a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:24.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.723+0000 7fdf0aa00640 1 --2- 172.21.15.53:0/3861226700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf0c105280 0x7fdf0c19a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:24.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.723+0000 7fdf0aa00640 1 -- 172.21.15.53:0/3861226700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf0c105c40 msgr2=0x7fdf0c1a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:24.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.723+0000 7fdf0aa00640 1 --2- 172.21.15.53:0/3861226700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf0c105c40 0x7fdf0c1a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:24.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.723+0000 7fdf0aa00640 1 -- 172.21.15.53:0/3861226700 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf0c1a1ea0 con 0x7fdf0c104090 2024-04-25T14:40:24.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.723+0000 7fdf0b400640 1 --2- 172.21.15.53:0/3861226700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf0c105c40 0x7fdf0c1a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:40:24.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.724+0000 7fdf0aa00640 1 --2- 172.21.15.53:0/3861226700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf0c104090 0x7fdf0c19a280 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdefc00e9c0 tx=0x7fdefc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:24.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.724+0000 7fdf03400640 1 -- 172.21.15.53:0/3861226700 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdefc00cde0 con 0x7fdf0c104090 2024-04-25T14:40:24.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.724+0000 7fdf03400640 1 -- 172.21.15.53:0/3861226700 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdefc0108e0 con 0x7fdf0c104090 2024-04-25T14:40:24.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.724+0000 7fdf0be00640 1 -- 172.21.15.53:0/3861226700 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf0c1a2190 con 0x7fdf0c104090 2024-04-25T14:40:24.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.724+0000 7fdf03400640 1 -- 172.21.15.53:0/3861226700 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdefc018960 con 0x7fdf0c104090 2024-04-25T14:40:24.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.724+0000 7fdf0be00640 1 -- 172.21.15.53:0/3861226700 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf0c1a2730 con 0x7fdf0c104090 2024-04-25T14:40:24.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.725+0000 7fdf0be00640 1 -- 172.21.15.53:0/3861226700 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf0c10b400 con 0x7fdf0c104090 2024-04-25T14:40:24.727 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.727+0000 7fdf03400640 1 -- 172.21.15.53:0/3861226700 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdefc01f020 con 0x7fdf0c104090 2024-04-25T14:40:24.728 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.729+0000 7fdf03400640 1 --2- 172.21.15.53:0/3861226700 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fdef00a3c30 0x7fdef00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:24.728 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.729+0000 7fdf03400640 1 -- 172.21.15.53:0/3861226700 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8285+0+0 (secure 0 0 0) 0x7fdefc0d0710 con 0x7fdf0c104090 2024-04-25T14:40:24.729 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.729+0000 7fdf0a000640 1 --2- 172.21.15.53:0/3861226700 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fdef00a3c30 0x7fdef00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:24.729 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.730+0000 7fdf0a000640 1 --2- 172.21.15.53:0/3861226700 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fdef00a3c30 0x7fdef00a60f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c19b790 tx=0x7fdef8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:24.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.734+0000 7fdf03400640 1 -- 172.21.15.53:0/3861226700 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdefc053ff0 con 0x7fdf0c104090 2024-04-25T14:40:24.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:24 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1541807407' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:40:24.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:24 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1112352685' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:40:24.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:24 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/237180917' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:40:24.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.944+0000 7fdf0be00640 1 -- 172.21.15.53:0/3861226700 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdf0c062b50 con 0x7fdf0c104090 2024-04-25T14:40:24.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.945+0000 7fdf03400640 1 -- 172.21.15.53:0/3861226700 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7fdefc09d060 con 0x7fdf0c104090 2024-04-25T14:40:24.944 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2024-04-25T14:40:24.947 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.947+0000 7fdf0be00640 1 -- 172.21.15.53:0/3861226700 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fdef00a3c30 msgr2=0x7fdef00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:24.947 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.947+0000 7fdf0be00640 1 --2- 172.21.15.53:0/3861226700 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fdef00a3c30 0x7fdef00a60f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c19b790 tx=0x7fdef8005f90 comp rx=0 tx=0).stop 2024-04-25T14:40:24.947 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.947+0000 7fdf0be00640 1 -- 172.21.15.53:0/3861226700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf0c104090 msgr2=0x7fdf0c19a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:24.947 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.947+0000 7fdf0be00640 1 --2- 172.21.15.53:0/3861226700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf0c104090 0x7fdf0c19a280 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdefc00e9c0 tx=0x7fdefc00ee80 comp rx=0 tx=0).stop 2024-04-25T14:40:24.948 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.948+0000 7fdf0be00640 1 -- 172.21.15.53:0/3861226700 shutdown_connections 2024-04-25T14:40:24.948 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.948+0000 7fdf0be00640 1 --2- 172.21.15.53:0/3861226700 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fdef00a3c30 0x7fdef00a60f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:24.948 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.948+0000 7fdf0be00640 1 --2- 172.21.15.53:0/3861226700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf0c105c40 0x7fdf0c1a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:24.948 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.948+0000 7fdf0be00640 1 --2- 172.21.15.53:0/3861226700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf0c105280 0x7fdf0c19a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:24.948 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.948+0000 7fdf0be00640 1 --2- 172.21.15.53:0/3861226700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf0c104090 0x7fdf0c19a280 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:24.948 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.948+0000 7fdf0be00640 1 -- 172.21.15.53:0/3861226700 >> 172.21.15.53:0/3861226700 conn(0x7fdf0c0ff600 msgr2=0x7fdf0c1085d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:24.948 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.948+0000 7fdf0be00640 1 -- 172.21.15.53:0/3861226700 shutdown_connections 2024-04-25T14:40:24.948 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:24.948+0000 7fdf0be00640 1 -- 172.21.15.53:0/3861226700 wait complete. 2024-04-25T14:40:24.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:24 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/1541807407' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:40:24.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:24 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/1112352685' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:40:24.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:24 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/237180917' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:40:24.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:24 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/1541807407' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:40:24.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:24 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/1112352685' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:40:24.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:24 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/237180917' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:40:24.967 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:40:25.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.174+0000 7ff296a00640 1 -- 172.21.15.53:0/718534923 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2901095c0 msgr2=0x7ff290109a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:25.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.174+0000 7ff296a00640 1 --2- 172.21.15.53:0/718534923 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2901095c0 0x7ff290109a40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff27c009900 tx=0x7ff27c02a3f0 comp rx=0 tx=0).stop 2024-04-25T14:40:25.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.174+0000 7ff296a00640 1 -- 172.21.15.53:0/718534923 shutdown_connections 2024-04-25T14:40:25.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.174+0000 7ff296a00640 1 --2- 172.21.15.53:0/718534923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff290109f80 0x7ff290110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:25.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.174+0000 7ff296a00640 1 --2- 172.21.15.53:0/718534923 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2901095c0 0x7ff290109a40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:25.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.174+0000 7ff296a00640 1 --2- 172.21.15.53:0/718534923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2901083d0 0x7ff2901087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:25.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.174+0000 7ff296a00640 1 -- 172.21.15.53:0/718534923 >> 172.21.15.53:0/718534923 conn(0x7ff290077390 msgr2=0x7ff2900797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:25.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.174+0000 7ff296a00640 1 -- 172.21.15.53:0/718534923 shutdown_connections 2024-04-25T14:40:25.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.174+0000 7ff296a00640 1 -- 172.21.15.53:0/718534923 wait complete. 2024-04-25T14:40:25.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.176+0000 7ff296a00640 1 Processor -- start 2024-04-25T14:40:25.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.176+0000 7ff296a00640 1 -- start start 2024-04-25T14:40:25.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.176+0000 7ff296a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2901083d0 0x7ff29019e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:25.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.176+0000 7ff296a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2901095c0 0x7ff29019ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:25.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.176+0000 7ff296a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff290109f80 0x7ff2901a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:25.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.176+0000 7ff296a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff290113f10 con 0x7ff2901095c0 2024-04-25T14:40:25.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.176+0000 7ff296a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff290113d90 con 0x7ff290109f80 2024-04-25T14:40:25.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.176+0000 7ff296a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff290114090 con 0x7ff2901083d0 2024-04-25T14:40:25.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.176+0000 7ff294c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2901095c0 0x7ff29019ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:25.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.176+0000 7ff294c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2901095c0 0x7ff29019ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36896/0 (socket says 172.21.15.53:36896) 2024-04-25T14:40:25.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.176+0000 7ff294c00640 1 -- 172.21.15.53:0/3201012820 learned_addr learned my addr 172.21.15.53:0/3201012820 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:25.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.177+0000 7ff295600640 1 --2- 172.21.15.53:0/3201012820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2901083d0 0x7ff29019e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:25.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.177+0000 7ff296000640 1 --2- 172.21.15.53:0/3201012820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff290109f80 0x7ff2901a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:25.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.177+0000 7ff295600640 1 -- 172.21.15.53:0/3201012820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff290109f80 msgr2=0x7ff2901a5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:25.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.177+0000 7ff295600640 1 --2- 172.21.15.53:0/3201012820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff290109f80 0x7ff2901a5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:25.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.177+0000 7ff295600640 1 -- 172.21.15.53:0/3201012820 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2901095c0 msgr2=0x7ff29019ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:25.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.177+0000 7ff295600640 1 --2- 172.21.15.53:0/3201012820 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2901095c0 0x7ff29019ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:25.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.177+0000 7ff295600640 1 -- 172.21.15.53:0/3201012820 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2901a61f0 con 0x7ff2901083d0 2024-04-25T14:40:25.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.177+0000 7ff294c00640 1 --2- 172.21.15.53:0/3201012820 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2901095c0 0x7ff29019ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:40:25.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.177+0000 7ff295600640 1 --2- 172.21.15.53:0/3201012820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2901083d0 0x7ff29019e670 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff28400e9c0 tx=0x7ff28400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:25.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.178+0000 7ff28e000640 1 -- 172.21.15.53:0/3201012820 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28400ccc0 con 0x7ff2901083d0 2024-04-25T14:40:25.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.178+0000 7ff28e000640 1 -- 172.21.15.53:0/3201012820 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2840035f0 con 0x7ff2901083d0 2024-04-25T14:40:25.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.178+0000 7ff296a00640 1 -- 172.21.15.53:0/3201012820 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2901a64e0 con 0x7ff2901083d0 2024-04-25T14:40:25.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.178+0000 7ff28e000640 1 -- 172.21.15.53:0/3201012820 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28400ce60 con 0x7ff2901083d0 2024-04-25T14:40:25.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.178+0000 7ff296a00640 1 -- 172.21.15.53:0/3201012820 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2901a6a00 con 0x7ff2901083d0 2024-04-25T14:40:25.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.179+0000 7ff296a00640 1 -- 172.21.15.53:0/3201012820 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff29010f740 con 0x7ff2901083d0 2024-04-25T14:40:25.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.181+0000 7ff28e000640 1 -- 172.21.15.53:0/3201012820 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff284004000 con 0x7ff2901083d0 2024-04-25T14:40:25.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.182+0000 7ff28e000640 1 --2- 172.21.15.53:0/3201012820 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff2780a3bb0 0x7ff2780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:25.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.182+0000 7ff28e000640 1 -- 172.21.15.53:0/3201012820 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8285+0+0 (secure 0 0 0) 0x7ff2840d09a0 con 0x7ff2901083d0 2024-04-25T14:40:25.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.183+0000 7ff294c00640 1 --2- 172.21.15.53:0/3201012820 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff2780a3bb0 0x7ff2780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:25.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.183+0000 7ff294c00640 1 --2- 172.21.15.53:0/3201012820 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff2780a3bb0 0x7ff2780a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff27c009230 tx=0x7ff27c0058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:25.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.185+0000 7ff28e000640 1 -- 172.21.15.53:0/3201012820 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff28409d380 con 0x7ff2901083d0 2024-04-25T14:40:25.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.425+0000 7ff296a00640 1 -- 172.21.15.53:0/3201012820 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff29002d5c0 con 0x7ff2901083d0 2024-04-25T14:40:25.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.426+0000 7ff28e000640 1 -- 172.21.15.53:0/3201012820 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11862 (secure 0 0 0) 0x7ff28409cc80 con 0x7ff2901083d0 2024-04-25T14:40:25.427 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:40:25.427 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14460,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6841/41110727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6840","nonce":41110727},{"type":"v1","addr":"172.21.15.53:6841","nonce":41110727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2966992472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2966992472},{"type":"v1","addr":"172.21.15.150:6833","nonce":2966992472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/94143643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":94143643},{"type":"v1","addr":"172.21.15.183:6833","nonce":94143643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24470,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/506682943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":506682943},{"type":"v1","addr":"172.21.15.53:6835","nonce":506682943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1369488844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1369488844},{"type":"v1","addr":"172.21.15.150:6835","nonce":1369488844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3786232682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3786232682},{"type":"v1","addr":"172.21.15.53:6837","nonce":3786232682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T14:38:10.037451+0000","modified":"2024-04-25T14:38:15.629854+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24584,"mds_1":24565,"mds_2":24560},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24532":{"gid":24532,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1189328602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1189328602},{"type":"v1","addr":"172.21.15.183:6835","nonce":1189328602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/2344343627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2344343627},{"type":"v1","addr":"172.21.15.150:6837","nonce":2344343627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24544":{"gid":24544,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1163336384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1163336384},{"type":"v1","addr":"172.21.15.183:6837","nonce":1163336384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6839/570000540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":570000540},{"type":"v1","addr":"172.21.15.53:6839","nonce":570000540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.183:6839/1554258915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1554258915},{"type":"v1","addr":"172.21.15.183:6839","nonce":1554258915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/149573616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":149573616},{"type":"v1","addr":"172.21.15.150:6839","nonce":149573616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24565,24584,24560]},"id":1}]} 2024-04-25T14:40:25.427 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 21 2024-04-25T14:40:25.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.429+0000 7ff296a00640 1 -- 172.21.15.53:0/3201012820 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff2780a3bb0 msgr2=0x7ff2780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:25.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.429+0000 7ff296a00640 1 --2- 172.21.15.53:0/3201012820 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff2780a3bb0 0x7ff2780a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff27c009230 tx=0x7ff27c0058d0 comp rx=0 tx=0).stop 2024-04-25T14:40:25.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.429+0000 7ff296a00640 1 -- 172.21.15.53:0/3201012820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2901083d0 msgr2=0x7ff29019e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:25.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.429+0000 7ff296a00640 1 --2- 172.21.15.53:0/3201012820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2901083d0 0x7ff29019e670 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff28400e9c0 tx=0x7ff28400ee80 comp rx=0 tx=0).stop 2024-04-25T14:40:25.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.429+0000 7ff296a00640 1 -- 172.21.15.53:0/3201012820 shutdown_connections 2024-04-25T14:40:25.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.429+0000 7ff296a00640 1 --2- 172.21.15.53:0/3201012820 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff2780a3bb0 0x7ff2780a6070 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:25.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.429+0000 7ff296a00640 1 --2- 172.21.15.53:0/3201012820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff290109f80 0x7ff2901a5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:25.430 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.429+0000 7ff296a00640 1 --2- 172.21.15.53:0/3201012820 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2901095c0 0x7ff29019ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:25.430 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.429+0000 7ff296a00640 1 --2- 172.21.15.53:0/3201012820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2901083d0 0x7ff29019e670 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:25.430 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.429+0000 7ff296a00640 1 -- 172.21.15.53:0/3201012820 >> 172.21.15.53:0/3201012820 conn(0x7ff290077390 msgr2=0x7ff29010c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:25.430 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.429+0000 7ff296a00640 1 -- 172.21.15.53:0/3201012820 shutdown_connections 2024-04-25T14:40:25.430 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.429+0000 7ff296a00640 1 -- 172.21.15.53:0/3201012820 wait complete. 2024-04-25T14:40:25.449 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:40:25.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.658+0000 7ff489c00640 1 -- 172.21.15.53:0/656429759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4841095c0 msgr2=0x7ff484109a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:25.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.658+0000 7ff489c00640 1 --2- 172.21.15.53:0/656429759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4841095c0 0x7ff484109a40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff46c009900 tx=0x7ff46c02a400 comp rx=0 tx=0).stop 2024-04-25T14:40:25.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.658+0000 7ff489c00640 1 -- 172.21.15.53:0/656429759 shutdown_connections 2024-04-25T14:40:25.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.658+0000 7ff489c00640 1 --2- 172.21.15.53:0/656429759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff484109f80 0x7ff484110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:25.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.658+0000 7ff489c00640 1 --2- 172.21.15.53:0/656429759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4841095c0 0x7ff484109a40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:25.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.659+0000 7ff489c00640 1 --2- 172.21.15.53:0/656429759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4841083d0 0x7ff4841087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:25.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.659+0000 7ff489c00640 1 -- 172.21.15.53:0/656429759 >> 172.21.15.53:0/656429759 conn(0x7ff484077390 msgr2=0x7ff4840797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:25.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.659+0000 7ff489c00640 1 -- 172.21.15.53:0/656429759 shutdown_connections 2024-04-25T14:40:25.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.659+0000 7ff489c00640 1 -- 172.21.15.53:0/656429759 wait complete. 2024-04-25T14:40:25.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.660+0000 7ff489c00640 1 Processor -- start 2024-04-25T14:40:25.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.660+0000 7ff489c00640 1 -- start start 2024-04-25T14:40:25.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.660+0000 7ff489c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4841083d0 0x7ff48419e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:25.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.660+0000 7ff489c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4841095c0 0x7ff48419eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:25.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.661+0000 7ff489c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff484109f80 0x7ff4841a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:25.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.661+0000 7ff489c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff484113f10 con 0x7ff4841095c0 2024-04-25T14:40:25.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.661+0000 7ff489c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff484113d90 con 0x7ff4841083d0 2024-04-25T14:40:25.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.661+0000 7ff489c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff484114090 con 0x7ff484109f80 2024-04-25T14:40:25.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.661+0000 7ff483400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4841095c0 0x7ff48419eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:25.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.661+0000 7ff483400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4841095c0 0x7ff48419eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36916/0 (socket says 172.21.15.53:36916) 2024-04-25T14:40:25.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.661+0000 7ff483400640 1 -- 172.21.15.53:0/1556354036 learned_addr learned my addr 172.21.15.53:0/1556354036 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:25.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.661+0000 7ff483e00640 1 --2- 172.21.15.53:0/1556354036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4841083d0 0x7ff48419e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:25.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.661+0000 7ff489200640 1 --2- 172.21.15.53:0/1556354036 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff484109f80 0x7ff4841a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:25.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.661+0000 7ff483400640 1 -- 172.21.15.53:0/1556354036 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff484109f80 msgr2=0x7ff4841a5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:25.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.661+0000 7ff483400640 1 --2- 172.21.15.53:0/1556354036 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff484109f80 0x7ff4841a5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:25.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.661+0000 7ff483400640 1 -- 172.21.15.53:0/1556354036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4841083d0 msgr2=0x7ff48419e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:25.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.661+0000 7ff483400640 1 --2- 172.21.15.53:0/1556354036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4841083d0 0x7ff48419e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:25.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.661+0000 7ff483400640 1 -- 172.21.15.53:0/1556354036 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4841a61d0 con 0x7ff4841095c0 2024-04-25T14:40:25.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.661+0000 7ff483e00640 1 --2- 172.21.15.53:0/1556354036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4841083d0 0x7ff48419e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:40:25.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.662+0000 7ff483400640 1 --2- 172.21.15.53:0/1556354036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4841095c0 0x7ff48419eaf0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff46c005940 tx=0x7ff46c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:25.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.662+0000 7ff480c00640 1 -- 172.21.15.53:0/1556354036 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46c03f030 con 0x7ff4841095c0 2024-04-25T14:40:25.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.662+0000 7ff480c00640 1 -- 172.21.15.53:0/1556354036 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff46c003cb0 con 0x7ff4841095c0 2024-04-25T14:40:25.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.662+0000 7ff489c00640 1 -- 172.21.15.53:0/1556354036 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4841a6460 con 0x7ff4841095c0 2024-04-25T14:40:25.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.662+0000 7ff480c00640 1 -- 172.21.15.53:0/1556354036 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46c035070 con 0x7ff4841095c0 2024-04-25T14:40:25.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.662+0000 7ff489c00640 1 -- 172.21.15.53:0/1556354036 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4841a6950 con 0x7ff4841095c0 2024-04-25T14:40:25.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.663+0000 7ff489c00640 1 -- 172.21.15.53:0/1556354036 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff48410f770 con 0x7ff4841095c0 2024-04-25T14:40:25.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.665+0000 7ff480c00640 1 -- 172.21.15.53:0/1556354036 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff46c042030 con 0x7ff4841095c0 2024-04-25T14:40:25.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.666+0000 7ff480c00640 1 --2- 172.21.15.53:0/1556354036 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff4680a3bb0 0x7ff4680a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:25.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.667+0000 7ff480c00640 1 -- 172.21.15.53:0/1556354036 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8285+0+0 (secure 0 0 0) 0x7ff46c0f0840 con 0x7ff4841095c0 2024-04-25T14:40:25.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.667+0000 7ff483e00640 1 --2- 172.21.15.53:0/1556354036 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff4680a3bb0 0x7ff4680a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:25.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.667+0000 7ff483e00640 1 --2- 172.21.15.53:0/1556354036 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff4680a3bb0 0x7ff4680a6070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff484109420 tx=0x7ff474005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:25.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.671+0000 7ff480c00640 1 -- 172.21.15.53:0/1556354036 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff46c003e60 con 0x7ff4841095c0 2024-04-25T14:40:25.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:25 smithi053 ceph-mon[27773]: pgmap v387: 97 pgs: 97 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s 2024-04-25T14:40:25.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:25 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/3861226700' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:40:25.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:25 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/3201012820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:40:25.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.882+0000 7ff489c00640 1 -- 172.21.15.53:0/1556354036 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff484062b50 con 0x7ff4841095c0 2024-04-25T14:40:25.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.884+0000 7ff480c00640 1 -- 172.21.15.53:0/1556354036 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22732 (secure 0 0 0) 0x7ff46c0bcb40 con 0x7ff4841095c0 2024-04-25T14:40:25.884 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:40:25.885 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":92,"fsid":"fd30e6ce-030f-11ef-bc93-c7b262605968","created":"2024-04-25T14:28:54.330326+0000","modified":"2024-04-25T14:40:16.467090+0000","last_up_change":"2024-04-25T14:37:10.921148+0000","last_in_change":"2024-04-25T14:36:49.566459+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:33:02.338115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:38:07.517974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:38:08.994122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aecfbaf0-39da-4749-a49b-c7205dabd4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6803","nonce":464176015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6805","nonce":464176015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6809","nonce":464176015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6807","nonce":464176015}]},"public_addr":"172.21.15.53:6803/464176015","cluster_addr":"172.21.15.53:6805/464176015","heartbeat_back_addr":"172.21.15.53:6809/464176015","heartbeat_front_addr":"172.21.15.53:6807/464176015","state":["exists","up"]},{"osd":1,"uuid":"2c4a21ae-c43e-4839-8a56-c3fc63aeb7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6801","nonce":1304621982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6803","nonce":1304621982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6807","nonce":1304621982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6805","nonce":1304621982}]},"public_addr":"172.21.15.150:6801/1304621982","cluster_addr":"172.21.15.150:6803/1304621982","heartbeat_back_addr":"172.21.15.150:6807/1304621982","heartbeat_front_addr":"172.21.15.150:6805/1304621982","state":["exists","up"]},{"osd":2,"uuid":"e8effd43-adc0-472a-9eed-a9273b06dc0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6801","nonce":574509311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6803","nonce":574509311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6807","nonce":574509311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6805","nonce":574509311}]},"public_addr":"172.21.15.183:6801/574509311","cluster_addr":"172.21.15.183:6803/574509311","heartbeat_back_addr":"172.21.15.183:6807/574509311","heartbeat_front_addr":"172.21.15.183:6805/574509311","state":["exists","up"]},{"osd":3,"uuid":"e7cfced4-9b40-40de-a303-75ea8d5df4a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6811","nonce":570672219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6813","nonce":570672219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6817","nonce":570672219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6815","nonce":570672219}]},"public_addr":"172.21.15.53:6811/570672219","cluster_addr":"172.21.15.53:6813/570672219","heartbeat_back_addr":"172.21.15.53:6817/570672219","heartbeat_front_addr":"172.21.15.53:6815/570672219","state":["exists","up"]},{"osd":4,"uuid":"44a9e070-2282-43f1-9f05-b1f8ff81e540","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6809","nonce":1526850103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6811","nonce":1526850103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6815","nonce":1526850103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6813","nonce":1526850103}]},"public_addr":"172.21.15.150:6809/1526850103","cluster_addr":"172.21.15.150:6811/1526850103","heartbeat_back_addr":"172.21.15.150:6815/1526850103","heartbeat_front_addr":"172.21.15.150:6813/1526850103","state":["exists","up"]},{"osd":5,"uuid":"2c5619d2-c0f9-47d6-bf7e-2f4afd1b09f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6809","nonce":773726576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6811","nonce":773726576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6815","nonce":773726576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6813","nonce":773726576}]},"public_addr":"172.21.15.183:6809/773726576","cluster_addr":"172.21.15.183:6811/773726576","heartbeat_back_addr":"172.21.15.183:6815/773726576","heartbeat_front_addr":"172.21.15.183:6813/773726576","state":["exists","up"]},{"osd":6,"uuid":"1cf74d94-c118-4738-b6db-040af458a8eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6819","nonce":355884319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6821","nonce":355884319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6825","nonce":355884319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6823","nonce":355884319}]},"public_addr":"172.21.15.53:6819/355884319","cluster_addr":"172.21.15.53:6821/355884319","heartbeat_back_addr":"172.21.15.53:6825/355884319","heartbeat_front_addr":"172.21.15.53:6823/355884319","state":["exists","up"]},{"osd":7,"uuid":"0ce97c04-cd41-4080-bbbc-be9cf5fadc02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6817","nonce":229022562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6819","nonce":229022562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6823","nonce":229022562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6821","nonce":229022562}]},"public_addr":"172.21.15.150:6817/229022562","cluster_addr":"172.21.15.150:6819/229022562","heartbeat_back_addr":"172.21.15.150:6823/229022562","heartbeat_front_addr":"172.21.15.150:6821/229022562","state":["exists","up"]},{"osd":8,"uuid":"5d57787d-2a7c-4514-bd01-357ced413f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6817","nonce":301003844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6819","nonce":301003844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6823","nonce":301003844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6821","nonce":301003844}]},"public_addr":"172.21.15.183:6817/301003844","cluster_addr":"172.21.15.183:6819/301003844","heartbeat_back_addr":"172.21.15.183:6823/301003844","heartbeat_front_addr":"172.21.15.183:6821/301003844","state":["exists","up"]},{"osd":9,"uuid":"e11dd8eb-c55a-4d70-8c58-457540b7a50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6827","nonce":3449260978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6829","nonce":3449260978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6833","nonce":3449260978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6831","nonce":3449260978}]},"public_addr":"172.21.15.53:6827/3449260978","cluster_addr":"172.21.15.53:6829/3449260978","heartbeat_back_addr":"172.21.15.53:6833/3449260978","heartbeat_front_addr":"172.21.15.53:6831/3449260978","state":["exists","up"]},{"osd":10,"uuid":"5de4803d-ad37-4a90-add5-0048f44fc4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6825","nonce":3435992308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6827","nonce":3435992308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6831","nonce":3435992308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6829","nonce":3435992308}]},"public_addr":"172.21.15.150:6825/3435992308","cluster_addr":"172.21.15.150:6827/3435992308","heartbeat_back_addr":"172.21.15.150:6831/3435992308","heartbeat_front_addr":"172.21.15.150:6829/3435992308","state":["exists","up"]},{"osd":11,"uuid":"0e819a68-ed56-477e-b9c0-81e8cc0bce11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6825","nonce":1957842686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6827","nonce":1957842686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6831","nonce":1957842686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6829","nonce":1957842686}]},"public_addr":"172.21.15.183:6825/1957842686","cluster_addr":"172.21.15.183:6827/1957842686","heartbeat_back_addr":"172.21.15.183:6831/1957842686","heartbeat_front_addr":"172.21.15.183:6829/1957842686","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:09.840629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:34.769537+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:58.112969+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:33:29.656790+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:33:55.181287+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:34:19.772645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:34:53.833921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:35:19.076733+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:35:43.163390+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:36:16.568340+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:36:42.492768+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:37:07.944639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"3.38","mappings":[{"from":5,"to":7},{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6800/3887954889":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/1014501022":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/2576091441":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/1233527569":"2024-04-26T14:29:38.106242+0000","172.21.15.53:6801/3887954889":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/931889810":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/177196141":"2024-04-26T14:30:14.162383+0000","172.21.15.53:6800/1722497390":"2024-04-26T14:29:38.106242+0000","172.21.15.53:6801/1722497390":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/1927549198":"2024-04-26T14:30:14.162383+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:40:25.887 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.887+0000 7ff489c00640 1 -- 172.21.15.53:0/1556354036 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff4680a3bb0 msgr2=0x7ff4680a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:25.887 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.887+0000 7ff489c00640 1 --2- 172.21.15.53:0/1556354036 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff4680a3bb0 0x7ff4680a6070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff484109420 tx=0x7ff474005f90 comp rx=0 tx=0).stop 2024-04-25T14:40:25.887 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.887+0000 7ff489c00640 1 -- 172.21.15.53:0/1556354036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4841095c0 msgr2=0x7ff48419eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:25.887 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.887+0000 7ff489c00640 1 --2- 172.21.15.53:0/1556354036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4841095c0 0x7ff48419eaf0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff46c005940 tx=0x7ff46c0042f0 comp rx=0 tx=0).stop 2024-04-25T14:40:25.887 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.887+0000 7ff489c00640 1 -- 172.21.15.53:0/1556354036 shutdown_connections 2024-04-25T14:40:25.887 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.887+0000 7ff489c00640 1 --2- 172.21.15.53:0/1556354036 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ff4680a3bb0 0x7ff4680a6070 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:25.887 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.887+0000 7ff489c00640 1 --2- 172.21.15.53:0/1556354036 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff484109f80 0x7ff4841a5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:25.888 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.887+0000 7ff489c00640 1 --2- 172.21.15.53:0/1556354036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4841095c0 0x7ff48419eaf0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:25.888 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.887+0000 7ff489c00640 1 --2- 172.21.15.53:0/1556354036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4841083d0 0x7ff48419e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:25.888 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.887+0000 7ff489c00640 1 -- 172.21.15.53:0/1556354036 >> 172.21.15.53:0/1556354036 conn(0x7ff484077390 msgr2=0x7ff48410c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:25.888 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.888+0000 7ff489c00640 1 -- 172.21.15.53:0/1556354036 shutdown_connections 2024-04-25T14:40:25.888 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:25.888+0000 7ff489c00640 1 -- 172.21.15.53:0/1556354036 wait complete. 2024-04-25T14:40:25.907 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:33:02.338115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:38:07.517974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:38:08.994122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:40:25.907 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:40:25.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:25 smithi183 ceph-mon[32584]: pgmap v387: 97 pgs: 97 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s 2024-04-25T14:40:25.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:25 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/3861226700' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:40:25.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:25 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/3201012820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:40:25.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:25 smithi150 ceph-mon[32504]: pgmap v387: 97 pgs: 97 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s 2024-04-25T14:40:25.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:25 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/3861226700' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:40:25.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:25 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/3201012820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:40:26.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.120+0000 7f176fe00640 1 -- 172.21.15.53:0/2452840173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1770075f90 msgr2=0x7f17700743e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:26.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.120+0000 7f176fe00640 1 --2- 172.21.15.53:0/2452840173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1770075f90 0x7f17700743e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1760009860 tx=0x7f176002a340 comp rx=0 tx=0).stop 2024-04-25T14:40:26.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.120+0000 7f176fe00640 1 -- 172.21.15.53:0/2452840173 shutdown_connections 2024-04-25T14:40:26.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.120+0000 7f176fe00640 1 --2- 172.21.15.53:0/2452840173 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f177010e430 0x7f1770110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:26.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.120+0000 7f176fe00640 1 --2- 172.21.15.53:0/2452840173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17700749b0 0x7f1770074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:26.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.120+0000 7f176fe00640 1 --2- 172.21.15.53:0/2452840173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1770075f90 0x7f17700743e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:26.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.120+0000 7f176fe00640 1 -- 172.21.15.53:0/2452840173 >> 172.21.15.53:0/2452840173 conn(0x7f17700fd3f0 msgr2=0x7f17700ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:26.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.120+0000 7f176fe00640 1 -- 172.21.15.53:0/2452840173 shutdown_connections 2024-04-25T14:40:26.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.120+0000 7f176fe00640 1 -- 172.21.15.53:0/2452840173 wait complete. 2024-04-25T14:40:26.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.121+0000 7f176fe00640 1 Processor -- start 2024-04-25T14:40:26.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.121+0000 7f176fe00640 1 -- start start 2024-04-25T14:40:26.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.121+0000 7f176fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17700749b0 0x7f177019e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:26.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.121+0000 7f176fe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1770075f90 0x7f177019ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:26.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.122+0000 7f176fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f177010e430 0x7f17701a5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:26.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.122+0000 7f176fe00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1770113f60 con 0x7f17700749b0 2024-04-25T14:40:26.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.122+0000 7f176fe00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1770113de0 con 0x7f1770075f90 2024-04-25T14:40:26.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.122+0000 7f176fe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f17701140e0 con 0x7f177010e430 2024-04-25T14:40:26.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.122+0000 7f176ea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17700749b0 0x7f177019e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:26.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.122+0000 7f176ea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17700749b0 0x7f177019e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36942/0 (socket says 172.21.15.53:36942) 2024-04-25T14:40:26.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.122+0000 7f176ea00640 1 -- 172.21.15.53:0/3509227796 learned_addr learned my addr 172.21.15.53:0/3509227796 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:26.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.122+0000 7f176f400640 1 --2- 172.21.15.53:0/3509227796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f177010e430 0x7f17701a5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:26.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.122+0000 7f176e000640 1 --2- 172.21.15.53:0/3509227796 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1770075f90 0x7f177019ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:26.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.122+0000 7f176ea00640 1 -- 172.21.15.53:0/3509227796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f177010e430 msgr2=0x7f17701a5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:26.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.122+0000 7f176ea00640 1 --2- 172.21.15.53:0/3509227796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f177010e430 0x7f17701a5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:26.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.122+0000 7f176ea00640 1 -- 172.21.15.53:0/3509227796 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1770075f90 msgr2=0x7f177019ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:26.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.122+0000 7f176ea00640 1 --2- 172.21.15.53:0/3509227796 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1770075f90 0x7f177019ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:26.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.122+0000 7f176ea00640 1 -- 172.21.15.53:0/3509227796 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17701a62a0 con 0x7f17700749b0 2024-04-25T14:40:26.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.123+0000 7f176ea00640 1 --2- 172.21.15.53:0/3509227796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17700749b0 0x7f177019e680 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f17600057c0 tx=0x7f1760032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:26.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.123+0000 7f1767400640 1 -- 172.21.15.53:0/3509227796 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1760037040 con 0x7f17700749b0 2024-04-25T14:40:26.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.123+0000 7f176fe00640 1 -- 172.21.15.53:0/3509227796 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17701a6530 con 0x7f17700749b0 2024-04-25T14:40:26.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.123+0000 7f1767400640 1 -- 172.21.15.53:0/3509227796 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1760030bc0 con 0x7f17700749b0 2024-04-25T14:40:26.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.123+0000 7f1767400640 1 -- 172.21.15.53:0/3509227796 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1760005280 con 0x7f17700749b0 2024-04-25T14:40:26.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.123+0000 7f176fe00640 1 -- 172.21.15.53:0/3509227796 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17701a69c0 con 0x7f17700749b0 2024-04-25T14:40:26.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.124+0000 7f176fe00640 1 -- 172.21.15.53:0/3509227796 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1770104600 con 0x7f17700749b0 2024-04-25T14:40:26.126 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.126+0000 7f1767400640 1 -- 172.21.15.53:0/3509227796 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1760043020 con 0x7f17700749b0 2024-04-25T14:40:26.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.128+0000 7f1767400640 1 --2- 172.21.15.53:0/3509227796 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f17540a3bb0 0x7f17540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:26.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.128+0000 7f1767400640 1 -- 172.21.15.53:0/3509227796 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8285+0+0 (secure 0 0 0) 0x7f17600efa80 con 0x7f17700749b0 2024-04-25T14:40:26.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.128+0000 7f176e000640 1 --2- 172.21.15.53:0/3509227796 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f17540a3bb0 0x7f17540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:26.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.129+0000 7f176e000640 1 --2- 172.21.15.53:0/3509227796 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f17540a3bb0 0x7f17540a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f177019fb90 tx=0x7f175c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:26.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.133+0000 7f1767400640 1 -- 172.21.15.53:0/3509227796 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f17600f1e20 con 0x7f17700749b0 2024-04-25T14:40:26.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.339+0000 7f176fe00640 1 -- 172.21.15.53:0/3509227796 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1770062b50 con 0x7f17700749b0 2024-04-25T14:40:26.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.340+0000 7f1767400640 1 -- 172.21.15.53:0/3509227796 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f17600bc3d0 con 0x7f17700749b0 2024-04-25T14:40:26.340 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2024-04-25T14:40:26.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.342+0000 7f176fe00640 1 -- 172.21.15.53:0/3509227796 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f17540a3bb0 msgr2=0x7f17540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:26.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.342+0000 7f176fe00640 1 --2- 172.21.15.53:0/3509227796 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f17540a3bb0 0x7f17540a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f177019fb90 tx=0x7f175c005f90 comp rx=0 tx=0).stop 2024-04-25T14:40:26.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.342+0000 7f176fe00640 1 -- 172.21.15.53:0/3509227796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17700749b0 msgr2=0x7f177019e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:26.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.342+0000 7f176fe00640 1 --2- 172.21.15.53:0/3509227796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17700749b0 0x7f177019e680 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f17600057c0 tx=0x7f1760032da0 comp rx=0 tx=0).stop 2024-04-25T14:40:26.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.342+0000 7f176fe00640 1 -- 172.21.15.53:0/3509227796 shutdown_connections 2024-04-25T14:40:26.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.342+0000 7f176fe00640 1 --2- 172.21.15.53:0/3509227796 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f17540a3bb0 0x7f17540a6070 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:26.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.342+0000 7f176fe00640 1 --2- 172.21.15.53:0/3509227796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f177010e430 0x7f17701a5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:26.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.342+0000 7f176fe00640 1 --2- 172.21.15.53:0/3509227796 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1770075f90 0x7f177019ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:26.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.342+0000 7f176fe00640 1 --2- 172.21.15.53:0/3509227796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17700749b0 0x7f177019e680 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:26.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.342+0000 7f176fe00640 1 -- 172.21.15.53:0/3509227796 >> 172.21.15.53:0/3509227796 conn(0x7f17700fd3f0 msgr2=0x7f17701017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:26.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.343+0000 7f176fe00640 1 -- 172.21.15.53:0/3509227796 shutdown_connections 2024-04-25T14:40:26.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.343+0000 7f176fe00640 1 -- 172.21.15.53:0/3509227796 wait complete. 2024-04-25T14:40:26.364 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:40:26.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.584+0000 7fc555000640 1 -- 172.21.15.53:0/3919976294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc550108490 msgr2=0x7fc5501088b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:26.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.584+0000 7fc555000640 1 --2- 172.21.15.53:0/3919976294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc550108490 0x7fc5501088b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc540009ca0 tx=0x7fc54002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:40:26.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.584+0000 7fc555000640 1 -- 172.21.15.53:0/3919976294 shutdown_connections 2024-04-25T14:40:26.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.584+0000 7fc555000640 1 --2- 172.21.15.53:0/3919976294 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc55010a040 0x7fc5501108d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:26.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.584+0000 7fc555000640 1 --2- 172.21.15.53:0/3919976294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc550109680 0x7fc550109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:26.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.584+0000 7fc555000640 1 --2- 172.21.15.53:0/3919976294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc550108490 0x7fc5501088b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:26.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.584+0000 7fc555000640 1 -- 172.21.15.53:0/3919976294 >> 172.21.15.53:0/3919976294 conn(0x7fc550077420 msgr2=0x7fc550079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:26.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.584+0000 7fc555000640 1 -- 172.21.15.53:0/3919976294 shutdown_connections 2024-04-25T14:40:26.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.584+0000 7fc555000640 1 -- 172.21.15.53:0/3919976294 wait complete. 2024-04-25T14:40:26.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.585+0000 7fc555000640 1 Processor -- start 2024-04-25T14:40:26.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.585+0000 7fc555000640 1 -- start start 2024-04-25T14:40:26.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.585+0000 7fc555000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc550108490 0x7fc55019e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:26.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.585+0000 7fc555000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc550109680 0x7fc55019ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:26.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.585+0000 7fc555000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc55010a040 0x7fc5501a5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:26.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.585+0000 7fc555000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc550114020 con 0x7fc550109680 2024-04-25T14:40:26.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.585+0000 7fc555000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc550113ea0 con 0x7fc55010a040 2024-04-25T14:40:26.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.585+0000 7fc555000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc5501141a0 con 0x7fc550108490 2024-04-25T14:40:26.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.586+0000 7fc546a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc550109680 0x7fc55019ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:26.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.586+0000 7fc546a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc550109680 0x7fc55019ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36972/0 (socket says 172.21.15.53:36972) 2024-04-25T14:40:26.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.586+0000 7fc546a00640 1 -- 172.21.15.53:0/1739545037 learned_addr learned my addr 172.21.15.53:0/1739545037 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:26.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.586+0000 7fc54f400640 1 --2- 172.21.15.53:0/1739545037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc550108490 0x7fc55019e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:26.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.586+0000 7fc54fe00640 1 --2- 172.21.15.53:0/1739545037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc55010a040 0x7fc5501a5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:26.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.586+0000 7fc546a00640 1 -- 172.21.15.53:0/1739545037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc550108490 msgr2=0x7fc55019e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:26.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.586+0000 7fc546a00640 1 --2- 172.21.15.53:0/1739545037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc550108490 0x7fc55019e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:26.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.586+0000 7fc546a00640 1 -- 172.21.15.53:0/1739545037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc55010a040 msgr2=0x7fc5501a5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:26.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.586+0000 7fc546a00640 1 --2- 172.21.15.53:0/1739545037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc55010a040 0x7fc5501a5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:26.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.586+0000 7fc546a00640 1 -- 172.21.15.53:0/1739545037 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5501a62d0 con 0x7fc550109680 2024-04-25T14:40:26.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.587+0000 7fc546a00640 1 --2- 172.21.15.53:0/1739545037 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc550109680 0x7fc55019ebf0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc53800e9f0 tx=0x7fc53800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:26.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.587+0000 7fc54cc00640 1 -- 172.21.15.53:0/1739545037 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53800cd80 con 0x7fc550109680 2024-04-25T14:40:26.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.587+0000 7fc555000640 1 -- 172.21.15.53:0/1739545037 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5501a65c0 con 0x7fc550109680 2024-04-25T14:40:26.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.587+0000 7fc54cc00640 1 -- 172.21.15.53:0/1739545037 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5380054d0 con 0x7fc550109680 2024-04-25T14:40:26.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.587+0000 7fc54cc00640 1 -- 172.21.15.53:0/1739545037 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc538010690 con 0x7fc550109680 2024-04-25T14:40:26.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.587+0000 7fc555000640 1 -- 172.21.15.53:0/1739545037 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5501a6b10 con 0x7fc550109680 2024-04-25T14:40:26.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.588+0000 7fc555000640 1 -- 172.21.15.53:0/1739545037 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc55010f830 con 0x7fc550109680 2024-04-25T14:40:26.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.590+0000 7fc54cc00640 1 -- 172.21.15.53:0/1739545037 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc538004e90 con 0x7fc550109680 2024-04-25T14:40:26.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.591+0000 7fc54cc00640 1 --2- 172.21.15.53:0/1739545037 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fc5340a3bb0 0x7fc5340a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:26.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.592+0000 7fc54cc00640 1 -- 172.21.15.53:0/1739545037 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8285+0+0 (secure 0 0 0) 0x7fc5380d0880 con 0x7fc550109680 2024-04-25T14:40:26.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.592+0000 7fc54f400640 1 --2- 172.21.15.53:0/1739545037 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fc5340a3bb0 0x7fc5340a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:26.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.593+0000 7fc54f400640 1 --2- 172.21.15.53:0/1739545037 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fc5340a3bb0 0x7fc5340a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc540000f80 tx=0x7fc540005f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:26.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.596+0000 7fc54cc00640 1 -- 172.21.15.53:0/1739545037 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc53809d1d0 con 0x7fc550109680 2024-04-25T14:40:26.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:26 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1556354036' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:40:26.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:26 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/3509227796' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:40:26.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.808+0000 7fc555000640 1 -- 172.21.15.53:0/1739545037 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc550062b50 con 0x7fc550109680 2024-04-25T14:40:26.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.809+0000 7fc54cc00640 1 -- 172.21.15.53:0/1739545037 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7fc53809cad0 con 0x7fc550109680 2024-04-25T14:40:26.809 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2024-04-25T14:40:26.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.812+0000 7fc555000640 1 -- 172.21.15.53:0/1739545037 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fc5340a3bb0 msgr2=0x7fc5340a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:26.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.812+0000 7fc555000640 1 --2- 172.21.15.53:0/1739545037 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fc5340a3bb0 0x7fc5340a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc540000f80 tx=0x7fc540005f30 comp rx=0 tx=0).stop 2024-04-25T14:40:26.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.812+0000 7fc555000640 1 -- 172.21.15.53:0/1739545037 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc550109680 msgr2=0x7fc55019ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:26.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.812+0000 7fc555000640 1 --2- 172.21.15.53:0/1739545037 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc550109680 0x7fc55019ebf0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc53800e9f0 tx=0x7fc53800edf0 comp rx=0 tx=0).stop 2024-04-25T14:40:26.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.812+0000 7fc555000640 1 -- 172.21.15.53:0/1739545037 shutdown_connections 2024-04-25T14:40:26.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.812+0000 7fc555000640 1 --2- 172.21.15.53:0/1739545037 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fc5340a3bb0 0x7fc5340a6070 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:26.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.812+0000 7fc555000640 1 --2- 172.21.15.53:0/1739545037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc55010a040 0x7fc5501a5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:26.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.812+0000 7fc555000640 1 --2- 172.21.15.53:0/1739545037 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc550109680 0x7fc55019ebf0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:26.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.812+0000 7fc555000640 1 --2- 172.21.15.53:0/1739545037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc550108490 0x7fc55019e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:26.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.812+0000 7fc555000640 1 -- 172.21.15.53:0/1739545037 >> 172.21.15.53:0/1739545037 conn(0x7fc550077420 msgr2=0x7fc55010ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:26.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.812+0000 7fc555000640 1 -- 172.21.15.53:0/1739545037 shutdown_connections 2024-04-25T14:40:26.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:26.812+0000 7fc555000640 1 -- 172.21.15.53:0/1739545037 wait complete. 2024-04-25T14:40:26.833 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:40:26.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:26 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/1556354036' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:40:26.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:26 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/3509227796' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:40:26.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:26 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/1556354036' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:40:26.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:26 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/3509227796' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:40:27.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.053+0000 7f0253200640 1 -- 172.21.15.53:0/2635605159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f025410a040 msgr2=0x7f02541108d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:27.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.053+0000 7f0253200640 1 --2- 172.21.15.53:0/2635605159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f025410a040 0x7f02541108d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f024c009ca0 tx=0x7f024c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:40:27.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.053+0000 7f0253200640 1 -- 172.21.15.53:0/2635605159 shutdown_connections 2024-04-25T14:40:27.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.054+0000 7f0253200640 1 --2- 172.21.15.53:0/2635605159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f025410a040 0x7f02541108d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:27.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.054+0000 7f0253200640 1 --2- 172.21.15.53:0/2635605159 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0254109680 0x7f0254109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:27.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.054+0000 7f0253200640 1 --2- 172.21.15.53:0/2635605159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0254108490 0x7f02541088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:27.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.054+0000 7f0253200640 1 -- 172.21.15.53:0/2635605159 >> 172.21.15.53:0/2635605159 conn(0x7f0254077420 msgr2=0x7f0254079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:27.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.054+0000 7f0253200640 1 -- 172.21.15.53:0/2635605159 shutdown_connections 2024-04-25T14:40:27.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.054+0000 7f0253200640 1 -- 172.21.15.53:0/2635605159 wait complete. 2024-04-25T14:40:27.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.055+0000 7f0253200640 1 Processor -- start 2024-04-25T14:40:27.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.055+0000 7f0253200640 1 -- start start 2024-04-25T14:40:27.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.055+0000 7f0253200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0254108490 0x7f025419e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:27.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.055+0000 7f0253200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0254109680 0x7f025419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:27.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.055+0000 7f0253200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f025410a040 0x7f02541a5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:27.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.055+0000 7f0253200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0254113fd0 con 0x7f0254108490 2024-04-25T14:40:27.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.055+0000 7f0253200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0254113e50 con 0x7f0254109680 2024-04-25T14:40:27.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.055+0000 7f0253200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0254114150 con 0x7f025410a040 2024-04-25T14:40:27.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.055+0000 7f0251e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0254108490 0x7f025419e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:27.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.056+0000 7f0251e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0254108490 0x7f025419e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36986/0 (socket says 172.21.15.53:36986) 2024-04-25T14:40:27.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.056+0000 7f0251e00640 1 -- 172.21.15.53:0/1790735366 learned_addr learned my addr 172.21.15.53:0/1790735366 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:27.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.056+0000 7f024be00640 1 --2- 172.21.15.53:0/1790735366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0254109680 0x7f025419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:27.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.056+0000 7f0252800640 1 --2- 172.21.15.53:0/1790735366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f025410a040 0x7f02541a5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:27.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.056+0000 7f024be00640 1 -- 172.21.15.53:0/1790735366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f025410a040 msgr2=0x7f02541a5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:27.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.056+0000 7f024be00640 1 --2- 172.21.15.53:0/1790735366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f025410a040 0x7f02541a5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:27.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.056+0000 7f024be00640 1 -- 172.21.15.53:0/1790735366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0254108490 msgr2=0x7f025419e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:27.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.056+0000 7f024be00640 1 --2- 172.21.15.53:0/1790735366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0254108490 0x7f025419e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:27.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.056+0000 7f024be00640 1 -- 172.21.15.53:0/1790735366 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02541a6240 con 0x7f0254109680 2024-04-25T14:40:27.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.056+0000 7f024be00640 1 --2- 172.21.15.53:0/1790735366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0254109680 0x7f025419eb60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f023c00e9f0 tx=0x7f023c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:27.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.057+0000 7f024aa00640 1 -- 172.21.15.53:0/1790735366 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023c00ccc0 con 0x7f0254109680 2024-04-25T14:40:27.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.057+0000 7f0253200640 1 -- 172.21.15.53:0/1790735366 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02541a6530 con 0x7f0254109680 2024-04-25T14:40:27.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.057+0000 7f024aa00640 1 -- 172.21.15.53:0/1790735366 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f023c0108e0 con 0x7f0254109680 2024-04-25T14:40:27.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.057+0000 7f024aa00640 1 -- 172.21.15.53:0/1790735366 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023c018880 con 0x7f0254109680 2024-04-25T14:40:27.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.057+0000 7f0253200640 1 -- 172.21.15.53:0/1790735366 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02541a6ad0 con 0x7f0254109680 2024-04-25T14:40:27.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.058+0000 7f0253200640 1 -- 172.21.15.53:0/1790735366 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f025410f7b0 con 0x7f0254109680 2024-04-25T14:40:27.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.060+0000 7f024aa00640 1 -- 172.21.15.53:0/1790735366 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f023c010430 con 0x7f0254109680 2024-04-25T14:40:27.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.061+0000 7f024aa00640 1 --2- 172.21.15.53:0/1790735366 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f02380a3bb0 0x7f02380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:27.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.062+0000 7f024aa00640 1 -- 172.21.15.53:0/1790735366 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8285+0+0 (secure 0 0 0) 0x7f023c0d0780 con 0x7f0254109680 2024-04-25T14:40:27.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.062+0000 7f0251e00640 1 --2- 172.21.15.53:0/1790735366 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f02380a3bb0 0x7f02380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:27.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.062+0000 7f0251e00640 1 --2- 172.21.15.53:0/1790735366 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f02380a3bb0 0x7f02380a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f02440058c0 tx=0x7f0244005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:27.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.066+0000 7f024aa00640 1 -- 172.21.15.53:0/1790735366 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f023c09ca80 con 0x7f0254109680 2024-04-25T14:40:27.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.270+0000 7f0253200640 1 -- 172.21.15.53:0/1790735366 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0254062b50 con 0x7f0254109680 2024-04-25T14:40:27.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.271+0000 7f024aa00640 1 -- 172.21.15.53:0/1790735366 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f023c0a0d80 con 0x7f0254109680 2024-04-25T14:40:27.271 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2024-04-25T14:40:27.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.274+0000 7f0253200640 1 -- 172.21.15.53:0/1790735366 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f02380a3bb0 msgr2=0x7f02380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:27.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.274+0000 7f0253200640 1 --2- 172.21.15.53:0/1790735366 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f02380a3bb0 0x7f02380a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f02440058c0 tx=0x7f0244005f90 comp rx=0 tx=0).stop 2024-04-25T14:40:27.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.274+0000 7f0253200640 1 -- 172.21.15.53:0/1790735366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0254109680 msgr2=0x7f025419eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:27.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.274+0000 7f0253200640 1 --2- 172.21.15.53:0/1790735366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0254109680 0x7f025419eb60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f023c00e9f0 tx=0x7f023c00eeb0 comp rx=0 tx=0).stop 2024-04-25T14:40:27.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.274+0000 7f0253200640 1 -- 172.21.15.53:0/1790735366 shutdown_connections 2024-04-25T14:40:27.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.274+0000 7f0253200640 1 --2- 172.21.15.53:0/1790735366 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f02380a3bb0 0x7f02380a6070 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:27.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.274+0000 7f0253200640 1 --2- 172.21.15.53:0/1790735366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f025410a040 0x7f02541a5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:27.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.274+0000 7f0253200640 1 --2- 172.21.15.53:0/1790735366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0254109680 0x7f025419eb60 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:27.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.274+0000 7f0253200640 1 --2- 172.21.15.53:0/1790735366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0254108490 0x7f025419e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:27.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.274+0000 7f0253200640 1 -- 172.21.15.53:0/1790735366 >> 172.21.15.53:0/1790735366 conn(0x7f0254077420 msgr2=0x7f025410c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:27.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.274+0000 7f0253200640 1 -- 172.21.15.53:0/1790735366 shutdown_connections 2024-04-25T14:40:27.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.274+0000 7f0253200640 1 -- 172.21.15.53:0/1790735366 wait complete. 2024-04-25T14:40:27.293 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:40:27.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.508+0000 7f447d600640 1 -- 172.21.15.53:0/3108271678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44780ffe60 msgr2=0x7f44781002e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:27.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.508+0000 7f447d600640 1 --2- 172.21.15.53:0/3108271678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44780ffe60 0x7f44781002e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4460009860 tx=0x7f446002a340 comp rx=0 tx=0).stop 2024-04-25T14:40:27.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.508+0000 7f447d600640 1 -- 172.21.15.53:0/3108271678 shutdown_connections 2024-04-25T14:40:27.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.508+0000 7f447d600640 1 --2- 172.21.15.53:0/3108271678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4478100820 0x7f4478100cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:27.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.508+0000 7f447d600640 1 --2- 172.21.15.53:0/3108271678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44780ffe60 0x7f44781002e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:27.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.508+0000 7f447d600640 1 --2- 172.21.15.53:0/3108271678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4478102370 0x7f4478102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:27.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.508+0000 7f447d600640 1 -- 172.21.15.53:0/3108271678 >> 172.21.15.53:0/3108271678 conn(0x7f44780fb9e0 msgr2=0x7f44780fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:27.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.509+0000 7f447d600640 1 -- 172.21.15.53:0/3108271678 shutdown_connections 2024-04-25T14:40:27.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.509+0000 7f447d600640 1 -- 172.21.15.53:0/3108271678 wait complete. 2024-04-25T14:40:27.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.510+0000 7f447d600640 1 Processor -- start 2024-04-25T14:40:27.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.510+0000 7f447d600640 1 -- start start 2024-04-25T14:40:27.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.510+0000 7f447d600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44780ffe60 0x7f447819a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:27.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.510+0000 7f447d600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4478100820 0x7f447819a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:27.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.510+0000 7f447d600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4478102370 0x7f44781a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:27.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.510+0000 7f447d600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f447810fb40 con 0x7f4478102370 2024-04-25T14:40:27.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.510+0000 7f447d600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f447810f9c0 con 0x7f44780ffe60 2024-04-25T14:40:27.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.510+0000 7f447d600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f447810fcc0 con 0x7f4478100820 2024-04-25T14:40:27.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.510+0000 7f447cc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4478102370 0x7f44781a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:27.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.510+0000 7f446f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4478100820 0x7f447819a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:27.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.510+0000 7f4477e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44780ffe60 0x7f447819a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:27.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.510+0000 7f447cc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4478102370 0x7f44781a1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37010/0 (socket says 172.21.15.53:37010) 2024-04-25T14:40:27.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.510+0000 7f447cc00640 1 -- 172.21.15.53:0/3034884087 learned_addr learned my addr 172.21.15.53:0/3034884087 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:27.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.511+0000 7f446f400640 1 -- 172.21.15.53:0/3034884087 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44780ffe60 msgr2=0x7f447819a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:27.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.511+0000 7f446f400640 1 --2- 172.21.15.53:0/3034884087 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44780ffe60 0x7f447819a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:27.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.511+0000 7f446f400640 1 -- 172.21.15.53:0/3034884087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4478102370 msgr2=0x7f44781a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:27.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.511+0000 7f446f400640 1 --2- 172.21.15.53:0/3034884087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4478102370 0x7f44781a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:27.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.511+0000 7f446f400640 1 -- 172.21.15.53:0/3034884087 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44781a1ea0 con 0x7f4478100820 2024-04-25T14:40:27.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.511+0000 7f4477e00640 1 --2- 172.21.15.53:0/3034884087 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44780ffe60 0x7f447819a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:40:27.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.511+0000 7f446f400640 1 --2- 172.21.15.53:0/3034884087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4478100820 0x7f447819a7c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f446002a860 tx=0x7f4460032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:27.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.511+0000 7f4475600640 1 -- 172.21.15.53:0/3034884087 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4460040020 con 0x7f4478100820 2024-04-25T14:40:27.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.511+0000 7f447d600640 1 -- 172.21.15.53:0/3034884087 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44781a2130 con 0x7f4478100820 2024-04-25T14:40:27.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.511+0000 7f4475600640 1 -- 172.21.15.53:0/3034884087 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4460030d50 con 0x7f4478100820 2024-04-25T14:40:27.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.512+0000 7f4475600640 1 -- 172.21.15.53:0/3034884087 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4460037040 con 0x7f4478100820 2024-04-25T14:40:27.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.512+0000 7f447d600640 1 -- 172.21.15.53:0/3034884087 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44781a2670 con 0x7f4478100820 2024-04-25T14:40:27.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.513+0000 7f447d600640 1 -- 172.21.15.53:0/3034884087 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f447810bf50 con 0x7f4478100820 2024-04-25T14:40:27.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.515+0000 7f4475600640 1 -- 172.21.15.53:0/3034884087 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4460043020 con 0x7f4478100820 2024-04-25T14:40:27.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.516+0000 7f4475600640 1 --2- 172.21.15.53:0/3034884087 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f445c0a3bb0 0x7f445c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:27.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.516+0000 7f4475600640 1 -- 172.21.15.53:0/3034884087 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8285+0+0 (secure 0 0 0) 0x7f44600f0810 con 0x7f4478100820 2024-04-25T14:40:27.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.516+0000 7f4477e00640 1 --2- 172.21.15.53:0/3034884087 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f445c0a3bb0 0x7f445c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:27.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.517+0000 7f4477e00640 1 --2- 172.21.15.53:0/3034884087 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f445c0a3bb0 0x7f445c0a6070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f44680095f0 tx=0x7f4468005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:27.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.519+0000 7f4475600640 1 -- 172.21.15.53:0/3034884087 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f446004a020 con 0x7f4478100820 2024-04-25T14:40:27.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.768+0000 7f447d600640 1 -- 172.21.15.53:0/3034884087 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f447802d5c0 con 0x7f4478100820 2024-04-25T14:40:27.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.769+0000 7f4475600640 1 -- 172.21.15.53:0/3034884087 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11862 (secure 0 0 0) 0x7f44600bd1c0 con 0x7f4478100820 2024-04-25T14:40:27.770 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:40:27.770 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14460,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6841/41110727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6840","nonce":41110727},{"type":"v1","addr":"172.21.15.53:6841","nonce":41110727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2966992472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2966992472},{"type":"v1","addr":"172.21.15.150:6833","nonce":2966992472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/94143643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":94143643},{"type":"v1","addr":"172.21.15.183:6833","nonce":94143643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24470,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/506682943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":506682943},{"type":"v1","addr":"172.21.15.53:6835","nonce":506682943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1369488844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1369488844},{"type":"v1","addr":"172.21.15.150:6835","nonce":1369488844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3786232682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3786232682},{"type":"v1","addr":"172.21.15.53:6837","nonce":3786232682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T14:38:10.037451+0000","modified":"2024-04-25T14:38:15.629854+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24584,"mds_1":24565,"mds_2":24560},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24532":{"gid":24532,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1189328602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1189328602},{"type":"v1","addr":"172.21.15.183:6835","nonce":1189328602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/2344343627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2344343627},{"type":"v1","addr":"172.21.15.150:6837","nonce":2344343627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24544":{"gid":24544,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1163336384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1163336384},{"type":"v1","addr":"172.21.15.183:6837","nonce":1163336384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6839/570000540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":570000540},{"type":"v1","addr":"172.21.15.53:6839","nonce":570000540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.183:6839/1554258915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1554258915},{"type":"v1","addr":"172.21.15.183:6839","nonce":1554258915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/149573616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":149573616},{"type":"v1","addr":"172.21.15.150:6839","nonce":149573616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24565,24584,24560]},"id":1}]} 2024-04-25T14:40:27.770 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 21 2024-04-25T14:40:27.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.772+0000 7f447d600640 1 -- 172.21.15.53:0/3034884087 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f445c0a3bb0 msgr2=0x7f445c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:27.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.772+0000 7f447d600640 1 --2- 172.21.15.53:0/3034884087 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f445c0a3bb0 0x7f445c0a6070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f44680095f0 tx=0x7f4468005f90 comp rx=0 tx=0).stop 2024-04-25T14:40:27.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.772+0000 7f447d600640 1 -- 172.21.15.53:0/3034884087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4478100820 msgr2=0x7f447819a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:27.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.772+0000 7f447d600640 1 --2- 172.21.15.53:0/3034884087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4478100820 0x7f447819a7c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f446002a860 tx=0x7f4460032d50 comp rx=0 tx=0).stop 2024-04-25T14:40:27.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.772+0000 7f447d600640 1 -- 172.21.15.53:0/3034884087 shutdown_connections 2024-04-25T14:40:27.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.772+0000 7f447d600640 1 --2- 172.21.15.53:0/3034884087 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f445c0a3bb0 0x7f445c0a6070 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:27.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.772+0000 7f447d600640 1 --2- 172.21.15.53:0/3034884087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4478102370 0x7f44781a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:27.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.772+0000 7f447d600640 1 --2- 172.21.15.53:0/3034884087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4478100820 0x7f447819a7c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:27.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.772+0000 7f447d600640 1 --2- 172.21.15.53:0/3034884087 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44780ffe60 0x7f447819a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:27.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.772+0000 7f447d600640 1 -- 172.21.15.53:0/3034884087 >> 172.21.15.53:0/3034884087 conn(0x7f44780fb9e0 msgr2=0x7f44780fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:27.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.772+0000 7f447d600640 1 -- 172.21.15.53:0/3034884087 shutdown_connections 2024-04-25T14:40:27.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:27.772+0000 7f447d600640 1 -- 172.21.15.53:0/3034884087 wait complete. 2024-04-25T14:40:27.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:27 smithi053 ceph-mon[27773]: pgmap v388: 97 pgs: 97 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T14:40:27.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:27 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1739545037' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:40:27.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:27 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1790735366' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:40:27.792 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:40:27.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:27 smithi183 ceph-mon[32584]: pgmap v388: 97 pgs: 97 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T14:40:27.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:27 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/1739545037' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:40:27.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:27 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/1790735366' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:40:27.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:27 smithi150 ceph-mon[32504]: pgmap v388: 97 pgs: 97 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T14:40:27.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:27 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/1739545037' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:40:27.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:27 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/1790735366' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:40:28.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.005+0000 7fdf5be00640 1 -- 172.21.15.53:0/2966558017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf5c107ab0 msgr2=0x7fdf5c109f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:28.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.005+0000 7fdf5be00640 1 --2- 172.21.15.53:0/2966558017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf5c107ab0 0x7fdf5c109f00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdf54009be0 tx=0x7fdf5402a6c0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.005+0000 7fdf5be00640 1 -- 172.21.15.53:0/2966558017 shutdown_connections 2024-04-25T14:40:28.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.005+0000 7fdf5be00640 1 --2- 172.21.15.53:0/2966558017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf5c107ab0 0x7fdf5c109f00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.005+0000 7fdf5be00640 1 --2- 172.21.15.53:0/2966558017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf5c105180 0x7fdf5c107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.005+0000 7fdf5be00640 1 --2- 172.21.15.53:0/2966558017 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf5c102850 0x7fdf5c104c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.005+0000 7fdf5be00640 1 -- 172.21.15.53:0/2966558017 >> 172.21.15.53:0/2966558017 conn(0x7fdf5c0fc220 msgr2=0x7fdf5c0fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:28.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.005+0000 7fdf5be00640 1 -- 172.21.15.53:0/2966558017 shutdown_connections 2024-04-25T14:40:28.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.005+0000 7fdf5be00640 1 -- 172.21.15.53:0/2966558017 wait complete. 2024-04-25T14:40:28.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.006+0000 7fdf5be00640 1 Processor -- start 2024-04-25T14:40:28.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.006+0000 7fdf5be00640 1 -- start start 2024-04-25T14:40:28.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.006+0000 7fdf5be00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf5c102850 0x7fdf5c197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:28.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.006+0000 7fdf5be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf5c105180 0x7fdf5c1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:28.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.006+0000 7fdf5be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf5c107ab0 0x7fdf5c19f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:28.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.006+0000 7fdf5be00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5c10d920 con 0x7fdf5c105180 2024-04-25T14:40:28.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.006+0000 7fdf5be00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5c10d7a0 con 0x7fdf5c102850 2024-04-25T14:40:28.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.006+0000 7fdf5be00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5c10daa0 con 0x7fdf5c107ab0 2024-04-25T14:40:28.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.007+0000 7fdf5a000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf5c105180 0x7fdf5c1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:28.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.007+0000 7fdf5b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf5c107ab0 0x7fdf5c19f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:28.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.007+0000 7fdf5a000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf5c105180 0x7fdf5c1984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37030/0 (socket says 172.21.15.53:37030) 2024-04-25T14:40:28.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.007+0000 7fdf5a000640 1 -- 172.21.15.53:0/3899265818 learned_addr learned my addr 172.21.15.53:0/3899265818 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:28.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.007+0000 7fdf5aa00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf5c102850 0x7fdf5c197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:28.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.007+0000 7fdf5b400640 1 -- 172.21.15.53:0/3899265818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf5c102850 msgr2=0x7fdf5c197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:28.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.007+0000 7fdf5b400640 1 --2- 172.21.15.53:0/3899265818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf5c102850 0x7fdf5c197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.007+0000 7fdf5b400640 1 -- 172.21.15.53:0/3899265818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf5c105180 msgr2=0x7fdf5c1984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:28.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.007+0000 7fdf5b400640 1 --2- 172.21.15.53:0/3899265818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf5c105180 0x7fdf5c1984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.007+0000 7fdf5b400640 1 -- 172.21.15.53:0/3899265818 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf5c19fbc0 con 0x7fdf5c107ab0 2024-04-25T14:40:28.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.007+0000 7fdf5a000640 1 --2- 172.21.15.53:0/3899265818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf5c105180 0x7fdf5c1984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:40:28.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.007+0000 7fdf5b400640 1 --2- 172.21.15.53:0/3899265818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf5c107ab0 0x7fdf5c19f550 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdf54034040 tx=0x7fdf54030d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:28.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.008+0000 7fdf53400640 1 -- 172.21.15.53:0/3899265818 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf54041020 con 0x7fdf5c107ab0 2024-04-25T14:40:28.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.008+0000 7fdf5be00640 1 -- 172.21.15.53:0/3899265818 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf5c19fe50 con 0x7fdf5c107ab0 2024-04-25T14:40:28.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.008+0000 7fdf53400640 1 -- 172.21.15.53:0/3899265818 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf54030ec0 con 0x7fdf5c107ab0 2024-04-25T14:40:28.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.008+0000 7fdf53400640 1 -- 172.21.15.53:0/3899265818 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf54036040 con 0x7fdf5c107ab0 2024-04-25T14:40:28.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.008+0000 7fdf5be00640 1 -- 172.21.15.53:0/3899265818 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf5c1a0310 con 0x7fdf5c107ab0 2024-04-25T14:40:28.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.009+0000 7fdf5be00640 1 -- 172.21.15.53:0/3899265818 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf5c0fdd10 con 0x7fdf5c107ab0 2024-04-25T14:40:28.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.011+0000 7fdf53400640 1 -- 172.21.15.53:0/3899265818 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdf54044020 con 0x7fdf5c107ab0 2024-04-25T14:40:28.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.012+0000 7fdf53400640 1 --2- 172.21.15.53:0/3899265818 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fdf400a3c30 0x7fdf400a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:28.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.013+0000 7fdf53400640 1 -- 172.21.15.53:0/3899265818 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8285+0+0 (secure 0 0 0) 0x7fdf540f1320 con 0x7fdf5c107ab0 2024-04-25T14:40:28.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.013+0000 7fdf5aa00640 1 --2- 172.21.15.53:0/3899265818 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fdf400a3c30 0x7fdf400a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:28.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.013+0000 7fdf5aa00640 1 --2- 172.21.15.53:0/3899265818 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fdf400a3c30 0x7fdf400a60f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c009bd0 tx=0x7fdf4c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:28.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.015+0000 7fdf53400640 1 -- 172.21.15.53:0/3899265818 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdf5404b020 con 0x7fdf5c107ab0 2024-04-25T14:40:28.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.213+0000 7fdf5be00640 1 -- 172.21.15.53:0/3899265818 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdf5c02d810 con 0x7fdf5c107ab0 2024-04-25T14:40:28.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.215+0000 7fdf53400640 1 -- 172.21.15.53:0/3899265818 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22732 (secure 0 0 0) 0x7fdf540bdc70 con 0x7fdf5c107ab0 2024-04-25T14:40:28.215 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:40:28.215 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":92,"fsid":"fd30e6ce-030f-11ef-bc93-c7b262605968","created":"2024-04-25T14:28:54.330326+0000","modified":"2024-04-25T14:40:16.467090+0000","last_up_change":"2024-04-25T14:37:10.921148+0000","last_in_change":"2024-04-25T14:36:49.566459+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:33:02.338115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:38:07.517974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:38:08.994122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aecfbaf0-39da-4749-a49b-c7205dabd4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6803","nonce":464176015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6805","nonce":464176015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6809","nonce":464176015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6807","nonce":464176015}]},"public_addr":"172.21.15.53:6803/464176015","cluster_addr":"172.21.15.53:6805/464176015","heartbeat_back_addr":"172.21.15.53:6809/464176015","heartbeat_front_addr":"172.21.15.53:6807/464176015","state":["exists","up"]},{"osd":1,"uuid":"2c4a21ae-c43e-4839-8a56-c3fc63aeb7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6801","nonce":1304621982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6803","nonce":1304621982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6807","nonce":1304621982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6805","nonce":1304621982}]},"public_addr":"172.21.15.150:6801/1304621982","cluster_addr":"172.21.15.150:6803/1304621982","heartbeat_back_addr":"172.21.15.150:6807/1304621982","heartbeat_front_addr":"172.21.15.150:6805/1304621982","state":["exists","up"]},{"osd":2,"uuid":"e8effd43-adc0-472a-9eed-a9273b06dc0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6801","nonce":574509311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6803","nonce":574509311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6807","nonce":574509311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6805","nonce":574509311}]},"public_addr":"172.21.15.183:6801/574509311","cluster_addr":"172.21.15.183:6803/574509311","heartbeat_back_addr":"172.21.15.183:6807/574509311","heartbeat_front_addr":"172.21.15.183:6805/574509311","state":["exists","up"]},{"osd":3,"uuid":"e7cfced4-9b40-40de-a303-75ea8d5df4a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6811","nonce":570672219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6813","nonce":570672219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6817","nonce":570672219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6815","nonce":570672219}]},"public_addr":"172.21.15.53:6811/570672219","cluster_addr":"172.21.15.53:6813/570672219","heartbeat_back_addr":"172.21.15.53:6817/570672219","heartbeat_front_addr":"172.21.15.53:6815/570672219","state":["exists","up"]},{"osd":4,"uuid":"44a9e070-2282-43f1-9f05-b1f8ff81e540","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6809","nonce":1526850103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6811","nonce":1526850103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6815","nonce":1526850103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6813","nonce":1526850103}]},"public_addr":"172.21.15.150:6809/1526850103","cluster_addr":"172.21.15.150:6811/1526850103","heartbeat_back_addr":"172.21.15.150:6815/1526850103","heartbeat_front_addr":"172.21.15.150:6813/1526850103","state":["exists","up"]},{"osd":5,"uuid":"2c5619d2-c0f9-47d6-bf7e-2f4afd1b09f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6809","nonce":773726576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6811","nonce":773726576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6815","nonce":773726576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6813","nonce":773726576}]},"public_addr":"172.21.15.183:6809/773726576","cluster_addr":"172.21.15.183:6811/773726576","heartbeat_back_addr":"172.21.15.183:6815/773726576","heartbeat_front_addr":"172.21.15.183:6813/773726576","state":["exists","up"]},{"osd":6,"uuid":"1cf74d94-c118-4738-b6db-040af458a8eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6819","nonce":355884319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6821","nonce":355884319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6825","nonce":355884319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6823","nonce":355884319}]},"public_addr":"172.21.15.53:6819/355884319","cluster_addr":"172.21.15.53:6821/355884319","heartbeat_back_addr":"172.21.15.53:6825/355884319","heartbeat_front_addr":"172.21.15.53:6823/355884319","state":["exists","up"]},{"osd":7,"uuid":"0ce97c04-cd41-4080-bbbc-be9cf5fadc02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6817","nonce":229022562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6819","nonce":229022562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6823","nonce":229022562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6821","nonce":229022562}]},"public_addr":"172.21.15.150:6817/229022562","cluster_addr":"172.21.15.150:6819/229022562","heartbeat_back_addr":"172.21.15.150:6823/229022562","heartbeat_front_addr":"172.21.15.150:6821/229022562","state":["exists","up"]},{"osd":8,"uuid":"5d57787d-2a7c-4514-bd01-357ced413f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6817","nonce":301003844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6819","nonce":301003844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6823","nonce":301003844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6821","nonce":301003844}]},"public_addr":"172.21.15.183:6817/301003844","cluster_addr":"172.21.15.183:6819/301003844","heartbeat_back_addr":"172.21.15.183:6823/301003844","heartbeat_front_addr":"172.21.15.183:6821/301003844","state":["exists","up"]},{"osd":9,"uuid":"e11dd8eb-c55a-4d70-8c58-457540b7a50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6827","nonce":3449260978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6829","nonce":3449260978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6833","nonce":3449260978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6831","nonce":3449260978}]},"public_addr":"172.21.15.53:6827/3449260978","cluster_addr":"172.21.15.53:6829/3449260978","heartbeat_back_addr":"172.21.15.53:6833/3449260978","heartbeat_front_addr":"172.21.15.53:6831/3449260978","state":["exists","up"]},{"osd":10,"uuid":"5de4803d-ad37-4a90-add5-0048f44fc4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6825","nonce":3435992308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6827","nonce":3435992308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6831","nonce":3435992308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6829","nonce":3435992308}]},"public_addr":"172.21.15.150:6825/3435992308","cluster_addr":"172.21.15.150:6827/3435992308","heartbeat_back_addr":"172.21.15.150:6831/3435992308","heartbeat_front_addr":"172.21.15.150:6829/3435992308","state":["exists","up"]},{"osd":11,"uuid":"0e819a68-ed56-477e-b9c0-81e8cc0bce11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6825","nonce":1957842686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6827","nonce":1957842686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6831","nonce":1957842686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6829","nonce":1957842686}]},"public_addr":"172.21.15.183:6825/1957842686","cluster_addr":"172.21.15.183:6827/1957842686","heartbeat_back_addr":"172.21.15.183:6831/1957842686","heartbeat_front_addr":"172.21.15.183:6829/1957842686","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:09.840629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:34.769537+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:58.112969+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:33:29.656790+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:33:55.181287+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:34:19.772645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:34:53.833921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:35:19.076733+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:35:43.163390+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:36:16.568340+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:36:42.492768+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:37:07.944639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"3.38","mappings":[{"from":5,"to":7},{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6800/3887954889":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/1014501022":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/2576091441":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/1233527569":"2024-04-26T14:29:38.106242+0000","172.21.15.53:6801/3887954889":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/931889810":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/177196141":"2024-04-26T14:30:14.162383+0000","172.21.15.53:6800/1722497390":"2024-04-26T14:29:38.106242+0000","172.21.15.53:6801/1722497390":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/1927549198":"2024-04-26T14:30:14.162383+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:40:28.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.218+0000 7fdf5be00640 1 -- 172.21.15.53:0/3899265818 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fdf400a3c30 msgr2=0x7fdf400a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:28.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.218+0000 7fdf5be00640 1 --2- 172.21.15.53:0/3899265818 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fdf400a3c30 0x7fdf400a60f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c009bd0 tx=0x7fdf4c005f90 comp rx=0 tx=0).stop 2024-04-25T14:40:28.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.218+0000 7fdf5be00640 1 -- 172.21.15.53:0/3899265818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf5c107ab0 msgr2=0x7fdf5c19f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:28.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.218+0000 7fdf5be00640 1 --2- 172.21.15.53:0/3899265818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf5c107ab0 0x7fdf5c19f550 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdf54034040 tx=0x7fdf54030d50 comp rx=0 tx=0).stop 2024-04-25T14:40:28.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.218+0000 7fdf5be00640 1 -- 172.21.15.53:0/3899265818 shutdown_connections 2024-04-25T14:40:28.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.218+0000 7fdf5be00640 1 --2- 172.21.15.53:0/3899265818 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fdf400a3c30 0x7fdf400a60f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.218+0000 7fdf5be00640 1 --2- 172.21.15.53:0/3899265818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf5c107ab0 0x7fdf5c19f550 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.218+0000 7fdf5be00640 1 --2- 172.21.15.53:0/3899265818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf5c105180 0x7fdf5c1984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.218+0000 7fdf5be00640 1 --2- 172.21.15.53:0/3899265818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf5c102850 0x7fdf5c197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.218+0000 7fdf5be00640 1 -- 172.21.15.53:0/3899265818 >> 172.21.15.53:0/3899265818 conn(0x7fdf5c0fc220 msgr2=0x7fdf5c1030f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:28.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.218+0000 7fdf5be00640 1 -- 172.21.15.53:0/3899265818 shutdown_connections 2024-04-25T14:40:28.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.218+0000 7fdf5be00640 1 -- 172.21.15.53:0/3899265818 wait complete. 2024-04-25T14:40:28.235 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T14:38:10.037451+0000', 'modified': '2024-04-25T14:38:15.629854+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24584, 'mds_1': 24565, 'mds_2': 24560}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24532': {'gid': 24532, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6835/1189328602', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6834', 'nonce': 1189328602}, {'type': 'v1', 'addr': '172.21.15.183:6835', 'nonce': 1189328602}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24533': {'gid': 24533, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.150:6837/2344343627', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6836', 'nonce': 2344343627}, {'type': 'v1', 'addr': '172.21.15.150:6837', 'nonce': 2344343627}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24544': {'gid': 24544, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6837/1163336384', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6836', 'nonce': 1163336384}, {'type': 'v1', 'addr': '172.21.15.183:6837', 'nonce': 1163336384}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24560': {'gid': 24560, 'name': 'g', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.53:6839/570000540', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6838', 'nonce': 570000540}, {'type': 'v1', 'addr': '172.21.15.53:6839', 'nonce': 570000540}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24565': {'gid': 24565, 'name': 'i', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.183:6839/1554258915', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6838', 'nonce': 1554258915}, {'type': 'v1', 'addr': '172.21.15.183:6839', 'nonce': 1554258915}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24584': {'gid': 24584, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.150:6839/149573616', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6838', 'nonce': 149573616}, {'type': 'v1', 'addr': '172.21.15.150:6839', 'nonce': 149573616}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24584, 'qdb_cluster': [24565, 24584, 24560]} 2024-04-25T14:40:28.235 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-25T14:40:28.236 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2024-04-25T14:40:28.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.447+0000 7f11c2a00640 1 -- 172.21.15.53:0/3111132941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11bc105c40 msgr2=0x7f11bc10c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:28.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.447+0000 7f11c2a00640 1 --2- 172.21.15.53:0/3111132941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11bc105c40 0x7f11bc10c4d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f11b4009ca0 tx=0x7f11b402a8b0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.447+0000 7f11c2a00640 1 -- 172.21.15.53:0/3111132941 shutdown_connections 2024-04-25T14:40:28.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.447+0000 7f11c2a00640 1 --2- 172.21.15.53:0/3111132941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11bc105c40 0x7f11bc10c4d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.447+0000 7f11c2a00640 1 --2- 172.21.15.53:0/3111132941 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11bc105280 0x7f11bc105700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.447+0000 7f11c2a00640 1 --2- 172.21.15.53:0/3111132941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11bc104090 0x7f11bc1044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.447+0000 7f11c2a00640 1 -- 172.21.15.53:0/3111132941 >> 172.21.15.53:0/3111132941 conn(0x7f11bc0ff600 msgr2=0x7f11bc101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:28.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.447+0000 7f11c2a00640 1 -- 172.21.15.53:0/3111132941 shutdown_connections 2024-04-25T14:40:28.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.447+0000 7f11c2a00640 1 -- 172.21.15.53:0/3111132941 wait complete. 2024-04-25T14:40:28.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.448+0000 7f11c2a00640 1 Processor -- start 2024-04-25T14:40:28.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.448+0000 7f11c2a00640 1 -- start start 2024-04-25T14:40:28.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.449+0000 7f11c2a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11bc104090 0x7f11bc19a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:28.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.449+0000 7f11c2a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11bc105280 0x7f11bc19a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:28.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.449+0000 7f11c2a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11bc105c40 0x7f11bc1a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:28.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.449+0000 7f11c2a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc10fc20 con 0x7f11bc104090 2024-04-25T14:40:28.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.449+0000 7f11c2a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc10faa0 con 0x7f11bc105280 2024-04-25T14:40:28.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.449+0000 7f11c2a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc10fda0 con 0x7f11bc105c40 2024-04-25T14:40:28.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.449+0000 7f11c1600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11bc104090 0x7f11bc19a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:28.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.449+0000 7f11c1600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11bc104090 0x7f11bc19a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37062/0 (socket says 172.21.15.53:37062) 2024-04-25T14:40:28.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.449+0000 7f11c1600640 1 -- 172.21.15.53:0/496162625 learned_addr learned my addr 172.21.15.53:0/496162625 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:28.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.449+0000 7f11c2000640 1 --2- 172.21.15.53:0/496162625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11bc105c40 0x7f11bc1a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:28.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.449+0000 7f11c0c00640 1 --2- 172.21.15.53:0/496162625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11bc105280 0x7f11bc19a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:28.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.449+0000 7f11c1600640 1 -- 172.21.15.53:0/496162625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11bc105c40 msgr2=0x7f11bc1a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:28.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.449+0000 7f11c1600640 1 --2- 172.21.15.53:0/496162625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11bc105c40 0x7f11bc1a1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.449+0000 7f11c1600640 1 -- 172.21.15.53:0/496162625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11bc105280 msgr2=0x7f11bc19a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:28.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.449+0000 7f11c1600640 1 --2- 172.21.15.53:0/496162625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11bc105280 0x7f11bc19a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.449+0000 7f11c1600640 1 -- 172.21.15.53:0/496162625 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11bc1a1e90 con 0x7f11bc104090 2024-04-25T14:40:28.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.450+0000 7f11c1600640 1 --2- 172.21.15.53:0/496162625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11bc104090 0x7f11bc19a270 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f11b000e9c0 tx=0x7f11b000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:28.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.450+0000 7f11ba000640 1 -- 172.21.15.53:0/496162625 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b000cd80 con 0x7f11bc104090 2024-04-25T14:40:28.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.450+0000 7f11ba000640 1 -- 172.21.15.53:0/496162625 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11b00035f0 con 0x7f11bc104090 2024-04-25T14:40:28.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.450+0000 7f11c2a00640 1 -- 172.21.15.53:0/496162625 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11bc1a2180 con 0x7f11bc104090 2024-04-25T14:40:28.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.450+0000 7f11ba000640 1 -- 172.21.15.53:0/496162625 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b0010690 con 0x7f11bc104090 2024-04-25T14:40:28.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.450+0000 7f11c2a00640 1 -- 172.21.15.53:0/496162625 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11bc1a2680 con 0x7f11bc104090 2024-04-25T14:40:28.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.453+0000 7f11c6fe5740 1 Processor -- start 2024-04-25T14:40:28.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.453+0000 7f11c6fe5740 1 -- start start 2024-04-25T14:40:28.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.454+0000 7f11c6fe5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562659296150 0x562659296570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:28.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.454+0000 7f11c6fe5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562659296b40 0x56265929b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:28.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.454+0000 7f11c6fe5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56265929bcb0 0x56265929e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:28.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.454+0000 7f11c6fe5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5626591fab10 con 0x562659296150 2024-04-25T14:40:28.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.454+0000 7f11c6fe5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5626591a12b0 con 0x562659296b40 2024-04-25T14:40:28.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.454+0000 7f11c6fe5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5626591f6ad0 con 0x56265929bcb0 2024-04-25T14:40:28.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.454+0000 7f11ba000640 1 -- 172.21.15.53:0/496162625 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f11b0004000 con 0x7f11bc104090 2024-04-25T14:40:28.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.454+0000 7f11c2000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562659296150 0x562659296570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:28.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.454+0000 7f11c2000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562659296150 0x562659296570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37068/0 (socket says 172.21.15.53:37068) 2024-04-25T14:40:28.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.454+0000 7f11c2000640 1 -- 172.21.15.53:0/4057634076 learned_addr learned my addr 172.21.15.53:0/4057634076 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:28.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.454+0000 7f11c0c00640 1 --2- 172.21.15.53:0/4057634076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56265929bcb0 0x56265929e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:28.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.454+0000 7f11c1600640 1 --2- 172.21.15.53:0/4057634076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562659296b40 0x56265929b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:28.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.454+0000 7f11c2000640 1 -- 172.21.15.53:0/4057634076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56265929bcb0 msgr2=0x56265929e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:40:28.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.454+0000 7f11c2000640 1 --2- 172.21.15.53:0/4057634076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56265929bcb0 0x56265929e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.454+0000 7f11c2000640 1 -- 172.21.15.53:0/4057634076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562659296b40 msgr2=0x56265929b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:40:28.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.454+0000 7f11c2000640 1 --2- 172.21.15.53:0/4057634076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562659296b40 0x56265929b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.454+0000 7f11c2000640 1 -- 172.21.15.53:0/4057634076 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562658f0d2d0 con 0x562659296150 2024-04-25T14:40:28.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.454+0000 7f11c1600640 1 --2- 172.21.15.53:0/4057634076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562659296b40 0x56265929b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:40:28.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.455+0000 7f11c2000640 1 --2- 172.21.15.53:0/4057634076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562659296150 0x562659296570 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f11b4035000 tx=0x7f11b4009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c39ce910d6a22968 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:28.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.455+0000 7f11ae600640 1 -- 172.21.15.53:0/4057634076 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b4030900 con 0x562659296150 2024-04-25T14:40:28.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.455+0000 7f11ae600640 1 -- 172.21.15.53:0/4057634076 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11b4036030 con 0x562659296150 2024-04-25T14:40:28.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.455+0000 7f11ae600640 1 -- 172.21.15.53:0/4057634076 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b40044a0 con 0x562659296150 2024-04-25T14:40:28.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.456+0000 7f11ba000640 1 --2- 172.21.15.53:0/496162625 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f11a40a3bb0 0x7f11a40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:28.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.456+0000 7f11c1600640 1 --2- 172.21.15.53:0/496162625 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f11a40a3bb0 0x7f11a40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:28.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.456+0000 7f11c6fe5740 1 -- 172.21.15.53:0/4057634076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562659296150 msgr2=0x562659296570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:28.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.456+0000 7f11ba000640 1 -- 172.21.15.53:0/496162625 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8285+0+0 (secure 0 0 0) 0x7f11b00d0920 con 0x7f11bc104090 2024-04-25T14:40:28.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.456+0000 7f11c6fe5740 1 --2- 172.21.15.53:0/4057634076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562659296150 0x562659296570 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f11b4035000 tx=0x7f11b4009a20 comp rx=0 tx=0).stop 2024-04-25T14:40:28.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.456+0000 7f11c6fe5740 1 -- 172.21.15.53:0/4057634076 shutdown_connections 2024-04-25T14:40:28.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.456+0000 7f11c6fe5740 1 --2- 172.21.15.53:0/4057634076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56265929bcb0 0x56265929e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.456+0000 7f11c6fe5740 1 --2- 172.21.15.53:0/4057634076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562659296b40 0x56265929b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.456+0000 7f11c6fe5740 1 --2- 172.21.15.53:0/4057634076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562659296150 0x562659296570 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.456+0000 7f11c6fe5740 1 -- 172.21.15.53:0/4057634076 >> 172.21.15.53:0/4057634076 conn(0x5626592917c0 msgr2=0x562659291bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:28.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.456+0000 7f11c6fe5740 1 -- 172.21.15.53:0/4057634076 shutdown_connections 2024-04-25T14:40:28.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.456+0000 7f11c6fe5740 1 -- 172.21.15.53:0/4057634076 wait complete. 2024-04-25T14:40:28.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.457+0000 7f11c1600640 1 --2- 172.21.15.53:0/496162625 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f11a40a3bb0 0x7f11a40a6070 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f11b00ce760 tx=0x7f11b0003e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:28.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.457+0000 7f11c6fe5740 1 -- start start 2024-04-25T14:40:28.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.457+0000 7f11c6fe5740 1 Processor -- start 2024-04-25T14:40:28.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.457+0000 7f11c6fe5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562659296150 0x5626592a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:28.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.457+0000 7f11c6fe5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562659296b40 0x5626592a0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:28.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.457+0000 7f11c6fe5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56265929bcb0 0x5626592a7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:28.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.457+0000 7f11c6fe5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56265929ec00 con 0x562659296b40 2024-04-25T14:40:28.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.457+0000 7f11c6fe5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5626591b4570 con 0x56265929bcb0 2024-04-25T14:40:28.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.457+0000 7f11c6fe5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56265929ed80 con 0x562659296150 2024-04-25T14:40:28.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.458+0000 7f11c6fe5740 10 client.0 _ll_register_callbacks cb 0x562659182fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:40:28.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.458+0000 7f11c0c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562659296b40 0x5626592a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:28.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.458+0000 7f11c0c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562659296b40 0x5626592a0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37080/0 (socket says 172.21.15.53:37080) 2024-04-25T14:40:28.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.458+0000 7f11c0c00640 1 -- 172.21.15.53:0/2841345610 learned_addr learned my addr 172.21.15.53:0/2841345610 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:28.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.458+0000 7f11c2000640 1 --2- 172.21.15.53:0/2841345610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562659296150 0x5626592a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:28.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.458+0000 7f11c2000640 1 --2- 172.21.15.53:0/2841345610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56265929bcb0 0x5626592a7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:28.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.458+0000 7f11c0c00640 1 -- 172.21.15.53:0/2841345610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562659296150 msgr2=0x5626592a02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:40:28.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.458+0000 7f11c0c00640 1 --2- 172.21.15.53:0/2841345610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562659296150 0x5626592a02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.458+0000 7f11c0c00640 1 -- 172.21.15.53:0/2841345610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56265929bcb0 msgr2=0x5626592a7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:40:28.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.458+0000 7f11c0c00640 1 --2- 172.21.15.53:0/2841345610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56265929bcb0 0x5626592a7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.458+0000 7f11c0c00640 1 -- 172.21.15.53:0/2841345610 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562658fd76b0 con 0x562659296b40 2024-04-25T14:40:28.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.458+0000 7f11c2000640 1 --2- 172.21.15.53:0/2841345610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56265929bcb0 0x5626592a7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:40:28.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.458+0000 7f11c0c00640 1 --2- 172.21.15.53:0/2841345610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562659296b40 0x5626592a0830 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f11a8012720 tx=0x7f11a8012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d579acf678b18e37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:28.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.458+0000 7f11a2a00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-04-25T14:40:28.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.459+0000 7f11a2a00640 1 -- 172.21.15.53:0/2841345610 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a800f8c0 con 0x562659296b40 2024-04-25T14:40:28.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.459+0000 7f11a2a00640 1 -- 172.21.15.53:0/2841345610 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8285+0+0 (secure 0 0 0) 0x7f11a800fa60 con 0x562659296b40 2024-04-25T14:40:28.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.459+0000 7f11c6fe5740 1 -- 172.21.15.53:0/2841345610 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11b4036030 con 0x562659296b40 2024-04-25T14:40:28.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.459+0000 7f11a2a00640 1 -- 172.21.15.53:0/2841345610 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f11a800fca0 con 0x562659296b40 2024-04-25T14:40:28.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.459+0000 7f11c6fe5740 10 client.15036 fetch_fsmap learned FSMap version 21 2024-04-25T14:40:28.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.459+0000 7f11c6fe5740 1 -- 172.21.15.53:0/2841345610 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5626592aac20 con 0x562659296b40 2024-04-25T14:40:28.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.459+0000 7f11a2a00640 1 -- 172.21.15.53:0/2841345610 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11b4036030 con 0x562659296b40 2024-04-25T14:40:28.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.459+0000 7f11a2a00640 1 -- 172.21.15.53:0/2841345610 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f11a8025070 con 0x562659296b40 2024-04-25T14:40:28.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.460+0000 7f11c6fe5740 10 client.15036 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T14:40:28.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.460+0000 7f11c6fe5740 10 client.15036 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-25T14:40:28.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.460+0000 7f11c6fe5740 20 client.15036 populate_metadata read hostname 'smithi053' 2024-04-25T14:40:28.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.460+0000 7f11c6fe5740 1 --2- 172.21.15.53:0/2841345610 >> [v2:172.21.15.53:6838/570000540,v1:172.21.15.53:6839/570000540] conn(0x5626592ad2a0 0x5626592ae710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:28.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.460+0000 7f11c6fe5740 4 client.15036 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:40:28.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.460+0000 7f11c6fe5740 1 -- 172.21.15.53:0/2841345610 --> [v2:172.21.15.53:6838/570000540,v1:172.21.15.53:6839/570000540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562659208530 con 0x5626592ad2a0 2024-04-25T14:40:28.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.460+0000 7f11c2000640 1 --2- 172.21.15.53:0/2841345610 >> [v2:172.21.15.53:6838/570000540,v1:172.21.15.53:6839/570000540] conn(0x5626592ad2a0 0x5626592ae710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:28.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.460+0000 7f11c2000640 1 --2- 172.21.15.53:0/2841345610 >> [v2:172.21.15.53:6838/570000540,v1:172.21.15.53:6839/570000540] conn(0x5626592ad2a0 0x5626592ae710 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=24ed833eb6d194be server_cookie=22579140bca4a4ab in_seq=0 out_seq=0 2024-04-25T14:40:28.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.460+0000 7f11a2a00640 10 client.15036 ms_handle_connect on v2:172.21.15.53:6838/570000540 2024-04-25T14:40:28.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.461+0000 7f11a2a00640 1 -- 172.21.15.53:0/2841345610 <== mds.2 v2:172.21.15.53:6838/570000540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562659208530 con 0x5626592ad2a0 2024-04-25T14:40:28.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.461+0000 7f11a2a00640 10 client.15036 handle_command_reply: tid=0 2024-04-25T14:40:28.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.461+0000 7f11c6fe5740 1 client.15036 shutdown 2024-04-25T14:40:28.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.461+0000 7f11c6fe5740 1 -- 172.21.15.53:0/2841345610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562659296b40 msgr2=0x5626592a0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:28.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.462+0000 7f11c6fe5740 1 --2- 172.21.15.53:0/2841345610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562659296b40 0x5626592a0830 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f11a8012720 tx=0x7f11a8012be0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.462+0000 7f11c6fe5740 1 -- 172.21.15.53:0/2841345610 shutdown_connections 2024-04-25T14:40:28.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.462+0000 7f11c6fe5740 1 --2- 172.21.15.53:0/2841345610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56265929bcb0 0x5626592a7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.462+0000 7f11c6fe5740 1 --2- 172.21.15.53:0/2841345610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562659296b40 0x5626592a0830 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.462+0000 7f11c6fe5740 1 --2- 172.21.15.53:0/2841345610 >> [v2:172.21.15.53:6838/570000540,v1:172.21.15.53:6839/570000540] conn(0x5626592ad2a0 0x5626592ae710 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.462+0000 7f11c6fe5740 1 --2- 172.21.15.53:0/2841345610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562659296150 0x5626592a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.462+0000 7f11c6fe5740 1 -- 172.21.15.53:0/2841345610 >> 172.21.15.53:0/2841345610 conn(0x5626592917c0 msgr2=0x562659290a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:28.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.462+0000 7f11a2a00640 0 client.15036 ms_handle_reset on v2:172.21.15.53:6838/570000540 2024-04-25T14:40:28.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.462+0000 7f11c6fe5740 1 -- 172.21.15.53:0/2841345610 shutdown_connections 2024-04-25T14:40:28.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.462+0000 7f11c6fe5740 1 -- 172.21.15.53:0/2841345610 wait complete. 2024-04-25T14:40:28.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.462+0000 7f11c6fe5740 20 client.15036 trim_cache size 0 max 16384 2024-04-25T14:40:28.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.482+0000 7f11c6fe5740 1 Processor -- start 2024-04-25T14:40:28.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.482+0000 7f11c6fe5740 1 -- start start 2024-04-25T14:40:28.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.482+0000 7f11c6fe5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562659296150 0x56265929b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:28.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.482+0000 7f11c6fe5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562659296b40 0x5626592a7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:28.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.482+0000 7f11c6fe5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56265929bcb0 0x5626592944a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:28.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.482+0000 7f11c6fe5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5626592aca50 con 0x56265929bcb0 2024-04-25T14:40:28.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.482+0000 7f11c6fe5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5626592ad120 con 0x562659296b40 2024-04-25T14:40:28.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.482+0000 7f11c6fe5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5626592ad7f0 con 0x562659296150 2024-04-25T14:40:28.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.482+0000 7f11c0c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562659296b40 0x5626592a7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:28.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.482+0000 7f11c0c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562659296b40 0x5626592a7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.53:40724/0 (socket says 172.21.15.53:40724) 2024-04-25T14:40:28.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.482+0000 7f11c0c00640 1 -- 172.21.15.53:0/952460787 learned_addr learned my addr 172.21.15.53:0/952460787 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:28.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.482+0000 7f11c2000640 1 --2- 172.21.15.53:0/952460787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56265929bcb0 0x5626592944a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:28.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.482+0000 7f11c2000640 1 --2- 172.21.15.53:0/952460787 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562659296150 0x56265929b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:28.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.483+0000 7f11c0c00640 1 -- 172.21.15.53:0/952460787 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562659296150 msgr2=0x56265929b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:40:28.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.483+0000 7f11c0c00640 1 --2- 172.21.15.53:0/952460787 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562659296150 0x56265929b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.483+0000 7f11c0c00640 1 -- 172.21.15.53:0/952460787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56265929bcb0 msgr2=0x5626592944a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:40:28.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.483+0000 7f11c0c00640 1 --2- 172.21.15.53:0/952460787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56265929bcb0 0x5626592944a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.483+0000 7f11c0c00640 1 -- 172.21.15.53:0/952460787 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56265928bef0 con 0x562659296b40 2024-04-25T14:40:28.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.483+0000 7f11c2000640 1 --2- 172.21.15.53:0/952460787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56265929bcb0 0x5626592944a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:40:28.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.483+0000 7f11c0c00640 1 --2- 172.21.15.53:0/952460787 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562659296b40 0x5626592a7830 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f11a8009e20 tx=0x7f11a80096f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0b3b595e9e1b656 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:28.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.483+0000 7f11adc00640 1 -- 172.21.15.53:0/952460787 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a800f850 con 0x562659296b40 2024-04-25T14:40:28.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.483+0000 7f11adc00640 1 -- 172.21.15.53:0/952460787 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11a8022070 con 0x562659296b40 2024-04-25T14:40:28.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.483+0000 7f11adc00640 1 -- 172.21.15.53:0/952460787 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a8005750 con 0x562659296b40 2024-04-25T14:40:28.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.484+0000 7f11c6fe5740 1 -- 172.21.15.53:0/952460787 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562659296b40 msgr2=0x5626592a7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:28.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.484+0000 7f11c6fe5740 1 --2- 172.21.15.53:0/952460787 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562659296b40 0x5626592a7830 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f11a8009e20 tx=0x7f11a80096f0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.484+0000 7f11c6fe5740 1 -- 172.21.15.53:0/952460787 shutdown_connections 2024-04-25T14:40:28.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.484+0000 7f11c6fe5740 1 --2- 172.21.15.53:0/952460787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56265929bcb0 0x5626592944a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.484+0000 7f11c6fe5740 1 --2- 172.21.15.53:0/952460787 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562659296b40 0x5626592a7830 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.484+0000 7f11c6fe5740 1 --2- 172.21.15.53:0/952460787 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562659296150 0x56265929b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.484+0000 7f11c6fe5740 1 -- 172.21.15.53:0/952460787 >> 172.21.15.53:0/952460787 conn(0x5626592917c0 msgr2=0x562659291ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:28.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.484+0000 7f11c6fe5740 1 -- 172.21.15.53:0/952460787 shutdown_connections 2024-04-25T14:40:28.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.484+0000 7f11c6fe5740 1 -- 172.21.15.53:0/952460787 wait complete. 2024-04-25T14:40:28.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.484+0000 7f11c6fe5740 1 -- start start 2024-04-25T14:40:28.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.484+0000 7f11c6fe5740 1 Processor -- start 2024-04-25T14:40:28.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.485+0000 7f11c6fe5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562659296150 0x5626592a59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:28.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.485+0000 7f11c6fe5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56265929bcb0 0x5626592a5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:28.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.485+0000 7f11c6fe5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626592a6450 0x5626592a68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:28.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.485+0000 7f11c6fe5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5626592ae590 con 0x562659296150 2024-04-25T14:40:28.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.485+0000 7f11c6fe5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5626592adec0 con 0x5626592a6450 2024-04-25T14:40:28.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.485+0000 7f11c6fe5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5626592aec60 con 0x56265929bcb0 2024-04-25T14:40:28.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.485+0000 7f11c6fe5740 10 client.0 _ll_register_callbacks cb 0x7f11a8009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:40:28.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.485+0000 7f11c2000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562659296150 0x5626592a59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:28.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.485+0000 7f11c2000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562659296150 0x5626592a59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37096/0 (socket says 172.21.15.53:37096) 2024-04-25T14:40:28.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.485+0000 7f11c2000640 1 -- 172.21.15.53:0/2226526338 learned_addr learned my addr 172.21.15.53:0/2226526338 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:28.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.485+0000 7f11c0c00640 1 --2- 172.21.15.53:0/2226526338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56265929bcb0 0x5626592a5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:28.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.485+0000 7f11c2000640 1 --2- 172.21.15.53:0/2226526338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626592a6450 0x5626592a68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:28.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.486+0000 7f11c0c00640 1 -- 172.21.15.53:0/2226526338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626592a6450 msgr2=0x5626592a68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:40:28.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.486+0000 7f11c0c00640 1 --2- 172.21.15.53:0/2226526338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626592a6450 0x5626592a68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.486+0000 7f11c0c00640 1 -- 172.21.15.53:0/2226526338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562659296150 msgr2=0x5626592a59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:40:28.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.486+0000 7f11c0c00640 1 --2- 172.21.15.53:0/2226526338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562659296150 0x5626592a59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.486+0000 7f11c0c00640 1 -- 172.21.15.53:0/2226526338 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626592a70b0 con 0x56265929bcb0 2024-04-25T14:40:28.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.486+0000 7f11c0c00640 1 --2- 172.21.15.53:0/2226526338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56265929bcb0 0x5626592a5f10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x5626592ade60 tx=0x7f11a8005660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b370c3f8930739c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:28.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.486+0000 7f11a2a00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T14:40:28.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.486+0000 7f11a2a00640 1 -- 172.21.15.53:0/2226526338 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a800fe40 con 0x56265929bcb0 2024-04-25T14:40:28.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.486+0000 7f11a2a00640 1 -- 172.21.15.53:0/2226526338 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8285+0+0 (secure 0 0 0) 0x7f11a8010b50 con 0x56265929bcb0 2024-04-25T14:40:28.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.486+0000 7f11c6fe5740 1 -- 172.21.15.53:0/2226526338 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11a8022070 con 0x56265929bcb0 2024-04-25T14:40:28.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.487+0000 7f11a2a00640 1 -- 172.21.15.53:0/2226526338 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f11a8010d50 con 0x56265929bcb0 2024-04-25T14:40:28.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.487+0000 7f11c6fe5740 10 client.25174 fetch_fsmap learned FSMap version 21 2024-04-25T14:40:28.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.487+0000 7f11c6fe5740 1 -- 172.21.15.53:0/2226526338 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x562659313890 con 0x56265929bcb0 2024-04-25T14:40:28.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.487+0000 7f11a2a00640 1 -- 172.21.15.53:0/2226526338 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11a8022070 con 0x56265929bcb0 2024-04-25T14:40:28.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.487+0000 7f11a2a00640 1 -- 172.21.15.53:0/2226526338 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f11a8025070 con 0x56265929bcb0 2024-04-25T14:40:28.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.487+0000 7f11c6fe5740 10 client.25174 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T14:40:28.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.487+0000 7f11c6fe5740 10 client.25174 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-25T14:40:28.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.487+0000 7f11c6fe5740 20 client.25174 populate_metadata read hostname 'smithi053' 2024-04-25T14:40:28.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.487+0000 7f11c6fe5740 1 --2- 172.21.15.53:0/2226526338 >> [v2:172.21.15.53:6838/570000540,v1:172.21.15.53:6839/570000540] conn(0x562659318110 0x562659316e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:28.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.487+0000 7f11c6fe5740 4 client.25174 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T14:40:28.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.487+0000 7f11c6fe5740 1 -- 172.21.15.53:0/2226526338 --> [v2:172.21.15.53:6838/570000540,v1:172.21.15.53:6839/570000540] -- command(tid 0: {"prefix": "status"}) -- 0x5626592023a0 con 0x562659318110 2024-04-25T14:40:28.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.488+0000 7f11c2000640 1 --2- 172.21.15.53:0/2226526338 >> [v2:172.21.15.53:6838/570000540,v1:172.21.15.53:6839/570000540] conn(0x562659318110 0x562659316e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:28.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.488+0000 7f11c2000640 1 --2- 172.21.15.53:0/2226526338 >> [v2:172.21.15.53:6838/570000540,v1:172.21.15.53:6839/570000540] conn(0x562659318110 0x562659316e20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a8e58b8d48c63831 server_cookie=4f22f1ad41c6bac7 in_seq=0 out_seq=0 2024-04-25T14:40:28.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.488+0000 7f11a2a00640 10 client.25174 ms_handle_connect on v2:172.21.15.53:6838/570000540 2024-04-25T14:40:28.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.488+0000 7f11a2a00640 1 -- 172.21.15.53:0/2226526338 <== mds.2 v2:172.21.15.53:6838/570000540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5626592023a0 con 0x562659318110 2024-04-25T14:40:28.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.488+0000 7f11a2a00640 10 client.25174 handle_command_reply: tid=0 2024-04-25T14:40:28.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.488+0000 7f11c6fe5740 1 client.25174 shutdown 2024-04-25T14:40:28.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.488+0000 7f11c6fe5740 1 -- 172.21.15.53:0/2226526338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56265929bcb0 msgr2=0x5626592a5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:28.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.488+0000 7f11c6fe5740 1 --2- 172.21.15.53:0/2226526338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56265929bcb0 0x5626592a5f10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x5626592ade60 tx=0x7f11a8005660 comp rx=0 tx=0).stop 2024-04-25T14:40:28.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.489+0000 7f11c6fe5740 1 -- 172.21.15.53:0/2226526338 shutdown_connections 2024-04-25T14:40:28.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.489+0000 7f11c6fe5740 1 --2- 172.21.15.53:0/2226526338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626592a6450 0x5626592a68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.489+0000 7f11c6fe5740 1 --2- 172.21.15.53:0/2226526338 >> [v2:172.21.15.53:6838/570000540,v1:172.21.15.53:6839/570000540] conn(0x562659318110 0x562659316e20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.489+0000 7f11c6fe5740 1 --2- 172.21.15.53:0/2226526338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56265929bcb0 0x5626592a5f10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.489+0000 7f11c6fe5740 1 --2- 172.21.15.53:0/2226526338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562659296150 0x5626592a59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.489+0000 7f11a2a00640 0 client.25174 ms_handle_reset on v2:172.21.15.53:6838/570000540 2024-04-25T14:40:28.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.489+0000 7f11c6fe5740 1 -- 172.21.15.53:0/2226526338 >> 172.21.15.53:0/2226526338 conn(0x5626592917c0 msgr2=0x562659297dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:28.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.489+0000 7f11c6fe5740 1 -- 172.21.15.53:0/2226526338 shutdown_connections 2024-04-25T14:40:28.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.489+0000 7f11c6fe5740 1 -- 172.21.15.53:0/2226526338 wait complete. 2024-04-25T14:40:28.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.489+0000 7f11c6fe5740 20 client.25174 trim_cache size 0 max 16384 2024-04-25T14:40:28.494 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-04-25T14:40:28.495 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "fd30e6ce-030f-11ef-bc93-c7b262605968", 2024-04-25T14:40:28.495 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 2, 2024-04-25T14:40:28.495 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 24560, 2024-04-25T14:40:28.495 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2024-04-25T14:40:28.495 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2024-04-25T14:40:28.495 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2024-04-25T14:40:28.495 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 133.47590973000001, 2024-04-25T14:40:28.495 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 21, 2024-04-25T14:40:28.495 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 92, 2024-04-25T14:40:28.495 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 88, 2024-04-25T14:40:28.495 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 150.32051196099999 2024-04-25T14:40:28.495 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-04-25T14:40:28.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.489+0000 7f11c2a00640 1 -- 172.21.15.53:0/496162625 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f11a40a3bb0 msgr2=0x7f11a40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:28.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.489+0000 7f11c2a00640 1 --2- 172.21.15.53:0/496162625 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f11a40a3bb0 0x7f11a40a6070 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f11b00ce760 tx=0x7f11b0003e70 comp rx=0 tx=0).stop 2024-04-25T14:40:28.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.489+0000 7f11c2a00640 1 -- 172.21.15.53:0/496162625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11bc104090 msgr2=0x7f11bc19a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:28.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.489+0000 7f11c2a00640 1 --2- 172.21.15.53:0/496162625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11bc104090 0x7f11bc19a270 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f11b000e9c0 tx=0x7f11b000ee80 comp rx=0 tx=0).stop 2024-04-25T14:40:28.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.490+0000 7f11c2a00640 1 -- 172.21.15.53:0/496162625 shutdown_connections 2024-04-25T14:40:28.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.490+0000 7f11c2a00640 1 --2- 172.21.15.53:0/496162625 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f11a40a3bb0 0x7f11a40a6070 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.490+0000 7f11c2a00640 1 --2- 172.21.15.53:0/496162625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11bc105c40 0x7f11bc1a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.490+0000 7f11c2a00640 1 --2- 172.21.15.53:0/496162625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11bc105280 0x7f11bc19a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.490+0000 7f11c2a00640 1 --2- 172.21.15.53:0/496162625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11bc104090 0x7f11bc19a270 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.490+0000 7f11c2a00640 1 -- 172.21.15.53:0/496162625 >> 172.21.15.53:0/496162625 conn(0x7f11bc0ff600 msgr2=0x7f11bc108580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:28.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.490+0000 7f11c2a00640 1 -- 172.21.15.53:0/496162625 shutdown_connections 2024-04-25T14:40:28.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.490+0000 7f11c2a00640 1 -- 172.21.15.53:0/496162625 wait complete. 2024-04-25T14:40:28.509 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-25T14:40:28.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.726+0000 7f79ecc00640 1 -- 172.21.15.53:0/1922973809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79e8102fd0 msgr2=0x7f79e8103450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:28.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.726+0000 7f79ecc00640 1 --2- 172.21.15.53:0/1922973809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79e8102fd0 0x7f79e8103450 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f79d0009900 tx=0x7f79d002a3f0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.726+0000 7f79ecc00640 1 -- 172.21.15.53:0/1922973809 shutdown_connections 2024-04-25T14:40:28.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.726+0000 7f79ecc00640 1 --2- 172.21.15.53:0/1922973809 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79e8103990 0x7f79e810a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.726+0000 7f79ecc00640 1 --2- 172.21.15.53:0/1922973809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79e8102fd0 0x7f79e8103450 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.726+0000 7f79ecc00640 1 --2- 172.21.15.53:0/1922973809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79e8101e80 0x7f79e81022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.726+0000 7f79ecc00640 1 -- 172.21.15.53:0/1922973809 >> 172.21.15.53:0/1922973809 conn(0x7f79e80fd3f0 msgr2=0x7f79e80ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:28.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.726+0000 7f79ecc00640 1 -- 172.21.15.53:0/1922973809 shutdown_connections 2024-04-25T14:40:28.727 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.726+0000 7f79ecc00640 1 -- 172.21.15.53:0/1922973809 wait complete. 2024-04-25T14:40:28.727 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.728+0000 7f79ecc00640 1 Processor -- start 2024-04-25T14:40:28.727 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.728+0000 7f79ecc00640 1 -- start start 2024-04-25T14:40:28.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.728+0000 7f79ecc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79e8101e80 0x7f79e819a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:28.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.728+0000 7f79ecc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79e8102fd0 0x7f79e819a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:28.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.728+0000 7f79ecc00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79e8103990 0x7f79e81a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:28.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.728+0000 7f79ecc00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f79e810da10 con 0x7f79e8101e80 2024-04-25T14:40:28.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.728+0000 7f79ecc00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f79e810d890 con 0x7f79e8103990 2024-04-25T14:40:28.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.728+0000 7f79ecc00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f79e810db90 con 0x7f79e8102fd0 2024-04-25T14:40:28.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.729+0000 7f79e7400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79e8101e80 0x7f79e819a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:28.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.729+0000 7f79e7400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79e8101e80 0x7f79e819a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37120/0 (socket says 172.21.15.53:37120) 2024-04-25T14:40:28.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.729+0000 7f79e7400640 1 -- 172.21.15.53:0/499972915 learned_addr learned my addr 172.21.15.53:0/499972915 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:28.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.729+0000 7f79e6a00640 1 --2- 172.21.15.53:0/499972915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79e8102fd0 0x7f79e819a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:28.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.729+0000 7f79e7e00640 1 --2- 172.21.15.53:0/499972915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79e8103990 0x7f79e81a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:28.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.729+0000 7f79e7400640 1 -- 172.21.15.53:0/499972915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79e8102fd0 msgr2=0x7f79e819a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:28.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.729+0000 7f79e7400640 1 --2- 172.21.15.53:0/499972915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79e8102fd0 0x7f79e819a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.729+0000 7f79e7400640 1 -- 172.21.15.53:0/499972915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79e8103990 msgr2=0x7f79e81a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:28.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.729+0000 7f79e7400640 1 --2- 172.21.15.53:0/499972915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79e8103990 0x7f79e81a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.729+0000 7f79e7400640 1 -- 172.21.15.53:0/499972915 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79e81a1f50 con 0x7f79e8101e80 2024-04-25T14:40:28.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.729+0000 7f79e7e00640 1 --2- 172.21.15.53:0/499972915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79e8103990 0x7f79e81a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:40:28.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.729+0000 7f79e7400640 1 --2- 172.21.15.53:0/499972915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79e8101e80 0x7f79e819a330 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f79d800e9f0 tx=0x7f79d800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:28.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.730+0000 7f79dfe00640 1 -- 172.21.15.53:0/499972915 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d800cd80 con 0x7f79e8101e80 2024-04-25T14:40:28.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.730+0000 7f79dfe00640 1 -- 172.21.15.53:0/499972915 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79d80108e0 con 0x7f79e8101e80 2024-04-25T14:40:28.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.730+0000 7f79ecc00640 1 -- 172.21.15.53:0/499972915 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79e81a2240 con 0x7f79e8101e80 2024-04-25T14:40:28.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.730+0000 7f79dfe00640 1 -- 172.21.15.53:0/499972915 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d8018980 con 0x7f79e8101e80 2024-04-25T14:40:28.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.730+0000 7f79ecc00640 1 -- 172.21.15.53:0/499972915 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79e81a27b0 con 0x7f79e8101e80 2024-04-25T14:40:28.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.733+0000 7f79f0fb3740 1 Processor -- start 2024-04-25T14:40:28.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.733+0000 7f79dfe00640 1 -- 172.21.15.53:0/499972915 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f79d8010430 con 0x7f79e8101e80 2024-04-25T14:40:28.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.734+0000 7f79f0fb3740 1 -- start start 2024-04-25T14:40:28.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.734+0000 7f79f0fb3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc87357150 0x55cc87357570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:28.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.734+0000 7f79f0fb3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cc87357b40 0x55cc8735c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:28.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.734+0000 7f79f0fb3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc8735ccb0 0x55cc8735f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:28.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.734+0000 7f79f0fb3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55cc872bbb10 con 0x55cc8735ccb0 2024-04-25T14:40:28.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.734+0000 7f79f0fb3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55cc872622b0 con 0x55cc87357b40 2024-04-25T14:40:28.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.734+0000 7f79f0fb3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cc872b7ad0 con 0x55cc87357150 2024-04-25T14:40:28.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.734+0000 7f79e7400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cc87357b40 0x55cc8735c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:28.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.734+0000 7f79e6a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc8735ccb0 0x55cc8735f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:28.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.734+0000 7f79e6a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc8735ccb0 0x55cc8735f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37122/0 (socket says 172.21.15.53:37122) 2024-04-25T14:40:28.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.734+0000 7f79e6a00640 1 -- 172.21.15.53:0/2477819641 learned_addr learned my addr 172.21.15.53:0/2477819641 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:28.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.734+0000 7f79e7e00640 1 --2- 172.21.15.53:0/2477819641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc87357150 0x55cc87357570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:28.738 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.735+0000 7f79e6a00640 1 -- 172.21.15.53:0/2477819641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc87357150 msgr2=0x55cc87357570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:40:28.738 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.735+0000 7f79e6a00640 1 --2- 172.21.15.53:0/2477819641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc87357150 0x55cc87357570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.738 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.735+0000 7f79e6a00640 1 -- 172.21.15.53:0/2477819641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cc87357b40 msgr2=0x55cc8735c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:40:28.738 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.735+0000 7f79e6a00640 1 --2- 172.21.15.53:0/2477819641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cc87357b40 0x55cc8735c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.738 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.735+0000 7f79e6a00640 1 -- 172.21.15.53:0/2477819641 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc86fce2d0 con 0x55cc8735ccb0 2024-04-25T14:40:28.738 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.735+0000 7f79e6a00640 1 --2- 172.21.15.53:0/2477819641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc8735ccb0 0x55cc8735f140 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f79d0006010 tx=0x7f79d0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584e09e575a0ae04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:28.738 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.735+0000 7f79dfe00640 1 --2- 172.21.15.53:0/499972915 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f79cc0a3c30 0x7f79cc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:28.738 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.735+0000 7f79d7e00640 1 -- 172.21.15.53:0/2477819641 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d0035030 con 0x55cc8735ccb0 2024-04-25T14:40:28.738 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.735+0000 7f79d7e00640 1 -- 172.21.15.53:0/2477819641 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79d0040070 con 0x55cc8735ccb0 2024-04-25T14:40:28.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.735+0000 7f79d7e00640 1 -- 172.21.15.53:0/2477819641 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d0031890 con 0x55cc8735ccb0 2024-04-25T14:40:28.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.736+0000 7f79dfe00640 1 -- 172.21.15.53:0/499972915 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8285+0+0 (secure 0 0 0) 0x7f79d80d0800 con 0x7f79e8101e80 2024-04-25T14:40:28.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.736+0000 7f79e7e00640 1 --2- 172.21.15.53:0/499972915 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f79cc0a3c30 0x7f79cc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:28.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.736+0000 7f79f0fb3740 1 -- 172.21.15.53:0/2477819641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc8735ccb0 msgr2=0x55cc8735f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:28.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.736+0000 7f79f0fb3740 1 --2- 172.21.15.53:0/2477819641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc8735ccb0 0x55cc8735f140 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f79d0006010 tx=0x7f79d0004040 comp rx=0 tx=0).stop 2024-04-25T14:40:28.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.736+0000 7f79f0fb3740 1 -- 172.21.15.53:0/2477819641 shutdown_connections 2024-04-25T14:40:28.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.736+0000 7f79f0fb3740 1 --2- 172.21.15.53:0/2477819641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc8735ccb0 0x55cc8735f140 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.736+0000 7f79f0fb3740 1 --2- 172.21.15.53:0/2477819641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cc87357b40 0x55cc8735c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.736+0000 7f79f0fb3740 1 --2- 172.21.15.53:0/2477819641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc87357150 0x55cc87357570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.736+0000 7f79f0fb3740 1 -- 172.21.15.53:0/2477819641 >> 172.21.15.53:0/2477819641 conn(0x55cc873527c0 msgr2=0x55cc87352bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:28.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.736+0000 7f79e7e00640 1 --2- 172.21.15.53:0/499972915 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f79cc0a3c30 0x7f79cc0a60f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x55cc873583a0 tx=0x7f79e0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:28.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.736+0000 7f79f0fb3740 1 -- 172.21.15.53:0/2477819641 shutdown_connections 2024-04-25T14:40:28.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.736+0000 7f79f0fb3740 1 -- 172.21.15.53:0/2477819641 wait complete. 2024-04-25T14:40:28.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.737+0000 7f79f0fb3740 1 -- start start 2024-04-25T14:40:28.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.737+0000 7f79f0fb3740 1 Processor -- start 2024-04-25T14:40:28.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.738+0000 7f79f0fb3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc87357150 0x55cc87361360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:28.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.738+0000 7f79f0fb3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc87357b40 0x55cc873618a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:28.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.738+0000 7f79f0fb3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cc8735ccb0 0x55cc873688e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:28.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.738+0000 7f79f0fb3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55cc8735fc00 con 0x55cc87357150 2024-04-25T14:40:28.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.738+0000 7f79f0fb3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55cc87275570 con 0x55cc8735ccb0 2024-04-25T14:40:28.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.738+0000 7f79f0fb3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cc8735fd80 con 0x55cc87357b40 2024-04-25T14:40:28.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.738+0000 7f79f0fb3740 10 client.0 _ll_register_callbacks cb 0x55cc87243fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:40:28.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.738+0000 7f79e7400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc87357150 0x55cc87361360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:28.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.738+0000 7f79e7400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc87357150 0x55cc87361360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37138/0 (socket says 172.21.15.53:37138) 2024-04-25T14:40:28.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.738+0000 7f79e7400640 1 -- 172.21.15.53:0/1814423396 learned_addr learned my addr 172.21.15.53:0/1814423396 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:28.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.738+0000 7f79e6a00640 1 --2- 172.21.15.53:0/1814423396 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc87357b40 0x55cc873618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:28.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.738+0000 7f79e7e00640 1 --2- 172.21.15.53:0/1814423396 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cc8735ccb0 0x55cc873688e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:28.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.738+0000 7f79e7400640 1 -- 172.21.15.53:0/1814423396 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc87357b40 msgr2=0x55cc873618a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:40:28.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.738+0000 7f79e7400640 1 --2- 172.21.15.53:0/1814423396 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc87357b40 0x55cc873618a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.738+0000 7f79e7400640 1 -- 172.21.15.53:0/1814423396 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cc8735ccb0 msgr2=0x55cc873688e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:40:28.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.738+0000 7f79e7400640 1 --2- 172.21.15.53:0/1814423396 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cc8735ccb0 0x55cc873688e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.738+0000 7f79e7400640 1 -- 172.21.15.53:0/1814423396 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc870986b0 con 0x55cc87357150 2024-04-25T14:40:28.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.739+0000 7f79e7400640 1 --2- 172.21.15.53:0/1814423396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc87357150 0x55cc87361360 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f79d8099420 tx=0x7f79d8018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4108954ab0e19057 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:28.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.739+0000 7f79d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-04-25T14:40:28.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.739+0000 7f79d4c00640 1 -- 172.21.15.53:0/1814423396 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d8010720 con 0x55cc87357150 2024-04-25T14:40:28.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.739+0000 7f79d4c00640 1 -- 172.21.15.53:0/1814423396 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8285+0+0 (secure 0 0 0) 0x7f79d809bea0 con 0x55cc87357150 2024-04-25T14:40:28.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.739+0000 7f79f0fb3740 1 -- 172.21.15.53:0/1814423396 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79d0040070 con 0x55cc87357150 2024-04-25T14:40:28.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.739+0000 7f79d4c00640 1 -- 172.21.15.53:0/1814423396 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f79d80bd0b0 con 0x55cc87357150 2024-04-25T14:40:28.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.739+0000 7f79f0fb3740 10 client.15054 fetch_fsmap learned FSMap version 21 2024-04-25T14:40:28.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.739+0000 7f79f0fb3740 1 -- 172.21.15.53:0/1814423396 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55cc8736bcc0 con 0x55cc87357150 2024-04-25T14:40:28.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.739+0000 7f79d4c00640 1 -- 172.21.15.53:0/1814423396 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79d0040070 con 0x55cc87357150 2024-04-25T14:40:28.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.740+0000 7f79d4c00640 1 -- 172.21.15.53:0/1814423396 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f79d8087110 con 0x55cc87357150 2024-04-25T14:40:28.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.740+0000 7f79f0fb3740 10 client.15054 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T14:40:28.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.740+0000 7f79f0fb3740 10 client.15054 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T14:40:28.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.740+0000 7f79f0fb3740 20 client.15054 populate_metadata read hostname 'smithi053' 2024-04-25T14:40:28.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.740+0000 7f79f0fb3740 1 --2- 172.21.15.53:0/1814423396 >> [v2:172.21.15.183:6838/1554258915,v1:172.21.15.183:6839/1554258915] conn(0x55cc8736e380 0x55cc8736f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:28.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.740+0000 7f79f0fb3740 4 client.15054 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:40:28.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.740+0000 7f79f0fb3740 1 -- 172.21.15.53:0/1814423396 --> [v2:172.21.15.183:6838/1554258915,v1:172.21.15.183:6839/1554258915] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc872c9530 con 0x55cc8736e380 2024-04-25T14:40:28.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.740+0000 7f79e6a00640 1 --2- 172.21.15.53:0/1814423396 >> [v2:172.21.15.183:6838/1554258915,v1:172.21.15.183:6839/1554258915] conn(0x55cc8736e380 0x55cc8736f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:28.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.741+0000 7f79e6a00640 1 --2- 172.21.15.53:0/1814423396 >> [v2:172.21.15.183:6838/1554258915,v1:172.21.15.183:6839/1554258915] conn(0x55cc8736e380 0x55cc8736f7f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d229ac979360d25b server_cookie=f46514f85ef70628 in_seq=0 out_seq=0 2024-04-25T14:40:28.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.741+0000 7f79d4c00640 10 client.15054 ms_handle_connect on v2:172.21.15.183:6838/1554258915 2024-04-25T14:40:28.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.742+0000 7f79d4c00640 1 -- 172.21.15.53:0/1814423396 <== mds.1 v2:172.21.15.183:6838/1554258915 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cc872c9530 con 0x55cc8736e380 2024-04-25T14:40:28.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.742+0000 7f79d4c00640 10 client.15054 handle_command_reply: tid=0 2024-04-25T14:40:28.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.742+0000 7f79f0fb3740 1 client.15054 shutdown 2024-04-25T14:40:28.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.742+0000 7f79f0fb3740 1 -- 172.21.15.53:0/1814423396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc87357150 msgr2=0x55cc87361360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:28.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.742+0000 7f79f0fb3740 1 --2- 172.21.15.53:0/1814423396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc87357150 0x55cc87361360 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f79d8099420 tx=0x7f79d8018420 comp rx=0 tx=0).stop 2024-04-25T14:40:28.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.742+0000 7f79f0fb3740 1 -- 172.21.15.53:0/1814423396 shutdown_connections 2024-04-25T14:40:28.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.742+0000 7f79f0fb3740 1 --2- 172.21.15.53:0/1814423396 >> [v2:172.21.15.183:6838/1554258915,v1:172.21.15.183:6839/1554258915] conn(0x55cc8736e380 0x55cc8736f7f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.742+0000 7f79f0fb3740 1 --2- 172.21.15.53:0/1814423396 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cc8735ccb0 0x55cc873688e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.742+0000 7f79f0fb3740 1 --2- 172.21.15.53:0/1814423396 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc87357b40 0x55cc873618a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.742+0000 7f79f0fb3740 1 --2- 172.21.15.53:0/1814423396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc87357150 0x55cc87361360 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.742+0000 7f79f0fb3740 1 -- 172.21.15.53:0/1814423396 >> 172.21.15.53:0/1814423396 conn(0x55cc873527c0 msgr2=0x55cc87351a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:28.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.742+0000 7f79d4c00640 0 client.15054 ms_handle_reset on v2:172.21.15.183:6838/1554258915 2024-04-25T14:40:28.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.742+0000 7f79f0fb3740 1 -- 172.21.15.53:0/1814423396 shutdown_connections 2024-04-25T14:40:28.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.742+0000 7f79f0fb3740 1 -- 172.21.15.53:0/1814423396 wait complete. 2024-04-25T14:40:28.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.742+0000 7f79f0fb3740 20 client.15054 trim_cache size 0 max 16384 2024-04-25T14:40:28.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.763+0000 7f79f0fb3740 1 Processor -- start 2024-04-25T14:40:28.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.764+0000 7f79f0fb3740 1 -- start start 2024-04-25T14:40:28.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.764+0000 7f79f0fb3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc87357150 0x55cc873583d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:28.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.764+0000 7f79f0fb3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cc87357b40 0x55cc87368870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:28.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.764+0000 7f79f0fb3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc8735ccb0 0x55cc8734d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:28.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.764+0000 7f79f0fb3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55cc8736db30 con 0x55cc87357150 2024-04-25T14:40:28.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.764+0000 7f79f0fb3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55cc8736e200 con 0x55cc87357b40 2024-04-25T14:40:28.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.764+0000 7f79f0fb3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cc8736e8d0 con 0x55cc8735ccb0 2024-04-25T14:40:28.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.764+0000 7f79e7400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc87357150 0x55cc873583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:28.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.764+0000 7f79e6a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cc87357b40 0x55cc87368870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:28.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.765+0000 7f79e7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc8735ccb0 0x55cc8734d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:28.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.765+0000 7f79e7400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc87357150 0x55cc873583d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37150/0 (socket says 172.21.15.53:37150) 2024-04-25T14:40:28.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.765+0000 7f79e7400640 1 -- 172.21.15.53:0/2527015190 learned_addr learned my addr 172.21.15.53:0/2527015190 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:28.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.765+0000 7f79e6a00640 1 -- 172.21.15.53:0/2527015190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc8735ccb0 msgr2=0x55cc8734d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:40:28.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.765+0000 7f79e6a00640 1 --2- 172.21.15.53:0/2527015190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc8735ccb0 0x55cc8734d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.765+0000 7f79e6a00640 1 -- 172.21.15.53:0/2527015190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc87357150 msgr2=0x55cc873583d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:40:28.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.765+0000 7f79e6a00640 1 --2- 172.21.15.53:0/2527015190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc87357150 0x55cc873583d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.765+0000 7f79e6a00640 1 -- 172.21.15.53:0/2527015190 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc8734cef0 con 0x55cc87357b40 2024-04-25T14:40:28.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.765+0000 7f79e6a00640 1 --2- 172.21.15.53:0/2527015190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cc87357b40 0x55cc87368870 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f79d002a9e0 tx=0x7f79d0004300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61552b7ab7e176f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:28.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.766+0000 7f79d7400640 1 -- 172.21.15.53:0/2527015190 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d0035030 con 0x55cc87357b40 2024-04-25T14:40:28.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.766+0000 7f79d7400640 1 -- 172.21.15.53:0/2527015190 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79d0040040 con 0x55cc87357b40 2024-04-25T14:40:28.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.766+0000 7f79d7400640 1 -- 172.21.15.53:0/2527015190 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d0003ea0 con 0x55cc87357b40 2024-04-25T14:40:28.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.766+0000 7f79f0fb3740 1 -- 172.21.15.53:0/2527015190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cc87357b40 msgr2=0x55cc87368870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:28.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.766+0000 7f79f0fb3740 1 --2- 172.21.15.53:0/2527015190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cc87357b40 0x55cc87368870 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f79d002a9e0 tx=0x7f79d0004300 comp rx=0 tx=0).stop 2024-04-25T14:40:28.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.766+0000 7f79f0fb3740 1 -- 172.21.15.53:0/2527015190 shutdown_connections 2024-04-25T14:40:28.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.766+0000 7f79f0fb3740 1 --2- 172.21.15.53:0/2527015190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc8735ccb0 0x55cc8734d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.766+0000 7f79f0fb3740 1 --2- 172.21.15.53:0/2527015190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cc87357b40 0x55cc87368870 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.766+0000 7f79f0fb3740 1 --2- 172.21.15.53:0/2527015190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc87357150 0x55cc873583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.766+0000 7f79f0fb3740 1 -- 172.21.15.53:0/2527015190 >> 172.21.15.53:0/2527015190 conn(0x55cc873527c0 msgr2=0x55cc87352ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:28.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.766+0000 7f79f0fb3740 1 -- 172.21.15.53:0/2527015190 shutdown_connections 2024-04-25T14:40:28.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.766+0000 7f79f0fb3740 1 -- 172.21.15.53:0/2527015190 wait complete. 2024-04-25T14:40:28.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.767+0000 7f79f0fb3740 1 -- start start 2024-04-25T14:40:28.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.767+0000 7f79f0fb3740 1 Processor -- start 2024-04-25T14:40:28.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.767+0000 7f79f0fb3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cc87357150 0x55cc8735aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:28.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.767+0000 7f79f0fb3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc87357b40 0x55cc8735b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:28.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.768+0000 7f79f0fb3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc8735ccb0 0x55cc873d2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:28.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.768+0000 7f79f0fb3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55cc8736f670 con 0x55cc8735ccb0 2024-04-25T14:40:28.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.768+0000 7f79f0fb3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55cc8736efa0 con 0x55cc87357150 2024-04-25T14:40:28.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.768+0000 7f79f0fb3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cc8736fd40 con 0x55cc87357b40 2024-04-25T14:40:28.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.768+0000 7f79f0fb3740 10 client.0 _ll_register_callbacks cb 0x7f79d8098cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:40:28.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.768+0000 7f79e7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc8735ccb0 0x55cc873d2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:28.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.768+0000 7f79e7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc8735ccb0 0x55cc873d2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37154/0 (socket says 172.21.15.53:37154) 2024-04-25T14:40:28.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.768+0000 7f79e7e00640 1 -- 172.21.15.53:0/2691266454 learned_addr learned my addr 172.21.15.53:0/2691266454 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:28.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.768+0000 7f79e6a00640 1 --2- 172.21.15.53:0/2691266454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc87357b40 0x55cc8735b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:28.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.768+0000 7f79e7400640 1 --2- 172.21.15.53:0/2691266454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cc87357150 0x55cc8735aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:28.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.768+0000 7f79e7400640 1 -- 172.21.15.53:0/2691266454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc87357b40 msgr2=0x55cc8735b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:40:28.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.768+0000 7f79e7400640 1 --2- 172.21.15.53:0/2691266454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc87357b40 0x55cc8735b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.768+0000 7f79e7400640 1 -- 172.21.15.53:0/2691266454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc8735ccb0 msgr2=0x55cc873d2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:40:28.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.768+0000 7f79e7400640 1 --2- 172.21.15.53:0/2691266454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc8735ccb0 0x55cc873d2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.768+0000 7f79e7400640 1 -- 172.21.15.53:0/2691266454 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc873d2860 con 0x55cc87357150 2024-04-25T14:40:28.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.768+0000 7f79e7e00640 1 --2- 172.21.15.53:0/2691266454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc8735ccb0 0x55cc873d2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:40:28.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.769+0000 7f79e7400640 1 --2- 172.21.15.53:0/2691266454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cc87357150 0x55cc8735aca0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f79d800e9c0 tx=0x7f79d8010cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f72f122889f0f71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:28.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.769+0000 7f79d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-25T14:40:28.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.769+0000 7f79d4c00640 1 -- 172.21.15.53:0/2691266454 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d809c590 con 0x55cc87357150 2024-04-25T14:40:28.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.769+0000 7f79f0fb3740 1 -- 172.21.15.53:0/2691266454 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79d0040040 con 0x55cc87357150 2024-04-25T14:40:28.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.769+0000 7f79d4c00640 1 -- 172.21.15.53:0/2691266454 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8285+0+0 (secure 0 0 0) 0x7f79d8018b20 con 0x55cc87357150 2024-04-25T14:40:28.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.769+0000 7f79d4c00640 1 -- 172.21.15.53:0/2691266454 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f79d8010720 con 0x55cc87357150 2024-04-25T14:40:28.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.769+0000 7f79f0fb3740 10 client.25172 fetch_fsmap learned FSMap version 21 2024-04-25T14:40:28.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.769+0000 7f79f0fb3740 1 -- 172.21.15.53:0/2691266454 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55cc873d3b20 con 0x55cc87357150 2024-04-25T14:40:28.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.769+0000 7f79d4c00640 1 -- 172.21.15.53:0/2691266454 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79d0040040 con 0x55cc87357150 2024-04-25T14:40:28.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.770+0000 7f79d4c00640 1 -- 172.21.15.53:0/2691266454 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f79d809b910 con 0x55cc87357150 2024-04-25T14:40:28.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.770+0000 7f79f0fb3740 10 client.25172 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T14:40:28.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.770+0000 7f79f0fb3740 10 client.25172 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T14:40:28.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.770+0000 7f79f0fb3740 20 client.25172 populate_metadata read hostname 'smithi053' 2024-04-25T14:40:28.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.770+0000 7f79f0fb3740 1 --2- 172.21.15.53:0/2691266454 >> [v2:172.21.15.183:6838/1554258915,v1:172.21.15.183:6839/1554258915] conn(0x55cc873d86a0 0x55cc873d73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:28.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.770+0000 7f79f0fb3740 4 client.25172 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T14:40:28.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.770+0000 7f79f0fb3740 1 -- 172.21.15.53:0/2691266454 --> [v2:172.21.15.183:6838/1554258915,v1:172.21.15.183:6839/1554258915] -- command(tid 0: {"prefix": "status"}) -- 0x55cc872c33a0 con 0x55cc873d86a0 2024-04-25T14:40:28.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.770+0000 7f79e6a00640 1 --2- 172.21.15.53:0/2691266454 >> [v2:172.21.15.183:6838/1554258915,v1:172.21.15.183:6839/1554258915] conn(0x55cc873d86a0 0x55cc873d73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:28.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.771+0000 7f79e6a00640 1 --2- 172.21.15.53:0/2691266454 >> [v2:172.21.15.183:6838/1554258915,v1:172.21.15.183:6839/1554258915] conn(0x55cc873d86a0 0x55cc873d73b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9bc9c0be45f59de9 server_cookie=e05f5979a65d454 in_seq=0 out_seq=0 2024-04-25T14:40:28.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.771+0000 7f79d4c00640 10 client.25172 ms_handle_connect on v2:172.21.15.183:6838/1554258915 2024-04-25T14:40:28.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.771+0000 7f79d4c00640 1 -- 172.21.15.53:0/2691266454 <== mds.1 v2:172.21.15.183:6838/1554258915 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55cc872c33a0 con 0x55cc873d86a0 2024-04-25T14:40:28.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.771+0000 7f79d4c00640 10 client.25172 handle_command_reply: tid=0 2024-04-25T14:40:28.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.771+0000 7f79f0fb3740 1 client.25172 shutdown 2024-04-25T14:40:28.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.771+0000 7f79f0fb3740 1 -- 172.21.15.53:0/2691266454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cc87357150 msgr2=0x55cc8735aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:28.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.771+0000 7f79f0fb3740 1 --2- 172.21.15.53:0/2691266454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cc87357150 0x55cc8735aca0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f79d800e9c0 tx=0x7f79d8010cc0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.771+0000 7f79f0fb3740 1 -- 172.21.15.53:0/2691266454 shutdown_connections 2024-04-25T14:40:28.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.771+0000 7f79f0fb3740 1 --2- 172.21.15.53:0/2691266454 >> [v2:172.21.15.183:6838/1554258915,v1:172.21.15.183:6839/1554258915] conn(0x55cc873d86a0 0x55cc873d73b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.771+0000 7f79f0fb3740 1 --2- 172.21.15.53:0/2691266454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc8735ccb0 0x55cc873d2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.771+0000 7f79f0fb3740 1 --2- 172.21.15.53:0/2691266454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc87357b40 0x55cc8735b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.771+0000 7f79f0fb3740 1 --2- 172.21.15.53:0/2691266454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cc87357150 0x55cc8735aca0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.771+0000 7f79d4c00640 0 client.25172 ms_handle_reset on v2:172.21.15.183:6838/1554258915 2024-04-25T14:40:28.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.771+0000 7f79f0fb3740 1 -- 172.21.15.53:0/2691266454 >> 172.21.15.53:0/2691266454 conn(0x55cc873527c0 msgr2=0x55cc87352ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:28.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.771+0000 7f79f0fb3740 1 -- 172.21.15.53:0/2691266454 shutdown_connections 2024-04-25T14:40:28.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.771+0000 7f79f0fb3740 1 -- 172.21.15.53:0/2691266454 wait complete. 2024-04-25T14:40:28.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.771+0000 7f79f0fb3740 20 client.25172 trim_cache size 0 max 16384 2024-04-25T14:40:28.777 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-04-25T14:40:28.777 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "fd30e6ce-030f-11ef-bc93-c7b262605968", 2024-04-25T14:40:28.777 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 1, 2024-04-25T14:40:28.777 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 24565, 2024-04-25T14:40:28.777 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2024-04-25T14:40:28.777 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2024-04-25T14:40:28.777 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2024-04-25T14:40:28.777 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 134.776028698, 2024-04-25T14:40:28.777 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 21, 2024-04-25T14:40:28.777 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 92, 2024-04-25T14:40:28.777 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 88, 2024-04-25T14:40:28.777 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 154.42817625199999 2024-04-25T14:40:28.777 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-04-25T14:40:28.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.772+0000 7f79ecc00640 1 -- 172.21.15.53:0/499972915 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f79cc0a3c30 msgr2=0x7f79cc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:28.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.772+0000 7f79ecc00640 1 --2- 172.21.15.53:0/499972915 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f79cc0a3c30 0x7f79cc0a60f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x55cc873583a0 tx=0x7f79e0009500 comp rx=0 tx=0).stop 2024-04-25T14:40:28.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.772+0000 7f79ecc00640 1 -- 172.21.15.53:0/499972915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79e8101e80 msgr2=0x7f79e819a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:28.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.772+0000 7f79ecc00640 1 --2- 172.21.15.53:0/499972915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79e8101e80 0x7f79e819a330 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f79d800e9f0 tx=0x7f79d800eeb0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.772+0000 7f79ecc00640 1 -- 172.21.15.53:0/499972915 shutdown_connections 2024-04-25T14:40:28.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.772+0000 7f79ecc00640 1 --2- 172.21.15.53:0/499972915 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f79cc0a3c30 0x7f79cc0a60f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.772+0000 7f79ecc00640 1 --2- 172.21.15.53:0/499972915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79e8103990 0x7f79e81a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.772+0000 7f79ecc00640 1 --2- 172.21.15.53:0/499972915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79e8102fd0 0x7f79e819a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.772+0000 7f79ecc00640 1 --2- 172.21.15.53:0/499972915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79e8101e80 0x7f79e819a330 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:28.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.772+0000 7f79ecc00640 1 -- 172.21.15.53:0/499972915 >> 172.21.15.53:0/499972915 conn(0x7f79e80fd3f0 msgr2=0x7f79e8074180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:28.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.772+0000 7f79ecc00640 1 -- 172.21.15.53:0/499972915 shutdown_connections 2024-04-25T14:40:28.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:28.772+0000 7f79ecc00640 1 -- 172.21.15.53:0/499972915 wait complete. 2024-04-25T14:40:28.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:28 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/3034884087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:40:28.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:28 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/3899265818' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:40:28.790 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T14:40:28.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:28 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/3034884087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:40:28.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:28 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/3899265818' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:40:28.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:28 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/3034884087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:40:28.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:28 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/3899265818' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:40:29.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.001+0000 7f7ea3e00640 1 -- 172.21.15.53:0/688458491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ea4108490 msgr2=0x7f7ea41088b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:29.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.001+0000 7f7ea3e00640 1 --2- 172.21.15.53:0/688458491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ea4108490 0x7f7ea41088b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7e94009900 tx=0x7f7e9402a3f0 comp rx=0 tx=0).stop 2024-04-25T14:40:29.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.001+0000 7f7ea3e00640 1 -- 172.21.15.53:0/688458491 shutdown_connections 2024-04-25T14:40:29.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.001+0000 7f7ea3e00640 1 --2- 172.21.15.53:0/688458491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ea410a040 0x7f7ea41108d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:29.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.001+0000 7f7ea3e00640 1 --2- 172.21.15.53:0/688458491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ea4109680 0x7f7ea4109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:29.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.001+0000 7f7ea3e00640 1 --2- 172.21.15.53:0/688458491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ea4108490 0x7f7ea41088b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:29.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.001+0000 7f7ea3e00640 1 -- 172.21.15.53:0/688458491 >> 172.21.15.53:0/688458491 conn(0x7f7ea4077420 msgr2=0x7f7ea4079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:29.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.001+0000 7f7ea3e00640 1 -- 172.21.15.53:0/688458491 shutdown_connections 2024-04-25T14:40:29.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.001+0000 7f7ea3e00640 1 -- 172.21.15.53:0/688458491 wait complete. 2024-04-25T14:40:29.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.002+0000 7f7ea3e00640 1 Processor -- start 2024-04-25T14:40:29.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.002+0000 7f7ea3e00640 1 -- start start 2024-04-25T14:40:29.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.003+0000 7f7ea3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ea4108490 0x7f7ea419e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:29.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.003+0000 7f7ea3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ea4109680 0x7f7ea419ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:29.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.003+0000 7f7ea3e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ea410a040 0x7f7ea41a5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:29.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.003+0000 7f7ea3e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea4113ff0 con 0x7f7ea4109680 2024-04-25T14:40:29.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.003+0000 7f7ea3e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea4113e70 con 0x7f7ea410a040 2024-04-25T14:40:29.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.003+0000 7f7ea3e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea4114170 con 0x7f7ea4108490 2024-04-25T14:40:29.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.003+0000 7f7e9be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ea4109680 0x7f7ea419ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:29.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.003+0000 7f7ea2a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ea4108490 0x7f7ea419e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:29.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.003+0000 7f7e9be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ea4109680 0x7f7ea419ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37178/0 (socket says 172.21.15.53:37178) 2024-04-25T14:40:29.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.003+0000 7f7e9be00640 1 -- 172.21.15.53:0/2137494403 learned_addr learned my addr 172.21.15.53:0/2137494403 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:29.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.003+0000 7f7ea2a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ea4108490 0x7f7ea419e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:45192/0 (socket says 172.21.15.53:45192) 2024-04-25T14:40:29.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.003+0000 7f7ea3400640 1 --2- 172.21.15.53:0/2137494403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ea410a040 0x7f7ea41a5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:29.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.003+0000 7f7e9be00640 1 -- 172.21.15.53:0/2137494403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ea4108490 msgr2=0x7f7ea419e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:29.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.003+0000 7f7e9be00640 1 --2- 172.21.15.53:0/2137494403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ea4108490 0x7f7ea419e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:29.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.003+0000 7f7e9be00640 1 -- 172.21.15.53:0/2137494403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ea410a040 msgr2=0x7f7ea41a5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:29.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.003+0000 7f7e9be00640 1 --2- 172.21.15.53:0/2137494403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ea410a040 0x7f7ea41a5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:29.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.003+0000 7f7e9be00640 1 -- 172.21.15.53:0/2137494403 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea41a62f0 con 0x7f7ea4109680 2024-04-25T14:40:29.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.004+0000 7f7e9be00640 1 --2- 172.21.15.53:0/2137494403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ea4109680 0x7f7ea419ec60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c00e9f0 tx=0x7f7e8c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:29.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.004+0000 7f7e9b400640 1 -- 172.21.15.53:0/2137494403 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c00cd80 con 0x7f7ea4109680 2024-04-25T14:40:29.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.004+0000 7f7e9b400640 1 -- 172.21.15.53:0/2137494403 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e8c0108e0 con 0x7f7ea4109680 2024-04-25T14:40:29.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.004+0000 7f7ea3e00640 1 -- 172.21.15.53:0/2137494403 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ea41a65e0 con 0x7f7ea4109680 2024-04-25T14:40:29.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.004+0000 7f7e9b400640 1 -- 172.21.15.53:0/2137494403 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c018980 con 0x7f7ea4109680 2024-04-25T14:40:29.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.004+0000 7f7ea3e00640 1 -- 172.21.15.53:0/2137494403 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ea41a6b80 con 0x7f7ea4109680 2024-04-25T14:40:29.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.007+0000 7f7eacbd2740 1 Processor -- start 2024-04-25T14:40:29.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.008+0000 7f7eacbd2740 1 -- start start 2024-04-25T14:40:29.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.008+0000 7f7e9b400640 1 -- 172.21.15.53:0/2137494403 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7e8c010430 con 0x7f7ea4109680 2024-04-25T14:40:29.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.008+0000 7f7eacbd2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f682871150 0x55f682871570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:29.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.008+0000 7f7eacbd2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f682871b40 0x55f682876640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:29.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.008+0000 7f7eacbd2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f682876cb0 0x55f682879140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:29.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.008+0000 7f7eacbd2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55f6827d5b10 con 0x55f682876cb0 2024-04-25T14:40:29.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.008+0000 7f7eacbd2740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f68277c2b0 con 0x55f682871150 2024-04-25T14:40:29.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.008+0000 7f7eacbd2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f6827d1ad0 con 0x55f682871b40 2024-04-25T14:40:29.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.008+0000 7f7ea3400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f682871150 0x55f682871570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:29.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.008+0000 7f7e9be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f682876cb0 0x55f682879140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:29.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.008+0000 7f7ea3400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f682871150 0x55f682871570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.53:40776/0 (socket says 172.21.15.53:40776) 2024-04-25T14:40:29.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.008+0000 7f7ea3400640 1 -- 172.21.15.53:0/924219731 learned_addr learned my addr 172.21.15.53:0/924219731 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:29.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.008+0000 7f7ea2a00640 1 --2- 172.21.15.53:0/924219731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f682871b40 0x55f682876640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:29.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.009+0000 7f7ea3400640 1 -- 172.21.15.53:0/924219731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f682871b40 msgr2=0x55f682876640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:40:29.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.009+0000 7f7ea3400640 1 --2- 172.21.15.53:0/924219731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f682871b40 0x55f682876640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:29.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.009+0000 7f7ea3400640 1 -- 172.21.15.53:0/924219731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f682876cb0 msgr2=0x55f682879140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:40:29.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.009+0000 7f7ea3400640 1 --2- 172.21.15.53:0/924219731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f682876cb0 0x55f682879140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:29.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.009+0000 7f7ea3400640 1 -- 172.21.15.53:0/924219731 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6824e82d0 con 0x55f682871150 2024-04-25T14:40:29.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.009+0000 7f7ea3400640 1 --2- 172.21.15.53:0/924219731 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f682871150 0x55f682871570 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c0039f0 tx=0x7f7e9c003eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2ba05bf5ab96c60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:29.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.009+0000 7f7e9b400640 1 --2- 172.21.15.53:0/2137494403 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f7e880a3c30 0x7f7e880a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:29.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.009+0000 7f7e93e00640 1 -- 172.21.15.53:0/924219731 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c01a470 con 0x55f682871150 2024-04-25T14:40:29.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.009+0000 7f7e93e00640 1 -- 172.21.15.53:0/924219731 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e9c015070 con 0x55f682871150 2024-04-25T14:40:29.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.009+0000 7f7e93e00640 1 -- 172.21.15.53:0/924219731 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c0107e0 con 0x55f682871150 2024-04-25T14:40:29.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.009+0000 7f7e9b400640 1 -- 172.21.15.53:0/2137494403 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8285+0+0 (secure 0 0 0) 0x7f7e8c0d0750 con 0x7f7ea4109680 2024-04-25T14:40:29.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.010+0000 7f7ea2a00640 1 --2- 172.21.15.53:0/2137494403 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f7e880a3c30 0x7f7e880a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:29.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.010+0000 7f7eacbd2740 1 -- 172.21.15.53:0/924219731 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f682871150 msgr2=0x55f682871570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:29.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.010+0000 7f7eacbd2740 1 --2- 172.21.15.53:0/924219731 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f682871150 0x55f682871570 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c0039f0 tx=0x7f7e9c003eb0 comp rx=0 tx=0).stop 2024-04-25T14:40:29.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.010+0000 7f7ea2a00640 1 --2- 172.21.15.53:0/2137494403 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f7e880a3c30 0x7f7e880a60f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7e94005ab0 tx=0x7f7e94005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:29.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.010+0000 7f7eacbd2740 1 -- 172.21.15.53:0/924219731 shutdown_connections 2024-04-25T14:40:29.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.010+0000 7f7eacbd2740 1 --2- 172.21.15.53:0/924219731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f682876cb0 0x55f682879140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:29.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.010+0000 7f7eacbd2740 1 --2- 172.21.15.53:0/924219731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f682871b40 0x55f682876640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:29.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.010+0000 7f7eacbd2740 1 --2- 172.21.15.53:0/924219731 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f682871150 0x55f682871570 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:29.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.010+0000 7f7eacbd2740 1 -- 172.21.15.53:0/924219731 >> 172.21.15.53:0/924219731 conn(0x55f68286c7c0 msgr2=0x55f68286cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:29.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.011+0000 7f7eacbd2740 1 -- 172.21.15.53:0/924219731 shutdown_connections 2024-04-25T14:40:29.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.011+0000 7f7eacbd2740 1 -- 172.21.15.53:0/924219731 wait complete. 2024-04-25T14:40:29.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.011+0000 7f7eacbd2740 1 -- start start 2024-04-25T14:40:29.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.011+0000 7f7eacbd2740 1 Processor -- start 2024-04-25T14:40:29.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.012+0000 7f7eacbd2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f682871150 0x55f68287b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:29.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.012+0000 7f7eacbd2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f682871b40 0x55f68287b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:29.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.012+0000 7f7eacbd2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f682876cb0 0x55f6828828c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:29.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.012+0000 7f7eacbd2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55f682879c60 con 0x55f682871b40 2024-04-25T14:40:29.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.012+0000 7f7eacbd2740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f68278f570 con 0x55f682876cb0 2024-04-25T14:40:29.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.012+0000 7f7eacbd2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f682879de0 con 0x55f682871150 2024-04-25T14:40:29.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.012+0000 7f7eacbd2740 10 client.0 _ll_register_callbacks cb 0x55f68275dfc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:40:29.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.012+0000 7f7ea2a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f682871150 0x55f68287b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:29.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.012+0000 7f7e9be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f682871b40 0x55f68287b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:29.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.012+0000 7f7ea2a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f682871150 0x55f68287b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:45204/0 (socket says 172.21.15.53:45204) 2024-04-25T14:40:29.016 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.012+0000 7f7ea2a00640 1 -- 172.21.15.53:0/140034655 learned_addr learned my addr 172.21.15.53:0/140034655 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:29.016 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.012+0000 7f7ea3400640 1 --2- 172.21.15.53:0/140034655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f682876cb0 0x55f6828828c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:29.016 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.012+0000 7f7ea2a00640 1 -- 172.21.15.53:0/140034655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f682876cb0 msgr2=0x55f6828828c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:40:29.016 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.012+0000 7f7ea2a00640 1 --2- 172.21.15.53:0/140034655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f682876cb0 0x55f6828828c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:29.016 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.012+0000 7f7ea2a00640 1 -- 172.21.15.53:0/140034655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f682871b40 msgr2=0x55f68287b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:40:29.016 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.012+0000 7f7ea2a00640 1 --2- 172.21.15.53:0/140034655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f682871b40 0x55f68287b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:29.016 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.012+0000 7f7ea2a00640 1 -- 172.21.15.53:0/140034655 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6825b26b0 con 0x55f682871150 2024-04-25T14:40:29.016 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.012+0000 7f7e9be00640 1 --2- 172.21.15.53:0/140034655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f682871b40 0x55f68287b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:40:29.016 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.013+0000 7f7ea2a00640 1 --2- 172.21.15.53:0/140034655 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f682871150 0x55f68287b340 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7e94000c00 tx=0x7f7e94009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d5a86bfaf2047cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:29.016 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.013+0000 7f7e90c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T14:40:29.016 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.013+0000 7f7e90c00640 1 -- 172.21.15.53:0/140034655 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e94035040 con 0x55f682871150 2024-04-25T14:40:29.016 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.013+0000 7f7e90c00640 1 -- 172.21.15.53:0/140034655 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8285+0+0 (secure 0 0 0) 0x7f7e94003a00 con 0x55f682871150 2024-04-25T14:40:29.016 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.013+0000 7f7eacbd2740 1 -- 172.21.15.53:0/140034655 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e9c015070 con 0x55f682871150 2024-04-25T14:40:29.016 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.013+0000 7f7e90c00640 1 -- 172.21.15.53:0/140034655 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f7e94003c00 con 0x55f682871150 2024-04-25T14:40:29.016 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.013+0000 7f7eacbd2740 10 client.25204 fetch_fsmap learned FSMap version 21 2024-04-25T14:40:29.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.013+0000 7f7eacbd2740 1 -- 172.21.15.53:0/140034655 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f682885cf0 con 0x55f682871150 2024-04-25T14:40:29.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.014+0000 7f7e90c00640 1 -- 172.21.15.53:0/140034655 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e9c015070 con 0x55f682871150 2024-04-25T14:40:29.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.014+0000 7f7e90c00640 1 -- 172.21.15.53:0/140034655 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f7e9405a070 con 0x55f682871150 2024-04-25T14:40:29.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.014+0000 7f7eacbd2740 10 client.25204 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T14:40:29.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.014+0000 7f7eacbd2740 10 client.25204 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T14:40:29.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.014+0000 7f7eacbd2740 20 client.25204 populate_metadata read hostname 'smithi053' 2024-04-25T14:40:29.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.014+0000 7f7eacbd2740 1 --2- 172.21.15.53:0/140034655 >> [v2:172.21.15.150:6838/149573616,v1:172.21.15.150:6839/149573616] conn(0x55f682888360 0x55f6828897d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:29.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.014+0000 7f7eacbd2740 4 client.25204 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:40:29.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.014+0000 7f7eacbd2740 1 -- 172.21.15.53:0/140034655 --> [v2:172.21.15.150:6838/149573616,v1:172.21.15.150:6839/149573616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6827e3530 con 0x55f682888360 2024-04-25T14:40:29.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.015+0000 7f7e9be00640 1 --2- 172.21.15.53:0/140034655 >> [v2:172.21.15.150:6838/149573616,v1:172.21.15.150:6839/149573616] conn(0x55f682888360 0x55f6828897d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:29.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.015+0000 7f7e9be00640 1 --2- 172.21.15.53:0/140034655 >> [v2:172.21.15.150:6838/149573616,v1:172.21.15.150:6839/149573616] conn(0x55f682888360 0x55f6828897d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1f5a806860098b3 server_cookie=478ee8b950674079 in_seq=0 out_seq=0 2024-04-25T14:40:29.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.015+0000 7f7e90c00640 10 client.25204 ms_handle_connect on v2:172.21.15.150:6838/149573616 2024-04-25T14:40:29.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.016+0000 7f7e90c00640 1 -- 172.21.15.53:0/140034655 <== mds.0 v2:172.21.15.150:6838/149573616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f6827e3530 con 0x55f682888360 2024-04-25T14:40:29.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.016+0000 7f7e90c00640 10 client.25204 handle_command_reply: tid=0 2024-04-25T14:40:29.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.016+0000 7f7eacbd2740 1 client.25204 shutdown 2024-04-25T14:40:29.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.016+0000 7f7eacbd2740 1 -- 172.21.15.53:0/140034655 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f682871150 msgr2=0x55f68287b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:29.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.016+0000 7f7eacbd2740 1 --2- 172.21.15.53:0/140034655 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f682871150 0x55f68287b340 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7e94000c00 tx=0x7f7e94009260 comp rx=0 tx=0).stop 2024-04-25T14:40:29.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.016+0000 7f7eacbd2740 1 -- 172.21.15.53:0/140034655 shutdown_connections 2024-04-25T14:40:29.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.016+0000 7f7eacbd2740 1 --2- 172.21.15.53:0/140034655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f682876cb0 0x55f6828828c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:29.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.016+0000 7f7eacbd2740 1 --2- 172.21.15.53:0/140034655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f682871b40 0x55f68287b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:29.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.016+0000 7f7eacbd2740 1 --2- 172.21.15.53:0/140034655 >> [v2:172.21.15.150:6838/149573616,v1:172.21.15.150:6839/149573616] conn(0x55f682888360 0x55f6828897d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:29.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.016+0000 7f7eacbd2740 1 --2- 172.21.15.53:0/140034655 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f682871150 0x55f68287b340 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:29.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.016+0000 7f7e90c00640 0 client.25204 ms_handle_reset on v2:172.21.15.150:6838/149573616 2024-04-25T14:40:29.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.016+0000 7f7eacbd2740 1 -- 172.21.15.53:0/140034655 >> 172.21.15.53:0/140034655 conn(0x55f68286c7c0 msgr2=0x55f68286ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:29.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.016+0000 7f7eacbd2740 1 -- 172.21.15.53:0/140034655 shutdown_connections 2024-04-25T14:40:29.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.016+0000 7f7eacbd2740 1 -- 172.21.15.53:0/140034655 wait complete. 2024-04-25T14:40:29.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.017+0000 7f7eacbd2740 20 client.25204 trim_cache size 0 max 16384 2024-04-25T14:40:29.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.037+0000 7f7eacbd2740 1 Processor -- start 2024-04-25T14:40:29.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.037+0000 7f7eacbd2740 1 -- start start 2024-04-25T14:40:29.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.037+0000 7f7eacbd2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f682871150 0x55f68286c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:29.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.037+0000 7f7eacbd2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f682871b40 0x55f682882880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:29.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.037+0000 7f7eacbd2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f682876cb0 0x55f6828790a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:29.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.037+0000 7f7eacbd2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55f682887b10 con 0x55f682871150 2024-04-25T14:40:29.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.037+0000 7f7eacbd2740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f6828881e0 con 0x55f682871b40 2024-04-25T14:40:29.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.037+0000 7f7eacbd2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f6828888b0 con 0x55f682876cb0 2024-04-25T14:40:29.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.037+0000 7f7ea2a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f682871150 0x55f68286c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:29.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.037+0000 7f7ea2a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f682871150 0x55f68286c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37200/0 (socket says 172.21.15.53:37200) 2024-04-25T14:40:29.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.037+0000 7f7ea2a00640 1 -- 172.21.15.53:0/211601619 learned_addr learned my addr 172.21.15.53:0/211601619 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:29.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.038+0000 7f7e9be00640 1 --2- 172.21.15.53:0/211601619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f682871b40 0x55f682882880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:29.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.038+0000 7f7ea3400640 1 --2- 172.21.15.53:0/211601619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f682876cb0 0x55f6828790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:29.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.038+0000 7f7ea2a00640 1 -- 172.21.15.53:0/211601619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f682876cb0 msgr2=0x55f6828790a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:40:29.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.038+0000 7f7ea2a00640 1 --2- 172.21.15.53:0/211601619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f682876cb0 0x55f6828790a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:29.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.038+0000 7f7ea2a00640 1 -- 172.21.15.53:0/211601619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f682871b40 msgr2=0x55f682882880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:40:29.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.038+0000 7f7ea2a00640 1 --2- 172.21.15.53:0/211601619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f682871b40 0x55f682882880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:29.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.038+0000 7f7ea2a00640 1 -- 172.21.15.53:0/211601619 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f682866ef0 con 0x55f682871150 2024-04-25T14:40:29.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.038+0000 7f7e9be00640 1 --2- 172.21.15.53:0/211601619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f682871b40 0x55f682882880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:40:29.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.038+0000 7f7ea2a00640 1 --2- 172.21.15.53:0/211601619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f682871150 0x55f68286c260 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7e94051f30 tx=0x7f7e9402a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68f30018af4c447a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:29.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.039+0000 7f7e93400640 1 -- 172.21.15.53:0/211601619 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e94035040 con 0x55f682871150 2024-04-25T14:40:29.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.039+0000 7f7e93400640 1 -- 172.21.15.53:0/211601619 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e94061070 con 0x55f682871150 2024-04-25T14:40:29.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.039+0000 7f7e93400640 1 -- 172.21.15.53:0/211601619 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e940516d0 con 0x55f682871150 2024-04-25T14:40:29.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.039+0000 7f7eacbd2740 1 -- 172.21.15.53:0/211601619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f682871150 msgr2=0x55f68286c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:29.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.039+0000 7f7eacbd2740 1 --2- 172.21.15.53:0/211601619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f682871150 0x55f68286c260 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7e94051f30 tx=0x7f7e9402a3f0 comp rx=0 tx=0).stop 2024-04-25T14:40:29.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.039+0000 7f7eacbd2740 1 -- 172.21.15.53:0/211601619 shutdown_connections 2024-04-25T14:40:29.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.039+0000 7f7eacbd2740 1 --2- 172.21.15.53:0/211601619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f682876cb0 0x55f6828790a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:29.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.039+0000 7f7eacbd2740 1 --2- 172.21.15.53:0/211601619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f682871b40 0x55f682882880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:29.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.039+0000 7f7eacbd2740 1 --2- 172.21.15.53:0/211601619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f682871150 0x55f68286c260 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:29.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.039+0000 7f7eacbd2740 1 -- 172.21.15.53:0/211601619 >> 172.21.15.53:0/211601619 conn(0x55f68286c7c0 msgr2=0x55f68286cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:29.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.039+0000 7f7eacbd2740 1 -- 172.21.15.53:0/211601619 shutdown_connections 2024-04-25T14:40:29.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.039+0000 7f7eacbd2740 1 -- 172.21.15.53:0/211601619 wait complete. 2024-04-25T14:40:29.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.040+0000 7f7eacbd2740 1 -- start start 2024-04-25T14:40:29.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.040+0000 7f7eacbd2740 1 Processor -- start 2024-04-25T14:40:29.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.040+0000 7f7eacbd2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f682871150 0x55f68287d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:29.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.040+0000 7f7eacbd2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f682871b40 0x55f68287dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:29.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.041+0000 7f7eacbd2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f682876cb0 0x55f6828ec300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:29.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.041+0000 7f7eacbd2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55f682889650 con 0x55f682871150 2024-04-25T14:40:29.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.041+0000 7f7eacbd2740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f682888f80 con 0x55f682876cb0 2024-04-25T14:40:29.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.041+0000 7f7eacbd2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f682889d20 con 0x55f682871b40 2024-04-25T14:40:29.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.041+0000 7f7eacbd2740 10 client.0 _ll_register_callbacks cb 0x7f7e94004460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:40:29.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.041+0000 7f7ea2a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f682871150 0x55f68287d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:29.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.041+0000 7f7ea2a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f682871150 0x55f68287d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37204/0 (socket says 172.21.15.53:37204) 2024-04-25T14:40:29.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.041+0000 7f7e9be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f682871b40 0x55f68287dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:29.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.041+0000 7f7ea2a00640 1 -- 172.21.15.53:0/1457317795 learned_addr learned my addr 172.21.15.53:0/1457317795 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:29.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.041+0000 7f7ea3400640 1 --2- 172.21.15.53:0/1457317795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f682876cb0 0x55f6828ec300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:29.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.041+0000 7f7ea2a00640 1 -- 172.21.15.53:0/1457317795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f682871b40 msgr2=0x55f68287dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:40:29.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.041+0000 7f7ea2a00640 1 --2- 172.21.15.53:0/1457317795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f682871b40 0x55f68287dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:29.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.041+0000 7f7ea2a00640 1 -- 172.21.15.53:0/1457317795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f682876cb0 msgr2=0x55f6828ec300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:40:29.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.041+0000 7f7ea2a00640 1 --2- 172.21.15.53:0/1457317795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f682876cb0 0x55f6828ec300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:29.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.041+0000 7f7ea2a00640 1 -- 172.21.15.53:0/1457317795 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6828ec840 con 0x55f682871150 2024-04-25T14:40:29.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.041+0000 7f7e9be00640 1 --2- 172.21.15.53:0/1457317795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f682871b40 0x55f68287dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:40:29.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.042+0000 7f7ea2a00640 1 --2- 172.21.15.53:0/1457317795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f682871150 0x55f68287d770 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7e94000c00 tx=0x7f7e94003a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f78bd9dc9eec9f4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:29.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.042+0000 7f7e90c00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-04-25T14:40:29.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.042+0000 7f7e90c00640 1 -- 172.21.15.53:0/1457317795 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e94035040 con 0x55f682871150 2024-04-25T14:40:29.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.042+0000 7f7e90c00640 1 -- 172.21.15.53:0/1457317795 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8285+0+0 (secure 0 0 0) 0x7f7e94051470 con 0x55f682871150 2024-04-25T14:40:29.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.042+0000 7f7eacbd2740 1 -- 172.21.15.53:0/1457317795 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e94061070 con 0x55f682871150 2024-04-25T14:40:29.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.042+0000 7f7e90c00640 1 -- 172.21.15.53:0/1457317795 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f7e940516b0 con 0x55f682871150 2024-04-25T14:40:29.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.042+0000 7f7eacbd2740 10 client.15078 fetch_fsmap learned FSMap version 21 2024-04-25T14:40:29.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.042+0000 7f7eacbd2740 1 -- 172.21.15.53:0/1457317795 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f6828eda20 con 0x55f682871150 2024-04-25T14:40:29.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.042+0000 7f7e90c00640 1 -- 172.21.15.53:0/1457317795 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e94061070 con 0x55f682871150 2024-04-25T14:40:29.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.043+0000 7f7e90c00640 1 -- 172.21.15.53:0/1457317795 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f7e9405a070 con 0x55f682871150 2024-04-25T14:40:29.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.043+0000 7f7eacbd2740 10 client.15078 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T14:40:29.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.043+0000 7f7eacbd2740 10 client.15078 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T14:40:29.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.043+0000 7f7eacbd2740 20 client.15078 populate_metadata read hostname 'smithi053' 2024-04-25T14:40:29.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.043+0000 7f7eacbd2740 1 --2- 172.21.15.53:0/1457317795 >> [v2:172.21.15.150:6838/149573616,v1:172.21.15.150:6839/149573616] conn(0x55f6828f2290 0x55f6828f0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:29.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.043+0000 7f7eacbd2740 4 client.15078 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T14:40:29.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.043+0000 7f7eacbd2740 1 -- 172.21.15.53:0/1457317795 --> [v2:172.21.15.150:6838/149573616,v1:172.21.15.150:6839/149573616] -- command(tid 0: {"prefix": "status"}) -- 0x55f6827dd3a0 con 0x55f6828f2290 2024-04-25T14:40:29.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.043+0000 7f7e9be00640 1 --2- 172.21.15.53:0/1457317795 >> [v2:172.21.15.150:6838/149573616,v1:172.21.15.150:6839/149573616] conn(0x55f6828f2290 0x55f6828f0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:29.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.044+0000 7f7e9be00640 1 --2- 172.21.15.53:0/1457317795 >> [v2:172.21.15.150:6838/149573616,v1:172.21.15.150:6839/149573616] conn(0x55f6828f2290 0x55f6828f0fa0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b421294c09ba99ed server_cookie=203fc800f5c8b605 in_seq=0 out_seq=0 2024-04-25T14:40:29.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.044+0000 7f7e90c00640 10 client.15078 ms_handle_connect on v2:172.21.15.150:6838/149573616 2024-04-25T14:40:29.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.044+0000 7f7e90c00640 1 -- 172.21.15.53:0/1457317795 <== mds.0 v2:172.21.15.150:6838/149573616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f6827dd3a0 con 0x55f6828f2290 2024-04-25T14:40:29.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.044+0000 7f7e90c00640 10 client.15078 handle_command_reply: tid=0 2024-04-25T14:40:29.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.044+0000 7f7eacbd2740 1 client.15078 shutdown 2024-04-25T14:40:29.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.044+0000 7f7eacbd2740 1 -- 172.21.15.53:0/1457317795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f682871150 msgr2=0x55f68287d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:29.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.044+0000 7f7eacbd2740 1 --2- 172.21.15.53:0/1457317795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f682871150 0x55f68287d770 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7e94000c00 tx=0x7f7e94003a30 comp rx=0 tx=0).stop 2024-04-25T14:40:29.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.044+0000 7f7eacbd2740 1 -- 172.21.15.53:0/1457317795 shutdown_connections 2024-04-25T14:40:29.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.044+0000 7f7eacbd2740 1 --2- 172.21.15.53:0/1457317795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f682876cb0 0x55f6828ec300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:29.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.044+0000 7f7eacbd2740 1 --2- 172.21.15.53:0/1457317795 >> [v2:172.21.15.150:6838/149573616,v1:172.21.15.150:6839/149573616] conn(0x55f6828f2290 0x55f6828f0fa0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:29.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.044+0000 7f7eacbd2740 1 --2- 172.21.15.53:0/1457317795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f682871b40 0x55f68287dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:29.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.044+0000 7f7eacbd2740 1 --2- 172.21.15.53:0/1457317795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f682871150 0x55f68287d770 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:29.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.044+0000 7f7eacbd2740 1 -- 172.21.15.53:0/1457317795 >> 172.21.15.53:0/1457317795 conn(0x55f68286c7c0 msgr2=0x55f682875f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:29.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.044+0000 7f7e90c00640 0 client.15078 ms_handle_reset on v2:172.21.15.150:6838/149573616 2024-04-25T14:40:29.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.044+0000 7f7eacbd2740 1 -- 172.21.15.53:0/1457317795 shutdown_connections 2024-04-25T14:40:29.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.044+0000 7f7eacbd2740 1 -- 172.21.15.53:0/1457317795 wait complete. 2024-04-25T14:40:29.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.045+0000 7f7eacbd2740 20 client.15078 trim_cache size 0 max 16384 2024-04-25T14:40:29.049 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-04-25T14:40:29.049 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "fd30e6ce-030f-11ef-bc93-c7b262605968", 2024-04-25T14:40:29.049 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2024-04-25T14:40:29.049 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 24584, 2024-04-25T14:40:29.049 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2024-04-25T14:40:29.049 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2024-04-25T14:40:29.049 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2024-04-25T14:40:29.049 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 138.51007454000001, 2024-04-25T14:40:29.049 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 21, 2024-04-25T14:40:29.049 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 92, 2024-04-25T14:40:29.049 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 88, 2024-04-25T14:40:29.049 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 143.82776740599999 2024-04-25T14:40:29.049 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-04-25T14:40:29.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.045+0000 7f7ea3e00640 1 -- 172.21.15.53:0/2137494403 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f7e880a3c30 msgr2=0x7f7e880a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:29.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.045+0000 7f7ea3e00640 1 --2- 172.21.15.53:0/2137494403 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f7e880a3c30 0x7f7e880a60f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7e94005ab0 tx=0x7f7e94005a20 comp rx=0 tx=0).stop 2024-04-25T14:40:29.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.045+0000 7f7ea3e00640 1 -- 172.21.15.53:0/2137494403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ea4109680 msgr2=0x7f7ea419ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:29.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.045+0000 7f7ea3e00640 1 --2- 172.21.15.53:0/2137494403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ea4109680 0x7f7ea419ec60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c00e9f0 tx=0x7f7e8c00eeb0 comp rx=0 tx=0).stop 2024-04-25T14:40:29.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.045+0000 7f7ea3e00640 1 -- 172.21.15.53:0/2137494403 shutdown_connections 2024-04-25T14:40:29.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.045+0000 7f7ea3e00640 1 --2- 172.21.15.53:0/2137494403 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f7e880a3c30 0x7f7e880a60f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:29.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.045+0000 7f7ea3e00640 1 --2- 172.21.15.53:0/2137494403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ea410a040 0x7f7ea41a5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:29.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.045+0000 7f7ea3e00640 1 --2- 172.21.15.53:0/2137494403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ea4109680 0x7f7ea419ec60 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:29.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.045+0000 7f7ea3e00640 1 --2- 172.21.15.53:0/2137494403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ea4108490 0x7f7ea419e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:29.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.045+0000 7f7ea3e00640 1 -- 172.21.15.53:0/2137494403 >> 172.21.15.53:0/2137494403 conn(0x7f7ea4077420 msgr2=0x7f7ea410caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:29.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.046+0000 7f7ea3e00640 1 -- 172.21.15.53:0/2137494403 shutdown_connections 2024-04-25T14:40:29.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.046+0000 7f7ea3e00640 1 -- 172.21.15.53:0/2137494403 wait complete. 2024-04-25T14:40:29.063 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T14:40:29.073 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T14:40:29.073 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T14:40:29.073 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T14:40:29.073 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-25T14:40:29.074 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T14:40:29.074 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi053.front.sepia.ceph.com 2024-04-25T14:40:29.075 DEBUG:teuthology.orchestra.run.smithi053:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-25T14:40:29.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.280+0000 7fcb73200640 1 -- 172.21.15.53:0/254296983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb74100440 msgr2=0x7fcb741020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:29.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.280+0000 7fcb73200640 1 --2- 172.21.15.53:0/254296983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb74100440 0x7fcb741020d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c009ca0 tx=0x7fcb6c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:40:29.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.280+0000 7fcb73200640 1 -- 172.21.15.53:0/254296983 shutdown_connections 2024-04-25T14:40:29.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.280+0000 7fcb73200640 1 --2- 172.21.15.53:0/254296983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb74100440 0x7fcb741020d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:29.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.280+0000 7fcb73200640 1 --2- 172.21.15.53:0/254296983 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb740ffa80 0x7fcb740fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:29.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.280+0000 7fcb73200640 1 --2- 172.21.15.53:0/254296983 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb74101030 0x7fcb74101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:29.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.280+0000 7fcb73200640 1 -- 172.21.15.53:0/254296983 >> 172.21.15.53:0/254296983 conn(0x7fcb740fb5b0 msgr2=0x7fcb740fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:29.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.280+0000 7fcb73200640 1 -- 172.21.15.53:0/254296983 shutdown_connections 2024-04-25T14:40:29.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.280+0000 7fcb73200640 1 -- 172.21.15.53:0/254296983 wait complete. 2024-04-25T14:40:29.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.282+0000 7fcb73200640 1 Processor -- start 2024-04-25T14:40:29.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.282+0000 7fcb73200640 1 -- start start 2024-04-25T14:40:29.282 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.282+0000 7fcb73200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb740ffa80 0x7fcb7419a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:29.282 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.282+0000 7fcb73200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb74100440 0x7fcb7419a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:29.282 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.282+0000 7fcb73200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb74101030 0x7fcb741a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:29.282 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.282+0000 7fcb73200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcb7410fb80 con 0x7fcb740ffa80 2024-04-25T14:40:29.282 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.282+0000 7fcb73200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcb7410fa00 con 0x7fcb74100440 2024-04-25T14:40:29.282 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.282+0000 7fcb73200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcb7410fd00 con 0x7fcb74101030 2024-04-25T14:40:29.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.283+0000 7fcb71e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb740ffa80 0x7fcb7419a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:29.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.283+0000 7fcb72800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb74101030 0x7fcb741a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:29.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.283+0000 7fcb71e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb740ffa80 0x7fcb7419a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37226/0 (socket says 172.21.15.53:37226) 2024-04-25T14:40:29.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.283+0000 7fcb71e00640 1 -- 172.21.15.53:0/2088410643 learned_addr learned my addr 172.21.15.53:0/2088410643 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:29.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.283+0000 7fcb71400640 1 --2- 172.21.15.53:0/2088410643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb74100440 0x7fcb7419a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:29.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.283+0000 7fcb71e00640 1 -- 172.21.15.53:0/2088410643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb74101030 msgr2=0x7fcb741a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:29.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.283+0000 7fcb71e00640 1 --2- 172.21.15.53:0/2088410643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb74101030 0x7fcb741a17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:29.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.283+0000 7fcb71e00640 1 -- 172.21.15.53:0/2088410643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb74100440 msgr2=0x7fcb7419a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:29.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.283+0000 7fcb71e00640 1 --2- 172.21.15.53:0/2088410643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb74100440 0x7fcb7419a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:29.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.283+0000 7fcb71e00640 1 -- 172.21.15.53:0/2088410643 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb741a1e20 con 0x7fcb740ffa80 2024-04-25T14:40:29.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.283+0000 7fcb72800640 1 --2- 172.21.15.53:0/2088410643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb74101030 0x7fcb741a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:40:29.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.284+0000 7fcb71e00640 1 --2- 172.21.15.53:0/2088410643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb740ffa80 0x7fcb7419a200 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcb6400e9f0 tx=0x7fcb6400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:29.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.284+0000 7fcb6aa00640 1 -- 172.21.15.53:0/2088410643 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6400cd80 con 0x7fcb740ffa80 2024-04-25T14:40:29.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.284+0000 7fcb6aa00640 1 -- 172.21.15.53:0/2088410643 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb640054d0 con 0x7fcb740ffa80 2024-04-25T14:40:29.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.284+0000 7fcb6aa00640 1 -- 172.21.15.53:0/2088410643 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb64010690 con 0x7fcb740ffa80 2024-04-25T14:40:29.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.284+0000 7fcb73200640 1 -- 172.21.15.53:0/2088410643 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb741a2110 con 0x7fcb740ffa80 2024-04-25T14:40:29.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.284+0000 7fcb73200640 1 -- 172.21.15.53:0/2088410643 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb741a2610 con 0x7fcb740ffa80 2024-04-25T14:40:29.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.285+0000 7fcb73200640 1 -- 172.21.15.53:0/2088410643 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb7410c2a0 con 0x7fcb740ffa80 2024-04-25T14:40:29.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.288+0000 7fcb6aa00640 1 -- 172.21.15.53:0/2088410643 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fcb64004e30 con 0x7fcb740ffa80 2024-04-25T14:40:29.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.289+0000 7fcb6aa00640 1 --2- 172.21.15.53:0/2088410643 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fcb580a3ba0 0x7fcb580a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:29.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.289+0000 7fcb6aa00640 1 -- 172.21.15.53:0/2088410643 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8285+0+0 (secure 0 0 0) 0x7fcb640d0aa0 con 0x7fcb740ffa80 2024-04-25T14:40:29.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.289+0000 7fcb71400640 1 --2- 172.21.15.53:0/2088410643 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fcb580a3ba0 0x7fcb580a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:29.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.290+0000 7fcb71400640 1 --2- 172.21.15.53:0/2088410643 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fcb580a3ba0 0x7fcb580a6060 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcb600095f0 tx=0x7fcb60005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:29.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.291+0000 7fcb6aa00640 1 -- 172.21.15.53:0/2088410643 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcb64053ff0 con 0x7fcb740ffa80 2024-04-25T14:40:29.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:29.553+0000 7fcb73200640 1 -- 172.21.15.53:0/2088410643 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fcb74062b50 con 0x7fcb740ffa80 2024-04-25T14:40:29.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:29 smithi183 ceph-mon[32584]: pgmap v389: 97 pgs: 97 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T14:40:29.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:29 smithi150 ceph-mon[32504]: pgmap v389: 97 pgs: 97 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T14:40:30.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:29 smithi053 ceph-mon[27773]: pgmap v389: 97 pgs: 97 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T14:40:30.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.553+0000 7fcb6aa00640 1 -- 172.21.15.53:0/2088410643 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7fcb6409d3f0 con 0x7fcb740ffa80 2024-04-25T14:40:30.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.557+0000 7fcb73200640 1 -- 172.21.15.53:0/2088410643 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fcb580a3ba0 msgr2=0x7fcb580a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:30.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.557+0000 7fcb73200640 1 --2- 172.21.15.53:0/2088410643 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fcb580a3ba0 0x7fcb580a6060 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcb600095f0 tx=0x7fcb60005f90 comp rx=0 tx=0).stop 2024-04-25T14:40:30.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.557+0000 7fcb73200640 1 -- 172.21.15.53:0/2088410643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb740ffa80 msgr2=0x7fcb7419a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:30.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.557+0000 7fcb73200640 1 --2- 172.21.15.53:0/2088410643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb740ffa80 0x7fcb7419a200 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcb6400e9f0 tx=0x7fcb6400edf0 comp rx=0 tx=0).stop 2024-04-25T14:40:30.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.557+0000 7fcb73200640 1 -- 172.21.15.53:0/2088410643 shutdown_connections 2024-04-25T14:40:30.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.557+0000 7fcb73200640 1 --2- 172.21.15.53:0/2088410643 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fcb580a3ba0 0x7fcb580a6060 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:30.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.557+0000 7fcb73200640 1 --2- 172.21.15.53:0/2088410643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb74101030 0x7fcb741a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:30.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.557+0000 7fcb73200640 1 --2- 172.21.15.53:0/2088410643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb74100440 0x7fcb7419a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:30.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.557+0000 7fcb73200640 1 --2- 172.21.15.53:0/2088410643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb740ffa80 0x7fcb7419a200 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:30.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.557+0000 7fcb73200640 1 -- 172.21.15.53:0/2088410643 >> 172.21.15.53:0/2088410643 conn(0x7fcb740fb5b0 msgr2=0x7fcb740fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:30.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.558+0000 7fcb73200640 1 -- 172.21.15.53:0/2088410643 shutdown_connections 2024-04-25T14:40:30.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.559+0000 7fcb73200640 1 -- 172.21.15.53:0/2088410643 wait complete. 2024-04-25T14:40:30.588 DEBUG:teuthology.orchestra.run.smithi053:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-25T14:40:30.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.775+0000 7f89d8400640 1 -- 172.21.15.53:0/372360419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89d0077b50 msgr2=0x7f89d0077fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:30.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.775+0000 7f89d8400640 1 --2- 172.21.15.53:0/372360419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89d0077b50 0x7f89d0077fb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f89bc009900 tx=0x7f89bc02a3f0 comp rx=0 tx=0).stop 2024-04-25T14:40:30.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.775+0000 7f89d8400640 1 -- 172.21.15.53:0/372360419 shutdown_connections 2024-04-25T14:40:30.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.775+0000 7f89d8400640 1 --2- 172.21.15.53:0/372360419 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89d00784f0 0x7f89d0078d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:30.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.775+0000 7f89d8400640 1 --2- 172.21.15.53:0/372360419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89d0077b50 0x7f89d0077fb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:30.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.775+0000 7f89d8400640 1 --2- 172.21.15.53:0/372360419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89d0076910 0x7f89d0076d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:30.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.775+0000 7f89d8400640 1 -- 172.21.15.53:0/372360419 >> 172.21.15.53:0/372360419 conn(0x7f89d00ff770 msgr2=0x7f89d0101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:30.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.776+0000 7f89d8400640 1 -- 172.21.15.53:0/372360419 shutdown_connections 2024-04-25T14:40:30.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.776+0000 7f89d8400640 1 -- 172.21.15.53:0/372360419 wait complete. 2024-04-25T14:40:30.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.777+0000 7f89d8400640 1 Processor -- start 2024-04-25T14:40:30.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.777+0000 7f89d8400640 1 -- start start 2024-04-25T14:40:30.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.777+0000 7f89d8400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89d0076910 0x7f89d019e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:30.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.777+0000 7f89d8400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89d0077b50 0x7f89d019eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:30.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.777+0000 7f89d8400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89d00784f0 0x7f89d01a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:30.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.777+0000 7f89d8400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f89d0114120 con 0x7f89d0077b50 2024-04-25T14:40:30.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.777+0000 7f89d8400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f89d0113fa0 con 0x7f89d00784f0 2024-04-25T14:40:30.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.777+0000 7f89d8400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f89d01142a0 con 0x7f89d0076910 2024-04-25T14:40:30.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.777+0000 7f89d5a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89d0076910 0x7f89d019e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:30.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.777+0000 7f89d5a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89d0076910 0x7f89d019e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:45262/0 (socket says 172.21.15.53:45262) 2024-04-25T14:40:30.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.777+0000 7f89d5a00640 1 -- 172.21.15.53:0/2618857544 learned_addr learned my addr 172.21.15.53:0/2618857544 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:30.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.777+0000 7f89d5000640 1 --2- 172.21.15.53:0/2618857544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89d0077b50 0x7f89d019eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:30.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.777+0000 7f89d6400640 1 --2- 172.21.15.53:0/2618857544 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89d00784f0 0x7f89d01a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:30.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.778+0000 7f89d5a00640 1 -- 172.21.15.53:0/2618857544 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89d00784f0 msgr2=0x7f89d01a5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:30.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.778+0000 7f89d5a00640 1 --2- 172.21.15.53:0/2618857544 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89d00784f0 0x7f89d01a5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:30.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.778+0000 7f89d5a00640 1 -- 172.21.15.53:0/2618857544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89d0077b50 msgr2=0x7f89d019eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:30.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.778+0000 7f89d5a00640 1 --2- 172.21.15.53:0/2618857544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89d0077b50 0x7f89d019eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:30.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.778+0000 7f89d5a00640 1 -- 172.21.15.53:0/2618857544 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d01a6330 con 0x7f89d0076910 2024-04-25T14:40:30.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.778+0000 7f89d6400640 1 --2- 172.21.15.53:0/2618857544 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89d00784f0 0x7f89d01a5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:40:30.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.778+0000 7f89d5a00640 1 --2- 172.21.15.53:0/2618857544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89d0076910 0x7f89d019e760 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f89c400ebc0 tx=0x7f89c400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:30.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.778+0000 7f89ce000640 1 -- 172.21.15.53:0/2618857544 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c4010630 con 0x7f89d0076910 2024-04-25T14:40:30.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.778+0000 7f89ce000640 1 -- 172.21.15.53:0/2618857544 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89c4010c80 con 0x7f89d0076910 2024-04-25T14:40:30.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.778+0000 7f89d8400640 1 -- 172.21.15.53:0/2618857544 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89d01a6620 con 0x7f89d0076910 2024-04-25T14:40:30.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.778+0000 7f89ce000640 1 -- 172.21.15.53:0/2618857544 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c4018b60 con 0x7f89d0076910 2024-04-25T14:40:30.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.779+0000 7f89d8400640 1 -- 172.21.15.53:0/2618857544 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89d01a6b40 con 0x7f89d0076910 2024-04-25T14:40:30.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.780+0000 7f89d8400640 1 -- 172.21.15.53:0/2618857544 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89d0110b50 con 0x7f89d0076910 2024-04-25T14:40:30.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.782+0000 7f89ce000640 1 -- 172.21.15.53:0/2618857544 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f89c4018d00 con 0x7f89d0076910 2024-04-25T14:40:30.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.783+0000 7f89ce000640 1 --2- 172.21.15.53:0/2618857544 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f89b80a3ca0 0x7f89b80a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:30.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.783+0000 7f89d5000640 1 -- 172.21.15.53:0/2618857544 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f89b80a3ca0 msgr2=0x7f89b80a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/4154933108 2024-04-25T14:40:30.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.783+0000 7f89ce000640 1 -- 172.21.15.53:0/2618857544 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8285+0+0 (secure 0 0 0) 0x7f89c40d0b10 con 0x7f89d0076910 2024-04-25T14:40:30.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.783+0000 7f89d5000640 1 --2- 172.21.15.53:0/2618857544 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f89b80a3ca0 0x7f89b80a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:40:30.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.787+0000 7f89ce000640 1 -- 172.21.15.53:0/2618857544 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f89c409d4c0 con 0x7f89d0076910 2024-04-25T14:40:30.855 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:30 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/2088410643' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T14:40:30.855 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:40:30 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: ignoring --setuser ceph since I am not root 2024-04-25T14:40:30.855 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:40:30 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: ignoring --setgroup ceph since I am not root 2024-04-25T14:40:30.855 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:40:30 smithi183 ceph-mgr[34458]: -- 172.21.15.183:0/2558985023 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d7b204e5a0 con 0x55d7b2022c00 2024-04-25T14:40:30.855 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:40:30 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:40:30.732+0000 7f03dde68200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:40:30.893 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:30 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/2088410643' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T14:40:30.893 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:40:30 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: ignoring --setuser ceph since I am not root 2024-04-25T14:40:30.893 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:40:30 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: ignoring --setgroup ceph since I am not root 2024-04-25T14:40:30.893 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:40:30 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:40:30.892+0000 7f49022f5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:40:30.917 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:30 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/2088410643' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T14:40:30.917 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:40:30 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: ignoring --setuser ceph since I am not root 2024-04-25T14:40:30.918 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:40:30 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: ignoring --setgroup ceph since I am not root 2024-04-25T14:40:30.918 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:40:30 smithi150 ceph-mgr[33883]: -- 172.21.15.150:0/356526145 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5596e8ecc5a0 con 0x5596e8ea0800 2024-04-25T14:40:30.918 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:40:30 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:40:30.733+0000 7f17f85b5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:40:30.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.984+0000 7f89d5000640 1 -- 172.21.15.53:0/2618857544 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f89b80a3ca0 msgr2=0x7f89b80a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/4154933108 2024-04-25T14:40:30.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.984+0000 7f89d5000640 1 --2- 172.21.15.53:0/2618857544 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f89b80a3ca0 0x7f89b80a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T14:40:30.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:30.996+0000 7f89d8400640 1 -- 172.21.15.53:0/2618857544 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f89d002d5c0 con 0x7f89d0076910 2024-04-25T14:40:31.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.003+0000 7f89ce000640 1 -- 172.21.15.53:0/2618857544 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) ==== 138+0+0 (secure 0 0 0) 0x7f89c409cdc0 con 0x7f89d0076910 2024-04-25T14:40:31.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.007+0000 7f89d8400640 1 -- 172.21.15.53:0/2618857544 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f89b80a3ca0 msgr2=0x7f89b80a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:40:31.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.007+0000 7f89d8400640 1 --2- 172.21.15.53:0/2618857544 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f89b80a3ca0 0x7f89b80a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:31.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.007+0000 7f89d8400640 1 -- 172.21.15.53:0/2618857544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89d0076910 msgr2=0x7f89d019e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:31.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.007+0000 7f89d8400640 1 --2- 172.21.15.53:0/2618857544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89d0076910 0x7f89d019e760 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f89c400ebc0 tx=0x7f89c400c6a0 comp rx=0 tx=0).stop 2024-04-25T14:40:31.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.007+0000 7f89d8400640 1 -- 172.21.15.53:0/2618857544 shutdown_connections 2024-04-25T14:40:31.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.007+0000 7f89d8400640 1 --2- 172.21.15.53:0/2618857544 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f89b80a3ca0 0x7f89b80a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:31.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.007+0000 7f89d8400640 1 --2- 172.21.15.53:0/2618857544 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89d00784f0 0x7f89d01a5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:31.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.007+0000 7f89d8400640 1 --2- 172.21.15.53:0/2618857544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89d0077b50 0x7f89d019eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:31.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.007+0000 7f89d8400640 1 --2- 172.21.15.53:0/2618857544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89d0076910 0x7f89d019e760 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:31.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.007+0000 7f89d8400640 1 -- 172.21.15.53:0/2618857544 >> 172.21.15.53:0/2618857544 conn(0x7f89d00ff770 msgr2=0x7f89d0101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:31.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.007+0000 7f89d8400640 1 -- 172.21.15.53:0/2618857544 shutdown_connections 2024-04-25T14:40:31.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.007+0000 7f89d8400640 1 -- 172.21.15.53:0/2618857544 wait complete. 2024-04-25T14:40:31.029 DEBUG:teuthology.orchestra.run.smithi053:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-25T14:40:31.203 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:40:30 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:40:30.917+0000 7f03dde68200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:40:31.204 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:40:31 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:40:31.020+0000 7f03dde68200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:40:31.210 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:40:30 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:40:30.917+0000 7f17f85b5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:40:31.210 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:40:31 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:40:31.020+0000 7f17f85b5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:40:31.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.228+0000 7fc3cb200640 1 -- 172.21.15.53:0/2436441505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3cc1028f0 msgr2=0x7fc3cc104ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:31.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.228+0000 7fc3cb200640 1 --2- 172.21.15.53:0/2436441505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3cc1028f0 0x7fc3cc104ce0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc009900 tx=0x7fc3bc02a410 comp rx=0 tx=0).stop 2024-04-25T14:40:31.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.229+0000 7fc3cb200640 1 -- 172.21.15.53:0/2436441505 shutdown_connections 2024-04-25T14:40:31.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.229+0000 7fc3cb200640 1 --2- 172.21.15.53:0/2436441505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3cc107b50 0x7fc3cc109f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:31.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.229+0000 7fc3cb200640 1 --2- 172.21.15.53:0/2436441505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3cc105220 0x7fc3cc107610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:31.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.229+0000 7fc3cb200640 1 --2- 172.21.15.53:0/2436441505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3cc1028f0 0x7fc3cc104ce0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:31.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.229+0000 7fc3cb200640 1 -- 172.21.15.53:0/2436441505 >> 172.21.15.53:0/2436441505 conn(0x7fc3cc0fc240 msgr2=0x7fc3cc0fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:31.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.229+0000 7fc3cb200640 1 -- 172.21.15.53:0/2436441505 shutdown_connections 2024-04-25T14:40:31.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.229+0000 7fc3cb200640 1 -- 172.21.15.53:0/2436441505 wait complete. 2024-04-25T14:40:31.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.230+0000 7fc3cb200640 1 Processor -- start 2024-04-25T14:40:31.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.231+0000 7fc3cb200640 1 -- start start 2024-04-25T14:40:31.232 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.231+0000 7fc3cb200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3cc1028f0 0x7fc3cc19a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:31.232 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.231+0000 7fc3cb200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3cc105220 0x7fc3cc19a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:31.232 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.231+0000 7fc3cb200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3cc107b50 0x7fc3cc1a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:31.232 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.231+0000 7fc3cb200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc10d910 con 0x7fc3cc1028f0 2024-04-25T14:40:31.232 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.231+0000 7fc3cb200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc10d790 con 0x7fc3cc105220 2024-04-25T14:40:31.232 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.231+0000 7fc3cb200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc10da90 con 0x7fc3cc107b50 2024-04-25T14:40:31.232 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.231+0000 7fc3c9e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3cc1028f0 0x7fc3cc19a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:31.232 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.231+0000 7fc3c9400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3cc105220 0x7fc3cc19a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:31.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.231+0000 7fc3c9e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3cc1028f0 0x7fc3cc19a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37268/0 (socket says 172.21.15.53:37268) 2024-04-25T14:40:31.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.231+0000 7fc3c9e00640 1 -- 172.21.15.53:0/2422491893 learned_addr learned my addr 172.21.15.53:0/2422491893 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:31.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.231+0000 7fc3ca800640 1 --2- 172.21.15.53:0/2422491893 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3cc107b50 0x7fc3cc1a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:31.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.232+0000 7fc3c9400640 1 -- 172.21.15.53:0/2422491893 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3cc107b50 msgr2=0x7fc3cc1a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:31.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.232+0000 7fc3c9400640 1 --2- 172.21.15.53:0/2422491893 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3cc107b50 0x7fc3cc1a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:31.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.232+0000 7fc3c9400640 1 -- 172.21.15.53:0/2422491893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3cc1028f0 msgr2=0x7fc3cc19a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:31.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.232+0000 7fc3c9400640 1 --2- 172.21.15.53:0/2422491893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3cc1028f0 0x7fc3cc19a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:31.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.232+0000 7fc3c9400640 1 -- 172.21.15.53:0/2422491893 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3cc1a1e50 con 0x7fc3cc105220 2024-04-25T14:40:31.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.232+0000 7fc3c9e00640 1 --2- 172.21.15.53:0/2422491893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3cc1028f0 0x7fc3cc19a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:40:31.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.232+0000 7fc3c9400640 1 --2- 172.21.15.53:0/2422491893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3cc105220 0x7fc3cc19a770 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc3b800e9c0 tx=0x7fc3b800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:31.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.232+0000 7fc3c2a00640 1 -- 172.21.15.53:0/2422491893 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b800cd80 con 0x7fc3cc105220 2024-04-25T14:40:31.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.232+0000 7fc3cb200640 1 -- 172.21.15.53:0/2422491893 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3cc1a2140 con 0x7fc3cc105220 2024-04-25T14:40:31.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.232+0000 7fc3c2a00640 1 -- 172.21.15.53:0/2422491893 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3b80035f0 con 0x7fc3cc105220 2024-04-25T14:40:31.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.232+0000 7fc3c2a00640 1 -- 172.21.15.53:0/2422491893 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b8010690 con 0x7fc3cc105220 2024-04-25T14:40:31.234 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.232+0000 7fc3cb200640 1 -- 172.21.15.53:0/2422491893 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3cc1a26e0 con 0x7fc3cc105220 2024-04-25T14:40:31.234 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.233+0000 7fc3cb200640 1 -- 172.21.15.53:0/2422491893 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3cc109640 con 0x7fc3cc105220 2024-04-25T14:40:31.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.236+0000 7fc3c2a00640 1 -- 172.21.15.53:0/2422491893 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fc3b80037a0 con 0x7fc3cc105220 2024-04-25T14:40:31.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.238+0000 7fc3c2a00640 1 --2- 172.21.15.53:0/2422491893 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fc3b00a3bf0 0x7fc3b00a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:31.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.238+0000 7fc3c9e00640 1 -- 172.21.15.53:0/2422491893 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fc3b00a3bf0 msgr2=0x7fc3b00a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/4154933108 2024-04-25T14:40:31.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.238+0000 7fc3c2a00640 1 -- 172.21.15.53:0/2422491893 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8285+0+0 (secure 0 0 0) 0x7fc3b80d0950 con 0x7fc3cc105220 2024-04-25T14:40:31.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.238+0000 7fc3c9e00640 1 --2- 172.21.15.53:0/2422491893 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fc3b00a3bf0 0x7fc3b00a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:40:31.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.239+0000 7fc3c2a00640 1 -- 172.21.15.53:0/2422491893 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc3b8043e20 con 0x7fc3cc105220 2024-04-25T14:40:31.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.439+0000 7fc3c9e00640 1 -- 172.21.15.53:0/2422491893 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fc3b00a3bf0 msgr2=0x7fc3b00a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/4154933108 2024-04-25T14:40:31.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.439+0000 7fc3c9e00640 1 --2- 172.21.15.53:0/2422491893 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fc3b00a3bf0 0x7fc3b00a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T14:40:31.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.445+0000 7fc3cb200640 1 -- 172.21.15.53:0/2422491893 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fc3cc02d5c0 con 0x7fc3cc105220 2024-04-25T14:40:31.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.451+0000 7fc3c2a00640 1 -- 172.21.15.53:0/2422491893 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) ==== 133+0+0 (secure 0 0 0) 0x7fc3b809d2a0 con 0x7fc3cc105220 2024-04-25T14:40:31.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.454+0000 7fc3cb200640 1 -- 172.21.15.53:0/2422491893 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fc3b00a3bf0 msgr2=0x7fc3b00a60b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:40:31.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.454+0000 7fc3cb200640 1 --2- 172.21.15.53:0/2422491893 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fc3b00a3bf0 0x7fc3b00a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:31.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.454+0000 7fc3cb200640 1 -- 172.21.15.53:0/2422491893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3cc105220 msgr2=0x7fc3cc19a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:31.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.454+0000 7fc3cb200640 1 --2- 172.21.15.53:0/2422491893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3cc105220 0x7fc3cc19a770 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc3b800e9c0 tx=0x7fc3b800ee80 comp rx=0 tx=0).stop 2024-04-25T14:40:31.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.454+0000 7fc3cb200640 1 -- 172.21.15.53:0/2422491893 shutdown_connections 2024-04-25T14:40:31.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.454+0000 7fc3cb200640 1 --2- 172.21.15.53:0/2422491893 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7fc3b00a3bf0 0x7fc3b00a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:31.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.454+0000 7fc3cb200640 1 --2- 172.21.15.53:0/2422491893 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3cc107b50 0x7fc3cc1a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:31.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.454+0000 7fc3cb200640 1 --2- 172.21.15.53:0/2422491893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3cc105220 0x7fc3cc19a770 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:31.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.454+0000 7fc3cb200640 1 --2- 172.21.15.53:0/2422491893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3cc1028f0 0x7fc3cc19a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:31.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.454+0000 7fc3cb200640 1 -- 172.21.15.53:0/2422491893 >> 172.21.15.53:0/2422491893 conn(0x7fc3cc0fc240 msgr2=0x7fc3cc102120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:31.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.454+0000 7fc3cb200640 1 -- 172.21.15.53:0/2422491893 shutdown_connections 2024-04-25T14:40:31.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:31.454+0000 7fc3cb200640 1 -- 172.21.15.53:0/2422491893 wait complete. 2024-04-25T14:40:31.469 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:40:31 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:40:31.153+0000 7f49022f5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:40:31.469 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:40:31 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:40:31.294+0000 7f49022f5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:40:31.471 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T14:40:31.494 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T14:40:31.494 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-25T14:40:31.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:40:31.495 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2024-04-25T14:40:31.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:40:31.548 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link delete ceph-brx 2024-04-25T14:40:31.631 INFO:teuthology.orchestra.run.smithi053.stderr:Cannot find device "ceph-brx" 2024-04-25T14:40:31.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:40:31.635 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-04-25T14:40:31.635 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T14:40:31.635 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T14:40:31.635 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-25T14:40:31.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:31 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/2088410643' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T14:40:31.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:31 smithi183 ceph-mon[32584]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-04-25T14:40:31.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:31 smithi183 ceph-mon[32584]: from='client.? ' entity='client.admin' 2024-04-25T14:40:31.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:31 smithi183 ceph-mon[32584]: from='client.? ' entity='client.admin' 2024-04-25T14:40:31.704 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:40:31 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:40:31.326+0000 7f03dde68200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:40:31.704 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:40:31 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:40:31.410+0000 7f03dde68200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:40:31.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:31 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/2088410643' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T14:40:31.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:31 smithi150 ceph-mon[32504]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-04-25T14:40:31.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:31 smithi150 ceph-mon[32504]: from='client.? ' entity='client.admin' 2024-04-25T14:40:31.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:31 smithi150 ceph-mon[32504]: from='client.? ' entity='client.admin' 2024-04-25T14:40:31.710 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:40:31 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:40:31.333+0000 7f17f85b5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:40:31.711 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:40:31 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:40:31.419+0000 7f17f85b5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:40:31.782 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:40:31 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:40:31.711+0000 7f49022f5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:40:31.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:31 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/2088410643' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T14:40:31.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:31 smithi053 ceph-mon[27773]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-04-25T14:40:31.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:31 smithi053 ceph-mon[27773]: from='client.? ' entity='client.admin' 2024-04-25T14:40:31.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:31 smithi053 ceph-mon[27773]: from='client.? ' entity='client.admin' 2024-04-25T14:40:32.075 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:40:32.221 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:40:31 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:40:31.830+0000 7f49022f5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:40:32.916 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:40:32 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:40:32.580+0000 7f03dde68200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:40:32.917 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:40:32 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:40:32.747+0000 7f03dde68200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:40:32.950 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:40:32 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:40:32.608+0000 7f17f85b5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:40:32.950 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:40:32 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:40:32.779+0000 7f17f85b5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:40:32.950 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:40:32 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:40:32.950+0000 7f17f85b5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:40:33.200 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:40:32 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:40:32.916+0000 7f03dde68200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:40:33.454 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:40:33 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:40:33.200+0000 7f03dde68200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:40:33.454 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:40:33 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:40:33.291+0000 7f03dde68200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:40:33.533 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:40:33 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:40:33.447+0000 7f49022f5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:40:33.539 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:40:33 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:40:33.240+0000 7f17f85b5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:40:33.539 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:40:33 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:40:33.333+0000 7f17f85b5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:40:33.539 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:40:33 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:40:33.538+0000 7f17f85b5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:40:33.914 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:40:33 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:40:33.687+0000 7f49022f5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:40:33.953 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:40:33 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:40:33.493+0000 7f03dde68200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:40:33.954 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:40:33 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:40:33.685+0000 7f03dde68200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:40:33.960 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:40:33 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:40:33.734+0000 7f17f85b5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:40:34.282 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:40:33 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:40:33.912+0000 7f49022f5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:40:34.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.558+0000 7ffb86800700 1 -- 172.21.15.53:0/2772677240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb80103450 msgr2=0x7ffb8010f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:34.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.558+0000 7ffb86800700 1 --2- 172.21.15.53:0/2772677240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb80103450 0x7ffb8010f8e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ffb7800b0a0 tx=0x7ffb7800b3b0 comp rx=0 tx=0).stop 2024-04-25T14:40:34.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.558+0000 7ffb86800700 1 -- 172.21.15.53:0/2772677240 shutdown_connections 2024-04-25T14:40:34.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.558+0000 7ffb86800700 1 --2- 172.21.15.53:0/2772677240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb80103450 0x7ffb8010f8e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:34.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.558+0000 7ffb86800700 1 --2- 172.21.15.53:0/2772677240 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb80108a80 0x7ffb80108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:34.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.558+0000 7ffb86800700 1 --2- 172.21.15.53:0/2772677240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb80102a90 0x7ffb80102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:34.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.558+0000 7ffb86800700 1 -- 172.21.15.53:0/2772677240 >> 172.21.15.53:0/2772677240 conn(0x7ffb80079bf0 msgr2=0x7ffb8007a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:34.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.558+0000 7ffb86800700 1 -- 172.21.15.53:0/2772677240 shutdown_connections 2024-04-25T14:40:34.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.558+0000 7ffb86800700 1 -- 172.21.15.53:0/2772677240 wait complete. 2024-04-25T14:40:34.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.559+0000 7ffb86800700 1 Processor -- start 2024-04-25T14:40:34.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.559+0000 7ffb86800700 1 -- start start 2024-04-25T14:40:34.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.560+0000 7ffb86800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb80102a90 0x7ffb8019b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:34.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.560+0000 7ffb86800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb80103450 0x7ffb8019b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:34.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.560+0000 7ffb86800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb80108a80 0x7ffb8019fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:34.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.560+0000 7ffb86800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ffb80112c80 con 0x7ffb80103450 2024-04-25T14:40:34.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.560+0000 7ffb86800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ffb80112b00 con 0x7ffb80102a90 2024-04-25T14:40:34.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.560+0000 7ffb86800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffb80112e00 con 0x7ffb80108a80 2024-04-25T14:40:34.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.560+0000 7ffb7f400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb80102a90 0x7ffb8019b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:34.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.560+0000 7ffb7fe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb80108a80 0x7ffb8019fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:34.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.560+0000 7ffb7f400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb80102a90 0x7ffb8019b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.53:60846/0 (socket says 172.21.15.53:60846) 2024-04-25T14:40:34.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.560+0000 7ffb7f400700 1 -- 172.21.15.53:0/3639041952 learned_addr learned my addr 172.21.15.53:0/3639041952 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:34.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.561+0000 7ffb7ea00700 1 --2- 172.21.15.53:0/3639041952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb80103450 0x7ffb8019b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:34.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.561+0000 7ffb7fe00700 1 -- 172.21.15.53:0/3639041952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb80102a90 msgr2=0x7ffb8019b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:34.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.561+0000 7ffb7fe00700 1 --2- 172.21.15.53:0/3639041952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb80102a90 0x7ffb8019b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:34.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.561+0000 7ffb7fe00700 1 -- 172.21.15.53:0/3639041952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb80103450 msgr2=0x7ffb8019b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:34.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.561+0000 7ffb7fe00700 1 --2- 172.21.15.53:0/3639041952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb80103450 0x7ffb8019b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:34.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.561+0000 7ffb7fe00700 1 -- 172.21.15.53:0/3639041952 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb801a03a0 con 0x7ffb80108a80 2024-04-25T14:40:34.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.561+0000 7ffb7f400700 1 --2- 172.21.15.53:0/3639041952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb80102a90 0x7ffb8019b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:40:34.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.561+0000 7ffb7ea00700 1 --2- 172.21.15.53:0/3639041952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb80103450 0x7ffb8019b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:40:34.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.561+0000 7ffb7fe00700 1 --2- 172.21.15.53:0/3639041952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb80108a80 0x7ffb8019fc60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ffb7800b070 tx=0x7ffb78004d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:34.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.562+0000 7ffb77e00700 1 -- 172.21.15.53:0/3639041952 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb78021070 con 0x7ffb80108a80 2024-04-25T14:40:34.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.562+0000 7ffb77e00700 1 -- 172.21.15.53:0/3639041952 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb78007e30 con 0x7ffb80108a80 2024-04-25T14:40:34.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.562+0000 7ffb77e00700 1 -- 172.21.15.53:0/3639041952 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb78004660 con 0x7ffb80108a80 2024-04-25T14:40:34.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.562+0000 7ffb86800700 1 -- 172.21.15.53:0/3639041952 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb801a0630 con 0x7ffb80108a80 2024-04-25T14:40:34.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.562+0000 7ffb86800700 1 -- 172.21.15.53:0/3639041952 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb801a7f10 con 0x7ffb80108a80 2024-04-25T14:40:34.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.563+0000 7ffb86800700 1 -- 172.21.15.53:0/3639041952 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb80069730 con 0x7ffb80108a80 2024-04-25T14:40:34.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.565+0000 7ffb77e00700 1 -- 172.21.15.53:0/3639041952 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7ffb78003c90 con 0x7ffb80108a80 2024-04-25T14:40:34.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.566+0000 7ffb77e00700 1 --2- 172.21.15.53:0/3639041952 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ffb640a3d10 0x7ffb640a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:34.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.566+0000 7ffb7f400700 1 -- 172.21.15.53:0/3639041952 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ffb640a3d10 msgr2=0x7ffb640a61d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/4154933108 2024-04-25T14:40:34.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.566+0000 7ffb77e00700 1 -- 172.21.15.53:0/3639041952 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8285+0+0 (secure 0 0 0) 0x7ffb7800f020 con 0x7ffb80108a80 2024-04-25T14:40:34.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.566+0000 7ffb7f400700 1 --2- 172.21.15.53:0/3639041952 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ffb640a3d10 0x7ffb640a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:40:34.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.571+0000 7ffb77e00700 1 -- 172.21.15.53:0/3639041952 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffb7802a020 con 0x7ffb80108a80 2024-04-25T14:40:34.699 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:40:34 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:40:34.280+0000 7f03dde68200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T14:40:34.710 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:40:34 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:40:34.336+0000 7f17f85b5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T14:40:34.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.767+0000 7ffb7f400700 1 -- 172.21.15.53:0/3639041952 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ffb640a3d10 msgr2=0x7ffb640a61d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/4154933108 2024-04-25T14:40:34.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.767+0000 7ffb7f400700 1 --2- 172.21.15.53:0/3639041952 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ffb640a3d10 0x7ffb640a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T14:40:34.769 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:40:34 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:40:34.335+0000 7f49022f5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:40:34.769 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:40:34 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:40:34.496+0000 7f49022f5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:40:34.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.849+0000 7ffb86800700 1 -- 172.21.15.53:0/3639041952 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ffb801a80e0 con 0x7ffb80108a80 2024-04-25T14:40:34.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.849+0000 7ffb77e00700 1 -- 172.21.15.53:0/3639041952 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v21) ==== 53+0+73 (secure 0 0 0) 0x7ffb7809e1e0 con 0x7ffb80108a80 2024-04-25T14:40:34.850 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T14:40:34.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.853+0000 7ffb86800700 1 -- 172.21.15.53:0/3639041952 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ffb640a3d10 msgr2=0x7ffb640a61d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:40:34.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.853+0000 7ffb86800700 1 --2- 172.21.15.53:0/3639041952 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ffb640a3d10 0x7ffb640a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:34.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.853+0000 7ffb86800700 1 -- 172.21.15.53:0/3639041952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb80108a80 msgr2=0x7ffb8019fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:34.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.853+0000 7ffb86800700 1 --2- 172.21.15.53:0/3639041952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb80108a80 0x7ffb8019fc60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ffb7800b070 tx=0x7ffb78004d30 comp rx=0 tx=0).stop 2024-04-25T14:40:34.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.853+0000 7ffb86800700 1 -- 172.21.15.53:0/3639041952 shutdown_connections 2024-04-25T14:40:34.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.853+0000 7ffb86800700 1 --2- 172.21.15.53:0/3639041952 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7ffb640a3d10 0x7ffb640a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:34.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.853+0000 7ffb86800700 1 --2- 172.21.15.53:0/3639041952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb80108a80 0x7ffb8019fc60 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:34.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.853+0000 7ffb86800700 1 --2- 172.21.15.53:0/3639041952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb80102a90 0x7ffb8019b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:34.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.853+0000 7ffb86800700 1 --2- 172.21.15.53:0/3639041952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb80103450 0x7ffb8019b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:34.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.853+0000 7ffb86800700 1 -- 172.21.15.53:0/3639041952 >> 172.21.15.53:0/3639041952 conn(0x7ffb80079bf0 msgr2=0x7ffb8010dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:34.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.853+0000 7ffb86800700 1 -- 172.21.15.53:0/3639041952 shutdown_connections 2024-04-25T14:40:34.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:34.853+0000 7ffb86800700 1 -- 172.21.15.53:0/3639041952 wait complete. 2024-04-25T14:40:34.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:34 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/3639041952' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T14:40:34.954 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:40:34 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:40:34.700+0000 7f03dde68200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:40:34.954 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:40:34 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:40:34.794+0000 7f03dde68200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:40:35.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:34 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/3639041952' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T14:40:35.033 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:40:34 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:40:34.788+0000 7f49022f5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:40:35.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:34 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/3639041952' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T14:40:35.068 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:40:34 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:40:34.767+0000 7f17f85b5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:40:35.068 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:40:34 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:40:34.864+0000 7f17f85b5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:40:35.453 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:40:34 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:40:34.996+0000 7f03dde68200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T14:40:35.460 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:40:35 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:40:35.067+0000 7f17f85b5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T14:40:35.481 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:40:35 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:40:35.075+0000 7f49022f5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:40:35.532 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T14:40:35.532 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T14:40:35.532 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2024-04-25T14:40:35.532 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T14:40:35.532 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T14:40:35.532 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T14:40:35.533 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T14:40:35.533 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T14:40:35.533 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T14:40:35.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:40:35.533 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2024-04-25T14:40:35.578 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T14:40:35.578 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T14:40:35.578 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T14:40:35.578 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:40:35.578 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2024-04-25T14:40:35.578 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:40:35.578 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T14:40:35.578 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2024-04-25T14:40:35.578 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2024-04-25T14:40:35.578 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T14:40:35.579 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2024-04-25T14:40:35.579 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2024-04-25T14:40:35.579 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T14:40:35.579 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:40:35.579 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2024-04-25T14:40:35.579 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T14:40:35.579 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 41988sec preferred_lft 41988sec 2024-04-25T14:40:35.579 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link noprefixroute 2024-04-25T14:40:35.579 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:40:35.579 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T14:40:35.579 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:40:35.579 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2024-04-25T14:40:35.580 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T14:40:35.580 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:40:35.580 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-04-25T14:40:35.580 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link add name ceph-brx type bridge 2024-04-25T14:40:35.580 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip addr flush dev ceph-brx 2024-04-25T14:40:35.580 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set ceph-brx up 2024-04-25T14:40:35.580 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T14:40:35.580 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-04-25T14:40:35.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:40:35.866 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:40:35 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:40:35.529+0000 7f03dde68200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:40:35.866 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:40:35 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:40:35.620+0000 7f03dde68200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:40:35.934 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:40:35 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:40:35.596+0000 7f17f85b5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:40:35.935 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:40:35 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:40:35.689+0000 7f17f85b5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:40:35.993 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:40:35 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:40:35.953+0000 7f49022f5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T14:40:36.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:40:36.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:40:36.112 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T14:40:36.176 INFO:teuthology.orchestra.run.smithi053.stdout:1 2024-04-25T14:40:36.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:40:36.180 DEBUG:teuthology.orchestra.run.smithi053:> route 2024-04-25T14:40:36.203 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:40:35 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:40:35.866+0000 7f03dde68200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:40:36.203 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:40:36 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:40:35.999+0000 7f03dde68200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:40:36.203 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:40:36 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:40:36.095+0000 7f03dde68200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:40:36.210 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:40:35 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:40:35.933+0000 7f17f85b5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:40:36.210 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:40:36 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:40:36.065+0000 7f17f85b5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:40:36.210 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:40:36 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:40:36.162+0000 7f17f85b5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:40:36.226 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2024-04-25T14:40:36.226 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T14:40:36.226 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T14:40:36.226 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T14:40:36.226 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T14:40:36.226 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:40:36.227 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-04-25T14:40:36.227 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T14:40:36.227 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T14:40:36.227 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T14:40:36.227 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-04-25T14:40:36.359 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:40:36.513 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:40:36 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:40:36.209+0000 7f03dde68200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:40:36.585 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:40:36 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:40:36.278+0000 7f17f85b5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:40:36.636 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:40:36 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:40:36.555+0000 7f49022f5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:40:36.953 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:40:36 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:40:36.514+0000 7f03dde68200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:40:36.974 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:40:36 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:40:36.691+0000 7f49022f5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:40:37.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:40:37.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:40:37.011 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2024-04-25T14:40:37.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:40:37.062 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list-id 2024-04-25T14:40:37.110 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:40:37.110 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-04-25T14:40:37.110 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T14:40:37.110 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T14:40:37.110 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-04-25T14:40:37.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:40:37.236 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:40:36 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:40:36.973+0000 7f49022f5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T14:40:37.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:40:37.294 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T14:40:37.294 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:40:37.295 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-04-25T14:40:37.295 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T14:40:37.295 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T14:40:37.295 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T14:40:37.295 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T14:40:37.295 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T14:40:37.295 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-04-25T14:40:37.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:40:37.405 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:40:37 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:40:37.113+0000 7f03dde68200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:40:37.405 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:40:37 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z[34454]: 2024-04-25T14:40:37.328+0000 7f03dde68200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:40:37.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:37 smithi150 ceph-mon[32504]: Standby manager daemon z restarted 2024-04-25T14:40:37.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:37 smithi150 ceph-mon[32504]: Standby manager daemon z started 2024-04-25T14:40:37.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:37 smithi150 ceph-mon[32504]: from='mgr.? 172.21.15.183:0/1470005397' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T14:40:37.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:37 smithi150 ceph-mon[32504]: from='mgr.? 172.21.15.183:0/1470005397' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T14:40:37.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:37 smithi150 ceph-mon[32504]: from='mgr.? 172.21.15.183:0/1470005397' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T14:40:37.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:37 smithi150 ceph-mon[32504]: from='mgr.? 172.21.15.183:0/1470005397' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T14:40:37.461 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:40:36 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:40:36.583+0000 7f17f85b5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:40:37.461 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:40:37 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:40:37.191+0000 7f17f85b5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:40:37.461 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:40:37 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y[33879]: 2024-04-25T14:40:37.415+0000 7f17f85b5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:40:37.521 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:37 smithi053 ceph-mon[27773]: Standby manager daemon z restarted 2024-04-25T14:40:37.521 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:37 smithi053 ceph-mon[27773]: Standby manager daemon z started 2024-04-25T14:40:37.521 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:37 smithi053 ceph-mon[27773]: from='mgr.? 172.21.15.183:0/1470005397' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T14:40:37.521 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:37 smithi053 ceph-mon[27773]: from='mgr.? 172.21.15.183:0/1470005397' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T14:40:37.521 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:37 smithi053 ceph-mon[27773]: from='mgr.? 172.21.15.183:0/1470005397' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T14:40:37.521 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:37 smithi053 ceph-mon[27773]: from='mgr.? 172.21.15.183:0/1470005397' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T14:40:37.638 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:40:37.647 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:40:37.647 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-04-25T14:40:37.647 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2024-04-25T14:40:37.647 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T14:40:37.648 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-04-25T14:40:37.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:37 smithi183 ceph-mon[32584]: Standby manager daemon z restarted 2024-04-25T14:40:37.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:37 smithi183 ceph-mon[32584]: Standby manager daemon z started 2024-04-25T14:40:37.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:37 smithi183 ceph-mon[32584]: from='mgr.? 172.21.15.183:0/1470005397' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T14:40:37.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:37 smithi183 ceph-mon[32584]: from='mgr.? 172.21.15.183:0/1470005397' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T14:40:37.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:37 smithi183 ceph-mon[32584]: from='mgr.? 172.21.15.183:0/1470005397' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T14:40:37.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:37 smithi183 ceph-mon[32584]: from='mgr.? 172.21.15.183:0/1470005397' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T14:40:37.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:40:37.771 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:40:37 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:40:37.677+0000 7f49022f5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:40:37.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:40:37.847 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:40:37.847 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T14:40:37.890 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T14:40:37.890 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:40:37.890 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T14:40:37.976 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-25T14:40:37.977 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-25T14:40:37.977 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/df7995a7-188f-4b69-96c8-3b33c8c560c6 2024-04-25T14:40:37.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:40:37.977 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/df7995a7-188f-4b69-96c8-3b33c8c560c6 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-25T14:40:38.032 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:40:37 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:40:37.809+0000 7f49022f5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:40:38.129 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-25T14:40:38.129 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2024-04-25T14:40:38.352 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:40:38 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:40:38.151+0000 7f49022f5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:40:38.353 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:40:38 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:40:38.350+0000 7f49022f5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:40:38.663 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:40:38 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:40:38.499+0000 7f49022f5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:40:38.663 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:40:38 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:40:38.661+0000 7f49022f5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:40:38.663 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:38 smithi053 ceph-mon[27773]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-04-25T14:40:38.663 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:38 smithi053 ceph-mon[27773]: Standby manager daemon y restarted 2024-04-25T14:40:38.663 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:38 smithi053 ceph-mon[27773]: Standby manager daemon y started 2024-04-25T14:40:38.663 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:38 smithi053 ceph-mon[27773]: from='mgr.? 172.21.15.150:0/1901922170' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T14:40:38.663 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:38 smithi053 ceph-mon[27773]: from='mgr.? 172.21.15.150:0/1901922170' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T14:40:38.663 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:38 smithi053 ceph-mon[27773]: from='mgr.? 172.21.15.150:0/1901922170' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T14:40:38.663 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:38 smithi053 ceph-mon[27773]: from='mgr.? 172.21.15.150:0/1901922170' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T14:40:38.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:38 smithi183 ceph-mon[32584]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-04-25T14:40:38.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:38 smithi183 ceph-mon[32584]: Standby manager daemon y restarted 2024-04-25T14:40:38.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:38 smithi183 ceph-mon[32584]: Standby manager daemon y started 2024-04-25T14:40:38.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:38 smithi183 ceph-mon[32584]: from='mgr.? 172.21.15.150:0/1901922170' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T14:40:38.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:38 smithi183 ceph-mon[32584]: from='mgr.? 172.21.15.150:0/1901922170' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T14:40:38.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:38 smithi183 ceph-mon[32584]: from='mgr.? 172.21.15.150:0/1901922170' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T14:40:38.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:38 smithi183 ceph-mon[32584]: from='mgr.? 172.21.15.150:0/1901922170' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T14:40:38.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:38 smithi150 ceph-mon[32504]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-04-25T14:40:38.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:38 smithi150 ceph-mon[32504]: Standby manager daemon y restarted 2024-04-25T14:40:38.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:38 smithi150 ceph-mon[32504]: Standby manager daemon y started 2024-04-25T14:40:38.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:38 smithi150 ceph-mon[32504]: from='mgr.? 172.21.15.150:0/1901922170' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T14:40:38.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:38 smithi150 ceph-mon[32504]: from='mgr.? 172.21.15.150:0/1901922170' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T14:40:38.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:38 smithi150 ceph-mon[32504]: from='mgr.? 172.21.15.150:0/1901922170' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T14:40:38.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:38 smithi150 ceph-mon[32504]: from='mgr.? 172.21.15.150:0/1901922170' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T14:40:39.038 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-25T14:40:39.039 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2024-04-25T14:40:39.039 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.150:3300,172.21.15.183:3300" 2024-04-25T14:40:39.039 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with new device syntax: 0@fd30e6ce-030f-11ef-bc93-c7b262605968.cephfs=/volumes/qa/sv_1/df7995a7-188f-4b69-96c8-3b33c8c560c6 2024-04-25T14:40:39.039 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync,name=0,key=0,mon_addr=172.21.15.53:3300/172.21.15.150:3300/172.21.15.183:3300" will pass to kernel 2024-04-25T14:40:39.044 INFO:teuthology.orchestra.run.smithi053.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T14:40:39.045 INFO:teuthology.orchestra.run.smithi053.stdout: You just mounted a file system that supports labels which does not 2024-04-25T14:40:39.045 INFO:teuthology.orchestra.run.smithi053.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T14:40:39.045 INFO:teuthology.orchestra.run.smithi053.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T14:40:39.045 INFO:teuthology.orchestra.run.smithi053.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T14:40:39.049 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T14:40:39.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:40:39.050 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T14:40:39.164 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T14:40:39.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T14:40:39.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:40:39.277 INFO:teuthology.orchestra.run.smithi053.stdout:client25249 2024-04-25T14:40:39.277 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-25T14:40:39.278 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:40:39.284 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:40:39 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:40:39.045+0000 7f49022f5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:40:39.325 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T14:40:39.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T14:40:39.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:40:39.478 INFO:teuthology.orchestra.run.smithi053.stdout:client25249 2024-04-25T14:40:39.478 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T14:40:39.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T14:40:39.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:40:39.598 INFO:teuthology.orchestra.run.smithi053.stdout:client25249 2024-04-25T14:40:39.599 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968.client25249/status') 2024-04-25T14:40:39.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:40:39.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:39 smithi183 ceph-mon[32584]: mgrmap e21: x(active, since 10m), standbys: y, z 2024-04-25T14:40:39.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:39 smithi150 ceph-mon[32504]: mgrmap e21: x(active, since 10m), standbys: y, z 2024-04-25T14:40:39.722 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:39 smithi053 ceph-mon[27773]: mgrmap e21: x(active, since 10m), standbys: y, z 2024-04-25T14:40:39.730 INFO:teuthology.orchestra.run.smithi053.stdout:instance: client.25249 (3)172.21.15.53:0/1166055124 2024-04-25T14:40:39.730 INFO:teuthology.orchestra.run.smithi053.stdout:blocklisted: false 2024-04-25T14:40:39.730 INFO:teuthology.orchestra.run.smithi053.stderr:0+1 records in 2024-04-25T14:40:39.730 INFO:teuthology.orchestra.run.smithi053.stderr:0+1 records out 2024-04-25T14:40:39.730 INFO:teuthology.orchestra.run.smithi053.stderr:71 bytes copied, 6.7778e-05 s, 1.0 MB/s 2024-04-25T14:40:39.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:40:39.742 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T14:40:39.756 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T14:40:39.757 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi053.front.sepia.ceph.com 2024-04-25T14:40:39.757 DEBUG:teuthology.orchestra.run.smithi053:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-04-25T14:40:39.940 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39.939+0000 7f03bbe00640 1 -- 172.21.15.53:0/1416159043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03bc0784f0 msgr2=0x7f03bc078d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:39.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39.939+0000 7f03bbe00640 1 --2- 172.21.15.53:0/1416159043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03bc0784f0 0x7f03bc078d90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f03b4009be0 tx=0x7f03b402a6c0 comp rx=0 tx=0).stop 2024-04-25T14:40:39.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39.939+0000 7f03bbe00640 1 -- 172.21.15.53:0/1416159043 shutdown_connections 2024-04-25T14:40:39.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39.939+0000 7f03bbe00640 1 --2- 172.21.15.53:0/1416159043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03bc0784f0 0x7f03bc078d90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:39.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39.939+0000 7f03bbe00640 1 --2- 172.21.15.53:0/1416159043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03bc077b50 0x7f03bc077fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:39.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39.939+0000 7f03bbe00640 1 --2- 172.21.15.53:0/1416159043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f03bc076910 0x7f03bc076d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:39.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39.939+0000 7f03bbe00640 1 -- 172.21.15.53:0/1416159043 >> 172.21.15.53:0/1416159043 conn(0x7f03bc0ff560 msgr2=0x7f03bc1019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:39.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39.940+0000 7f03bbe00640 1 -- 172.21.15.53:0/1416159043 shutdown_connections 2024-04-25T14:40:39.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39.940+0000 7f03bbe00640 1 -- 172.21.15.53:0/1416159043 wait complete. 2024-04-25T14:40:39.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39.941+0000 7f03bbe00640 1 Processor -- start 2024-04-25T14:40:39.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39.941+0000 7f03bbe00640 1 -- start start 2024-04-25T14:40:39.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39.941+0000 7f03bbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03bc076910 0x7f03bc19e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:39.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39.941+0000 7f03bbe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03bc077b50 0x7f03bc19eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:39.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39.941+0000 7f03bbe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f03bc0784f0 0x7f03bc1a5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:39.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39.941+0000 7f03bbe00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f03bc113f30 con 0x7f03bc077b50 2024-04-25T14:40:39.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39.941+0000 7f03bbe00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f03bc113db0 con 0x7f03bc0784f0 2024-04-25T14:40:39.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39.941+0000 7f03bbe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f03bc1140b0 con 0x7f03bc076910 2024-04-25T14:40:39.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39.942+0000 7f03b3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03bc077b50 0x7f03bc19eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:39.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39.942+0000 7f03b3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03bc077b50 0x7f03bc19eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33294/0 (socket says 172.21.15.53:33294) 2024-04-25T14:40:39.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39.942+0000 7f03b3e00640 1 -- 172.21.15.53:0/975417718 learned_addr learned my addr 172.21.15.53:0/975417718 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:39.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39.942+0000 7f03baa00640 1 --2- 172.21.15.53:0/975417718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03bc076910 0x7f03bc19e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:39.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39.942+0000 7f03bb400640 1 --2- 172.21.15.53:0/975417718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f03bc0784f0 0x7f03bc1a5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:39.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39.942+0000 7f03b3e00640 1 -- 172.21.15.53:0/975417718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03bc076910 msgr2=0x7f03bc19e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:39.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39.942+0000 7f03b3e00640 1 --2- 172.21.15.53:0/975417718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03bc076910 0x7f03bc19e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:39.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39.942+0000 7f03b3e00640 1 -- 172.21.15.53:0/975417718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f03bc0784f0 msgr2=0x7f03bc1a5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:39.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39.942+0000 7f03b3e00640 1 --2- 172.21.15.53:0/975417718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f03bc0784f0 0x7f03bc1a5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:39.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39.942+0000 7f03b3e00640 1 -- 172.21.15.53:0/975417718 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03bc1a61e0 con 0x7f03bc077b50 2024-04-25T14:40:39.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39.942+0000 7f03baa00640 1 --2- 172.21.15.53:0/975417718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03bc076910 0x7f03bc19e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:40:39.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39.942+0000 7f03b3e00640 1 --2- 172.21.15.53:0/975417718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03bc077b50 0x7f03bc19eb00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f03a400e9f0 tx=0x7f03a400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:39.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39.943+0000 7f03b3400640 1 -- 172.21.15.53:0/975417718 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a400ccc0 con 0x7f03bc077b50 2024-04-25T14:40:39.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39.943+0000 7f03b3400640 1 -- 172.21.15.53:0/975417718 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03a40108e0 con 0x7f03bc077b50 2024-04-25T14:40:39.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39.943+0000 7f03bbe00640 1 -- 172.21.15.53:0/975417718 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03bc1a64d0 con 0x7f03bc077b50 2024-04-25T14:40:39.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39.943+0000 7f03b3400640 1 -- 172.21.15.53:0/975417718 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a4018880 con 0x7f03bc077b50 2024-04-25T14:40:39.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39.943+0000 7f03bbe00640 1 -- 172.21.15.53:0/975417718 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03bc1a6a20 con 0x7f03bc077b50 2024-04-25T14:40:39.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39.944+0000 7f03bbe00640 1 -- 172.21.15.53:0/975417718 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03bc110960 con 0x7f03bc077b50 2024-04-25T14:40:39.946 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39.946+0000 7f03b3400640 1 -- 172.21.15.53:0/975417718 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 21) ==== 138663+0+0 (secure 0 0 0) 0x7f03a4010430 con 0x7f03bc077b50 2024-04-25T14:40:39.948 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39.947+0000 7f03b3400640 1 --2- 172.21.15.53:0/975417718 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f03a00a3bf0 0x7f03a00a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:39.948 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39.948+0000 7f03baa00640 1 -- 172.21.15.53:0/975417718 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f03a00a3bf0 msgr2=0x7f03a00a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/4154933108 2024-04-25T14:40:39.948 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39.948+0000 7f03b3400640 1 -- 172.21.15.53:0/975417718 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8285+0+0 (secure 0 0 0) 0x7f03a40d0890 con 0x7f03bc077b50 2024-04-25T14:40:39.948 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39.948+0000 7f03baa00640 1 --2- 172.21.15.53:0/975417718 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f03a00a3bf0 0x7f03a00a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:40:39.953 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:39.952+0000 7f03b3400640 1 -- 172.21.15.53:0/975417718 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f03a409d1e0 con 0x7f03bc077b50 2024-04-25T14:40:40.032 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:40:39 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:40:39.858+0000 7f49022f5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:40:40.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:40.149+0000 7f03baa00640 1 -- 172.21.15.53:0/975417718 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f03a00a3bf0 msgr2=0x7f03a00a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.53:6800/4154933108 2024-04-25T14:40:40.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:40.149+0000 7f03baa00640 1 --2- 172.21.15.53:0/975417718 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f03a00a3bf0 0x7f03a00a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T14:40:40.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:40.187+0000 7f03b3400640 1 -- 172.21.15.53:0/975417718 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 22) ==== 138331+0+0 (secure 0 0 0) 0x7f03a4099150 con 0x7f03bc077b50 2024-04-25T14:40:40.188 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:40.187+0000 7f03b3400640 1 -- 172.21.15.53:0/975417718 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f03a00a3bf0 msgr2=0x7f03a00a60b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:40:40.188 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:40.187+0000 7f03b3400640 1 --2- 172.21.15.53:0/975417718 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f03a00a3bf0 0x7f03a00a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:40.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:40.220+0000 7f03bbe00640 1 -- 172.21.15.53:0/975417718 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f03bc02d5c0 con 0x7f03bc077b50 2024-04-25T14:40:40.436 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:40:40.164+0000 7f49022f5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:40:40.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: Active manager daemon x restarted 2024-04-25T14:40:40.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: Activating manager daemon x 2024-04-25T14:40:40.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T14:40:40.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: mgrmap e22: x(active, starting, since 0.0143765s), standbys: y, z 2024-04-25T14:40:40.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:40:40.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:40:40.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:40:40.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T14:40:40.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T14:40:40.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T14:40:40.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T14:40:40.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T14:40:40.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T14:40:40.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T14:40:40.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T14:40:40.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T14:40:40.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T14:40:40.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T14:40:40.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T14:40:40.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T14:40:40.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T14:40:40.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T14:40:40.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:40:40.706 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:40:40.706 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:40:40.706 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:40:40.706 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:40:40.706 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:40:40.706 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:40:40.706 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:40:40.706 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:40:40.706 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:40:40.706 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:40:40.706 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:40:40.706 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T14:40:40.706 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T14:40:40.707 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T14:40:40.707 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: Manager daemon x is now available 2024-04-25T14:40:40.707 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/975417718' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-25T14:40:40.707 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:40:40.707 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T14:40:40.707 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T14:40:40.707 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T14:40:40.707 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:40 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T14:40:40.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: Active manager daemon x restarted 2024-04-25T14:40:40.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: Activating manager daemon x 2024-04-25T14:40:40.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T14:40:40.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: mgrmap e22: x(active, starting, since 0.0143765s), standbys: y, z 2024-04-25T14:40:40.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:40:40.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:40:40.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:40:40.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T14:40:40.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T14:40:40.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T14:40:40.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T14:40:40.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T14:40:40.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T14:40:40.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T14:40:40.712 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T14:40:40.712 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T14:40:40.712 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T14:40:40.712 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T14:40:40.712 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T14:40:40.712 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T14:40:40.712 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T14:40:40.712 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T14:40:40.712 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:40:40.712 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:40:40.712 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:40:40.712 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:40:40.712 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:40:40.712 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:40:40.712 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:40:40.712 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:40:40.712 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:40:40.713 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:40:40.713 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:40:40.713 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:40:40.713 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T14:40:40.713 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T14:40:40.713 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T14:40:40.713 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: Manager daemon x is now available 2024-04-25T14:40:40.713 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/975417718' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-25T14:40:40.713 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:40:40.713 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T14:40:40.713 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T14:40:40.713 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T14:40:40.713 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:40 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T14:40:40.782 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:40:40.472+0000 7f4858600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:40:40.782 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:40:40.472+0000 7f4858600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:40:40.782 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:40:40.472+0000 7f4858600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:40:40.782 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:40:40.472+0000 7f4858600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:40:40.783 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:40:40.472+0000 7f4858600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:40:40.783 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:40:40.472+0000 7f4854000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:40:40.783 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:40:40.472+0000 7f4854000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:40:40.783 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:40:40.472+0000 7f4854000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:40:40.783 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:40:40.473+0000 7f4854000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:40:40.783 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:40:40.473+0000 7f4854000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:40:40.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: Active manager daemon x restarted 2024-04-25T14:40:40.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: Activating manager daemon x 2024-04-25T14:40:40.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T14:40:40.784 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: mgrmap e22: x(active, starting, since 0.0143765s), standbys: y, z 2024-04-25T14:40:40.784 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:40:40.784 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:40:40.784 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:40:40.784 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T14:40:40.784 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T14:40:40.784 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T14:40:40.784 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T14:40:40.784 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T14:40:40.784 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T14:40:40.784 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T14:40:40.784 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T14:40:40.784 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T14:40:40.784 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T14:40:40.784 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T14:40:40.785 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T14:40:40.785 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T14:40:40.785 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T14:40:40.785 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T14:40:40.785 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:40:40.785 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:40:40.785 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:40:40.785 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:40:40.785 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:40:40.785 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:40:40.785 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:40:40.785 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:40:40.785 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:40:40.785 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:40:40.785 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:40:40.785 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:40:40.785 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T14:40:40.786 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T14:40:40.786 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T14:40:40.786 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: Manager daemon x is now available 2024-04-25T14:40:40.786 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/975417718' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-25T14:40:40.786 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:40:40.786 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T14:40:40.786 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T14:40:40.786 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T14:40:40.786 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:40 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T14:40:41.180 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.179+0000 7f03b3400640 1 -- 172.21.15.53:0/975417718 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v22) ==== 117+0+0 (secure 0 0 0) 0x7f03a408dc20 con 0x7f03bc077b50 2024-04-25T14:40:41.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.183+0000 7f03bbe00640 1 -- 172.21.15.53:0/975417718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03bc077b50 msgr2=0x7f03bc19eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:41.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.183+0000 7f03bbe00640 1 --2- 172.21.15.53:0/975417718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03bc077b50 0x7f03bc19eb00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f03a400e9f0 tx=0x7f03a400edf0 comp rx=0 tx=0).stop 2024-04-25T14:40:41.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.183+0000 7f03bbe00640 1 -- 172.21.15.53:0/975417718 shutdown_connections 2024-04-25T14:40:41.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.183+0000 7f03bbe00640 1 --2- 172.21.15.53:0/975417718 >> [v2:172.21.15.53:6800/4154933108,v1:172.21.15.53:6801/4154933108] conn(0x7f03a00a3bf0 0x7f03a00a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:41.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.183+0000 7f03bbe00640 1 --2- 172.21.15.53:0/975417718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f03bc0784f0 0x7f03bc1a5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:41.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.183+0000 7f03bbe00640 1 --2- 172.21.15.53:0/975417718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03bc077b50 0x7f03bc19eb00 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:41.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.183+0000 7f03bbe00640 1 --2- 172.21.15.53:0/975417718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03bc076910 0x7f03bc19e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:41.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.183+0000 7f03bbe00640 1 -- 172.21.15.53:0/975417718 >> 172.21.15.53:0/975417718 conn(0x7f03bc0ff560 msgr2=0x7f03bc101060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:41.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.184+0000 7f03bbe00640 1 -- 172.21.15.53:0/975417718 shutdown_connections 2024-04-25T14:40:41.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.184+0000 7f03bbe00640 1 -- 172.21.15.53:0/975417718 wait complete. 2024-04-25T14:40:41.213 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-25T14:40:41.223 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T14:40:41.231 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T14:40:41.231 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi053.front.sepia.ceph.com 2024-04-25T14:40:41.231 DEBUG:teuthology.orchestra.run.smithi053:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-25T14:40:41.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.410+0000 7f3c24400640 1 -- 172.21.15.53:0/1928743100 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c1c102a30 msgr2=0x7f3c1c104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:41.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.410+0000 7f3c24400640 1 --2- 172.21.15.53:0/1928743100 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c1c102a30 0x7f3c1c104e20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3c10009860 tx=0x7f3c1002a340 comp rx=0 tx=0).stop 2024-04-25T14:40:41.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.410+0000 7f3c24400640 1 -- 172.21.15.53:0/1928743100 shutdown_connections 2024-04-25T14:40:41.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.410+0000 7f3c24400640 1 --2- 172.21.15.53:0/1928743100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3c1c107c90 0x7f3c1c10a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:41.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.410+0000 7f3c24400640 1 --2- 172.21.15.53:0/1928743100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c1c105360 0x7f3c1c107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:41.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.410+0000 7f3c24400640 1 --2- 172.21.15.53:0/1928743100 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c1c102a30 0x7f3c1c104e20 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:41.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.410+0000 7f3c24400640 1 -- 172.21.15.53:0/1928743100 >> 172.21.15.53:0/1928743100 conn(0x7f3c1c0fc440 msgr2=0x7f3c1c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:41.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.410+0000 7f3c24400640 1 -- 172.21.15.53:0/1928743100 shutdown_connections 2024-04-25T14:40:41.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.410+0000 7f3c24400640 1 -- 172.21.15.53:0/1928743100 wait complete. 2024-04-25T14:40:41.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.412+0000 7f3c24400640 1 Processor -- start 2024-04-25T14:40:41.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.412+0000 7f3c24400640 1 -- start start 2024-04-25T14:40:41.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.412+0000 7f3c24400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3c1c102a30 0x7f3c1c1023a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:41.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.412+0000 7f3c24400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c1c105360 0x7f3c1c1009e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:41.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.412+0000 7f3c24400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c1c107c90 0x7f3c1c100f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:41.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.412+0000 7f3c24400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c10daf0 con 0x7f3c1c105360 2024-04-25T14:40:41.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.412+0000 7f3c24400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c10d970 con 0x7f3c1c102a30 2024-04-25T14:40:41.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.412+0000 7f3c24400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c10dc70 con 0x7f3c1c107c90 2024-04-25T14:40:41.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.413+0000 7f3c21000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c1c105360 0x7f3c1c1009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:41.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.413+0000 7f3c21000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c1c105360 0x7f3c1c1009e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33442/0 (socket says 172.21.15.53:33442) 2024-04-25T14:40:41.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.413+0000 7f3c21000640 1 -- 172.21.15.53:0/2727042468 learned_addr learned my addr 172.21.15.53:0/2727042468 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:41.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.413+0000 7f3c21a00640 1 --2- 172.21.15.53:0/2727042468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3c1c102a30 0x7f3c1c1023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:41.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.413+0000 7f3c22400640 1 --2- 172.21.15.53:0/2727042468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c1c107c90 0x7f3c1c100f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:41.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.413+0000 7f3c21000640 1 -- 172.21.15.53:0/2727042468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c1c107c90 msgr2=0x7f3c1c100f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:41.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.413+0000 7f3c21000640 1 --2- 172.21.15.53:0/2727042468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c1c107c90 0x7f3c1c100f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:41.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.413+0000 7f3c21000640 1 -- 172.21.15.53:0/2727042468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3c1c102a30 msgr2=0x7f3c1c1023a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:41.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.413+0000 7f3c21000640 1 --2- 172.21.15.53:0/2727042468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3c1c102a30 0x7f3c1c1023a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:41.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.413+0000 7f3c21000640 1 -- 172.21.15.53:0/2727042468 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c1c1017b0 con 0x7f3c1c105360 2024-04-25T14:40:41.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.413+0000 7f3c21a00640 1 --2- 172.21.15.53:0/2727042468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3c1c102a30 0x7f3c1c1023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:40:41.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.413+0000 7f3c21000640 1 --2- 172.21.15.53:0/2727042468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c1c105360 0x7f3c1c1009e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3c0800e9f0 tx=0x7f3c0800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:41.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.414+0000 7f3c1a000640 1 -- 172.21.15.53:0/2727042468 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0800ccc0 con 0x7f3c1c105360 2024-04-25T14:40:41.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.414+0000 7f3c1a000640 1 -- 172.21.15.53:0/2727042468 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c080108e0 con 0x7f3c1c105360 2024-04-25T14:40:41.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.414+0000 7f3c24400640 1 -- 172.21.15.53:0/2727042468 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c1c1a2230 con 0x7f3c1c105360 2024-04-25T14:40:41.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.414+0000 7f3c1a000640 1 -- 172.21.15.53:0/2727042468 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c08018880 con 0x7f3c1c105360 2024-04-25T14:40:41.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.414+0000 7f3c24400640 1 -- 172.21.15.53:0/2727042468 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c1c1a2780 con 0x7f3c1c105360 2024-04-25T14:40:41.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.415+0000 7f3c24400640 1 -- 172.21.15.53:0/2727042468 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c1c0fdf30 con 0x7f3c1c105360 2024-04-25T14:40:41.418 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.417+0000 7f3c1a000640 1 -- 172.21.15.53:0/2727042468 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 23) ==== 138458+0+0 (secure 0 0 0) 0x7f3c08010430 con 0x7f3c1c105360 2024-04-25T14:40:41.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.419+0000 7f3c1a000640 1 --2- 172.21.15.53:0/2727042468 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f3c040a3a60 0x7f3c040a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:41.420 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.419+0000 7f3c1a000640 1 -- 172.21.15.53:0/2727042468 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8543+0+0 (secure 0 0 0) 0x7f3c080d0390 con 0x7f3c1c105360 2024-04-25T14:40:41.420 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.419+0000 7f3c21a00640 1 --2- 172.21.15.53:0/2727042468 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f3c040a3a60 0x7f3c040a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:41.420 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.420+0000 7f3c21a00640 1 --2- 172.21.15.53:0/2727042468 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f3c040a3a60 0x7f3c040a5f20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3c10000c00 tx=0x7f3c10030470 comp rx=0 tx=0).ready entity=mgr.15093 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:41.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.426+0000 7f3c1a000640 1 -- 172.21.15.53:0/2727042468 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3c0809c8a0 con 0x7f3c1c105360 2024-04-25T14:40:41.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.676+0000 7f3c24400640 1 -- 172.21.15.53:0/2727042468 --> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f3c1c062390 con 0x7f3c040a3a60 2024-04-25T14:40:41.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.799+0000 7f3c1a000640 1 -- 172.21.15.53:0/2727042468 <== mgr.15093 v2:172.21.15.53:6800/802683714 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f3c1c062390 con 0x7f3c040a3a60 2024-04-25T14:40:41.800 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule set for path / 2024-04-25T14:40:41.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.803+0000 7f3c24400640 1 -- 172.21.15.53:0/2727042468 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f3c040a3a60 msgr2=0x7f3c040a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:41.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.803+0000 7f3c24400640 1 --2- 172.21.15.53:0/2727042468 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f3c040a3a60 0x7f3c040a5f20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3c10000c00 tx=0x7f3c10030470 comp rx=0 tx=0).stop 2024-04-25T14:40:41.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.803+0000 7f3c24400640 1 -- 172.21.15.53:0/2727042468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c1c105360 msgr2=0x7f3c1c1009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:41.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.803+0000 7f3c24400640 1 --2- 172.21.15.53:0/2727042468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c1c105360 0x7f3c1c1009e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3c0800e9f0 tx=0x7f3c0800eeb0 comp rx=0 tx=0).stop 2024-04-25T14:40:41.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.803+0000 7f3c24400640 1 -- 172.21.15.53:0/2727042468 shutdown_connections 2024-04-25T14:40:41.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.803+0000 7f3c24400640 1 --2- 172.21.15.53:0/2727042468 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f3c040a3a60 0x7f3c040a5f20 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:41.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.803+0000 7f3c24400640 1 --2- 172.21.15.53:0/2727042468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c1c107c90 0x7f3c1c100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:41.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.803+0000 7f3c24400640 1 --2- 172.21.15.53:0/2727042468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c1c105360 0x7f3c1c1009e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:41.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.803+0000 7f3c24400640 1 --2- 172.21.15.53:0/2727042468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3c1c102a30 0x7f3c1c1023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:41.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.803+0000 7f3c24400640 1 -- 172.21.15.53:0/2727042468 >> 172.21.15.53:0/2727042468 conn(0x7f3c1c0fc440 msgr2=0x7f3c1c103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:41.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.803+0000 7f3c24400640 1 -- 172.21.15.53:0/2727042468 shutdown_connections 2024-04-25T14:40:41.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:41.803+0000 7f3c24400640 1 -- 172.21.15.53:0/2727042468 wait complete. 2024-04-25T14:40:41.827 DEBUG:teuthology.orchestra.run.smithi053:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-25T14:40:42.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.020+0000 7f73b1c00640 1 -- 172.21.15.53:0/3530235110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73ac100f10 msgr2=0x7f73ac101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:42.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.020+0000 7f73b1c00640 1 --2- 172.21.15.53:0/3530235110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73ac100f10 0x7f73ac101350 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f739c009900 tx=0x7f739c02a3f0 comp rx=0 tx=0).stop 2024-04-25T14:40:42.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.020+0000 7f73b1c00640 1 -- 172.21.15.53:0/3530235110 shutdown_connections 2024-04-25T14:40:42.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.020+0000 7f73b1c00640 1 --2- 172.21.15.53:0/3530235110 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73ac102aa0 0x7f73ac1093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:42.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.020+0000 7f73b1c00640 1 --2- 172.21.15.53:0/3530235110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73ac1020e0 0x7f73ac102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:42.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.020+0000 7f73b1c00640 1 --2- 172.21.15.53:0/3530235110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73ac100f10 0x7f73ac101350 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:42.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.020+0000 7f73b1c00640 1 -- 172.21.15.53:0/3530235110 >> 172.21.15.53:0/3530235110 conn(0x7f73ac0fc4d0 msgr2=0x7f73ac0fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:42.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.020+0000 7f73b1c00640 1 -- 172.21.15.53:0/3530235110 shutdown_connections 2024-04-25T14:40:42.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.020+0000 7f73b1c00640 1 -- 172.21.15.53:0/3530235110 wait complete. 2024-04-25T14:40:42.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.021+0000 7f73b1c00640 1 Processor -- start 2024-04-25T14:40:42.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.022+0000 7f73b1c00640 1 -- start start 2024-04-25T14:40:42.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.022+0000 7f73b1c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73ac100f10 0x7f73ac06b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:42.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.022+0000 7f73b1c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73ac1020e0 0x7f73ac19ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:42.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.022+0000 7f73b1c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73ac102aa0 0x7f73ac19b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:42.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.022+0000 7f73b1c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac10ca60 con 0x7f73ac1020e0 2024-04-25T14:40:42.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.022+0000 7f73b1c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac10c8e0 con 0x7f73ac102aa0 2024-04-25T14:40:42.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.022+0000 7f73b1c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac10cbe0 con 0x7f73ac100f10 2024-04-25T14:40:42.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.022+0000 7f73aaa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73ac100f10 0x7f73ac06b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:42.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.022+0000 7f73aaa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73ac100f10 0x7f73ac06b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:51358/0 (socket says 172.21.15.53:51358) 2024-04-25T14:40:42.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.022+0000 7f73aaa00640 1 -- 172.21.15.53:0/819670327 learned_addr learned my addr 172.21.15.53:0/819670327 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:42.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.022+0000 7f73aa000640 1 --2- 172.21.15.53:0/819670327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73ac1020e0 0x7f73ac19ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:42.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.022+0000 7f73ab400640 1 --2- 172.21.15.53:0/819670327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73ac102aa0 0x7f73ac19b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:42.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.023+0000 7f73ab400640 1 -- 172.21.15.53:0/819670327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73ac100f10 msgr2=0x7f73ac06b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:42.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.023+0000 7f73ab400640 1 --2- 172.21.15.53:0/819670327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73ac100f10 0x7f73ac06b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:42.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.023+0000 7f73ab400640 1 -- 172.21.15.53:0/819670327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73ac1020e0 msgr2=0x7f73ac19ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:42.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.023+0000 7f73ab400640 1 --2- 172.21.15.53:0/819670327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73ac1020e0 0x7f73ac19ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:42.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.023+0000 7f73ab400640 1 -- 172.21.15.53:0/819670327 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73ac19bb20 con 0x7f73ac102aa0 2024-04-25T14:40:42.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.023+0000 7f73aa000640 1 --2- 172.21.15.53:0/819670327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73ac1020e0 0x7f73ac19ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:40:42.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.023+0000 7f73ab400640 1 --2- 172.21.15.53:0/819670327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73ac102aa0 0x7f73ac19b260 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f73a4009fd0 tx=0x7f73a400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:42.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.023+0000 7f73a3400640 1 -- 172.21.15.53:0/819670327 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a400ce20 con 0x7f73ac102aa0 2024-04-25T14:40:42.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.023+0000 7f73b1c00640 1 -- 172.21.15.53:0/819670327 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73ac19be10 con 0x7f73ac102aa0 2024-04-25T14:40:42.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.023+0000 7f73a3400640 1 -- 172.21.15.53:0/819670327 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73a4005020 con 0x7f73ac102aa0 2024-04-25T14:40:42.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.023+0000 7f73a3400640 1 -- 172.21.15.53:0/819670327 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a4010690 con 0x7f73ac102aa0 2024-04-25T14:40:42.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.023+0000 7f73b1c00640 1 -- 172.21.15.53:0/819670327 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73ac19c360 con 0x7f73ac102aa0 2024-04-25T14:40:42.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.025+0000 7f73b1c00640 1 -- 172.21.15.53:0/819670327 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73ac19cde0 con 0x7f73ac102aa0 2024-04-25T14:40:42.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.026+0000 7f73a3400640 1 -- 172.21.15.53:0/819670327 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 23) ==== 138458+0+0 (secure 0 0 0) 0x7f73a40054d0 con 0x7f73ac102aa0 2024-04-25T14:40:42.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.028+0000 7f73a3400640 1 --2- 172.21.15.53:0/819670327 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f73900a3ab0 0x7f73900a5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:42.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.028+0000 7f73a3400640 1 -- 172.21.15.53:0/819670327 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8543+0+0 (secure 0 0 0) 0x7f73a40d0840 con 0x7f73ac102aa0 2024-04-25T14:40:42.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.028+0000 7f73aaa00640 1 --2- 172.21.15.53:0/819670327 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f73900a3ab0 0x7f73900a5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:42.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.029+0000 7f73aaa00640 1 --2- 172.21.15.53:0/819670327 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f73900a3ab0 0x7f73900a5f70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f739c000c00 tx=0x7f739c0314a0 comp rx=0 tx=0).ready entity=mgr.15093 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:42.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.034+0000 7f73a3400640 1 -- 172.21.15.53:0/819670327 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f73a409d3a0 con 0x7f73ac102aa0 2024-04-25T14:40:42.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.269+0000 7f73b1c00640 1 -- 172.21.15.53:0/819670327 --> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f73ac1081a0 con 0x7f73900a3ab0 2024-04-25T14:40:42.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.372+0000 7f73a3400640 1 -- 172.21.15.53:0/819670327 <== mgr.15093 v2:172.21.15.53:6800/802683714 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f73ac1081a0 con 0x7f73900a3ab0 2024-04-25T14:40:42.373 INFO:teuthology.orchestra.run.smithi053.stdout:Retention added to path / 2024-04-25T14:40:42.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.375+0000 7f73b1c00640 1 -- 172.21.15.53:0/819670327 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f73900a3ab0 msgr2=0x7f73900a5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:42.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.375+0000 7f73b1c00640 1 --2- 172.21.15.53:0/819670327 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f73900a3ab0 0x7f73900a5f70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f739c000c00 tx=0x7f739c0314a0 comp rx=0 tx=0).stop 2024-04-25T14:40:42.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.375+0000 7f73b1c00640 1 -- 172.21.15.53:0/819670327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73ac102aa0 msgr2=0x7f73ac19b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:42.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.375+0000 7f73b1c00640 1 --2- 172.21.15.53:0/819670327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73ac102aa0 0x7f73ac19b260 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f73a4009fd0 tx=0x7f73a400ef10 comp rx=0 tx=0).stop 2024-04-25T14:40:42.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.375+0000 7f73b1c00640 1 -- 172.21.15.53:0/819670327 shutdown_connections 2024-04-25T14:40:42.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.375+0000 7f73b1c00640 1 --2- 172.21.15.53:0/819670327 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f73900a3ab0 0x7f73900a5f70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:42.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.375+0000 7f73b1c00640 1 --2- 172.21.15.53:0/819670327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73ac102aa0 0x7f73ac19b260 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:42.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.375+0000 7f73b1c00640 1 --2- 172.21.15.53:0/819670327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73ac1020e0 0x7f73ac19ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:42.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.375+0000 7f73b1c00640 1 --2- 172.21.15.53:0/819670327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73ac100f10 0x7f73ac06b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:42.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.375+0000 7f73b1c00640 1 -- 172.21.15.53:0/819670327 >> 172.21.15.53:0/819670327 conn(0x7f73ac0fc4d0 msgr2=0x7f73ac105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:42.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.375+0000 7f73b1c00640 1 -- 172.21.15.53:0/819670327 shutdown_connections 2024-04-25T14:40:42.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.375+0000 7f73b1c00640 1 -- 172.21.15.53:0/819670327 wait complete. 2024-04-25T14:40:42.397 DEBUG:teuthology.orchestra.run.smithi053:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T14:40:42.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:42 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/975417718' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-25T14:40:42.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:42 smithi183 ceph-mon[32584]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T14:40:42.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:42 smithi183 ceph-mon[32584]: mgrmap e23: x(active, since 1.03224s), standbys: y, z 2024-04-25T14:40:42.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:42 smithi183 ceph-mon[32584]: [25/Apr/2024:14:40:41] ENGINE Bus STARTING 2024-04-25T14:40:42.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:42 smithi183 ceph-mon[32584]: [25/Apr/2024:14:40:41] ENGINE Serving on http://172.21.15.53:8765 2024-04-25T14:40:42.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:42 smithi183 ceph-mon[32584]: [25/Apr/2024:14:40:41] ENGINE Serving on https://172.21.15.53:7150 2024-04-25T14:40:42.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:42 smithi183 ceph-mon[32584]: [25/Apr/2024:14:40:41] ENGINE Bus STARTED 2024-04-25T14:40:42.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:42 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:42.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:42 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:42.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:42 smithi183 ceph-mon[32584]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:40:42.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:42 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/975417718' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-25T14:40:42.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:42 smithi150 ceph-mon[32504]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T14:40:42.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:42 smithi150 ceph-mon[32504]: mgrmap e23: x(active, since 1.03224s), standbys: y, z 2024-04-25T14:40:42.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:42 smithi150 ceph-mon[32504]: [25/Apr/2024:14:40:41] ENGINE Bus STARTING 2024-04-25T14:40:42.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:42 smithi150 ceph-mon[32504]: [25/Apr/2024:14:40:41] ENGINE Serving on http://172.21.15.53:8765 2024-04-25T14:40:42.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:42 smithi150 ceph-mon[32504]: [25/Apr/2024:14:40:41] ENGINE Serving on https://172.21.15.53:7150 2024-04-25T14:40:42.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:42 smithi150 ceph-mon[32504]: [25/Apr/2024:14:40:41] ENGINE Bus STARTED 2024-04-25T14:40:42.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:42 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:42.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:42 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:42.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:42 smithi150 ceph-mon[32504]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:40:42.463 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:42 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/975417718' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-25T14:40:42.463 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:42 smithi053 ceph-mon[27773]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T14:40:42.463 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:42 smithi053 ceph-mon[27773]: mgrmap e23: x(active, since 1.03224s), standbys: y, z 2024-04-25T14:40:42.463 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:42 smithi053 ceph-mon[27773]: [25/Apr/2024:14:40:41] ENGINE Bus STARTING 2024-04-25T14:40:42.463 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:42 smithi053 ceph-mon[27773]: [25/Apr/2024:14:40:41] ENGINE Serving on http://172.21.15.53:8765 2024-04-25T14:40:42.463 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:42 smithi053 ceph-mon[27773]: [25/Apr/2024:14:40:41] ENGINE Serving on https://172.21.15.53:7150 2024-04-25T14:40:42.463 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:42 smithi053 ceph-mon[27773]: [25/Apr/2024:14:40:41] ENGINE Bus STARTED 2024-04-25T14:40:42.463 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:42 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:42.463 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:42 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:42.463 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:42 smithi053 ceph-mon[27773]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:40:42.599 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.598+0000 7f7cdaa00640 1 -- 172.21.15.53:0/2078579813 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cd4104010 msgr2=0x7f7cd4104430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:42.600 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.598+0000 7f7cdaa00640 1 --2- 172.21.15.53:0/2078579813 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cd4104010 0x7f7cd4104430 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8009900 tx=0x7f7cc802a400 comp rx=0 tx=0).stop 2024-04-25T14:40:42.600 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.599+0000 7f7cdaa00640 1 -- 172.21.15.53:0/2078579813 shutdown_connections 2024-04-25T14:40:42.600 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.599+0000 7f7cdaa00640 1 --2- 172.21.15.53:0/2078579813 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7cd4105bc0 0x7f7cd410c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:42.600 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.599+0000 7f7cdaa00640 1 --2- 172.21.15.53:0/2078579813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7cd4105200 0x7f7cd4105680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:42.600 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.599+0000 7f7cdaa00640 1 --2- 172.21.15.53:0/2078579813 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cd4104010 0x7f7cd4104430 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:42.600 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.599+0000 7f7cdaa00640 1 -- 172.21.15.53:0/2078579813 >> 172.21.15.53:0/2078579813 conn(0x7f7cd40ff580 msgr2=0x7f7cd41019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:42.600 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.599+0000 7f7cdaa00640 1 -- 172.21.15.53:0/2078579813 shutdown_connections 2024-04-25T14:40:42.600 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.599+0000 7f7cdaa00640 1 -- 172.21.15.53:0/2078579813 wait complete. 2024-04-25T14:40:42.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.600+0000 7f7cdaa00640 1 Processor -- start 2024-04-25T14:40:42.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.600+0000 7f7cdaa00640 1 -- start start 2024-04-25T14:40:42.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.601+0000 7f7cdaa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7cd4104010 0x7f7cd419a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:42.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.601+0000 7f7cdaa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7cd4105200 0x7f7cd419a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:42.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.601+0000 7f7cdaa00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cd4105bc0 0x7f7cd41a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:42.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.601+0000 7f7cdaa00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd410fba0 con 0x7f7cd4105200 2024-04-25T14:40:42.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.601+0000 7f7cdaa00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd410fa20 con 0x7f7cd4105bc0 2024-04-25T14:40:42.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.601+0000 7f7cdaa00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd410fd20 con 0x7f7cd4104010 2024-04-25T14:40:42.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.601+0000 7f7cd9600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7cd4104010 0x7f7cd419a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:42.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.601+0000 7f7cd9600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7cd4104010 0x7f7cd419a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:37732/0 (socket says 172.21.15.53:37732) 2024-04-25T14:40:42.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.601+0000 7f7cd9600640 1 -- 172.21.15.53:0/3766479506 learned_addr learned my addr 172.21.15.53:0/3766479506 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:42.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.601+0000 7f7cd8c00640 1 --2- 172.21.15.53:0/3766479506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7cd4105200 0x7f7cd419a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:42.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.601+0000 7f7cda000640 1 --2- 172.21.15.53:0/3766479506 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cd4105bc0 0x7f7cd41a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:42.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.601+0000 7f7cd9600640 1 -- 172.21.15.53:0/3766479506 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cd4105bc0 msgr2=0x7f7cd41a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:42.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.602+0000 7f7cd9600640 1 --2- 172.21.15.53:0/3766479506 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cd4105bc0 0x7f7cd41a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:42.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.602+0000 7f7cd9600640 1 -- 172.21.15.53:0/3766479506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7cd4105200 msgr2=0x7f7cd419a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:42.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.602+0000 7f7cd9600640 1 --2- 172.21.15.53:0/3766479506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7cd4105200 0x7f7cd419a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:42.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.602+0000 7f7cd9600640 1 -- 172.21.15.53:0/3766479506 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd41a1ea0 con 0x7f7cd4104010 2024-04-25T14:40:42.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.602+0000 7f7cda000640 1 --2- 172.21.15.53:0/3766479506 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cd4105bc0 0x7f7cd41a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:40:42.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.602+0000 7f7cd8c00640 1 --2- 172.21.15.53:0/3766479506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7cd4105200 0x7f7cd419a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:40:42.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.602+0000 7f7cd9600640 1 --2- 172.21.15.53:0/3766479506 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7cd4104010 0x7f7cd419a280 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8031500 tx=0x7f7cc8003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:42.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.602+0000 7f7cd2000640 1 -- 172.21.15.53:0/3766479506 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc803f070 con 0x7f7cd4104010 2024-04-25T14:40:42.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.602+0000 7f7cd2000640 1 -- 172.21.15.53:0/3766479506 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cc8031c50 con 0x7f7cd4104010 2024-04-25T14:40:42.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.602+0000 7f7cdaa00640 1 -- 172.21.15.53:0/3766479506 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cd41a2130 con 0x7f7cd4104010 2024-04-25T14:40:42.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.602+0000 7f7cd2000640 1 -- 172.21.15.53:0/3766479506 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc8035030 con 0x7f7cd4104010 2024-04-25T14:40:42.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.602+0000 7f7cdaa00640 1 -- 172.21.15.53:0/3766479506 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cd41a2670 con 0x7f7cd4104010 2024-04-25T14:40:42.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.603+0000 7f7cdaa00640 1 -- 172.21.15.53:0/3766479506 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cd410b400 con 0x7f7cd4104010 2024-04-25T14:40:42.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.605+0000 7f7cd2000640 1 -- 172.21.15.53:0/3766479506 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) ==== 138766+0+0 (secure 0 0 0) 0x7f7cc8042070 con 0x7f7cd4104010 2024-04-25T14:40:42.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.607+0000 7f7cd2000640 1 --2- 172.21.15.53:0/3766479506 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f7cbc0a3d90 0x7f7cbc0a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:42.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.607+0000 7f7cd2000640 1 -- 172.21.15.53:0/3766479506 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8543+0+0 (secure 0 0 0) 0x7f7cc80f0f90 con 0x7f7cd4104010 2024-04-25T14:40:42.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.607+0000 7f7cd8c00640 1 --2- 172.21.15.53:0/3766479506 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f7cbc0a3d90 0x7f7cbc0a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:42.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.608+0000 7f7cd8c00640 1 --2- 172.21.15.53:0/3766479506 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f7cbc0a3d90 0x7f7cbc0a6250 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7cd419b790 tx=0x7f7cc4005f90 comp rx=0 tx=0).ready entity=mgr.15093 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:42.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.611+0000 7f7cd2000640 1 -- 172.21.15.53:0/3766479506 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7cc80bcf70 con 0x7f7cd4104010 2024-04-25T14:40:42.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.844+0000 7f7cdaa00640 1 -- 172.21.15.53:0/3766479506 --> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f7cd4062390 con 0x7f7cbc0a3d90 2024-04-25T14:40:42.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.857+0000 7f7cd2000640 1 -- 172.21.15.53:0/3766479506 <== mgr.15093 v2:172.21.15.53:6800/802683714 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f7cd4062390 con 0x7f7cbc0a3d90 2024-04-25T14:40:42.858 INFO:teuthology.orchestra.run.smithi053.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T14:40:41", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-25T14:40:42.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.860+0000 7f7cdaa00640 1 -- 172.21.15.53:0/3766479506 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f7cbc0a3d90 msgr2=0x7f7cbc0a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:42.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.860+0000 7f7cdaa00640 1 --2- 172.21.15.53:0/3766479506 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f7cbc0a3d90 0x7f7cbc0a6250 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7cd419b790 tx=0x7f7cc4005f90 comp rx=0 tx=0).stop 2024-04-25T14:40:42.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.860+0000 7f7cdaa00640 1 -- 172.21.15.53:0/3766479506 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7cd4104010 msgr2=0x7f7cd419a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:42.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.860+0000 7f7cdaa00640 1 --2- 172.21.15.53:0/3766479506 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7cd4104010 0x7f7cd419a280 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8031500 tx=0x7f7cc8003c10 comp rx=0 tx=0).stop 2024-04-25T14:40:42.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.860+0000 7f7cdaa00640 1 -- 172.21.15.53:0/3766479506 shutdown_connections 2024-04-25T14:40:42.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.860+0000 7f7cdaa00640 1 --2- 172.21.15.53:0/3766479506 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f7cbc0a3d90 0x7f7cbc0a6250 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:42.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.860+0000 7f7cdaa00640 1 --2- 172.21.15.53:0/3766479506 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cd4105bc0 0x7f7cd41a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:42.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.860+0000 7f7cdaa00640 1 --2- 172.21.15.53:0/3766479506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7cd4105200 0x7f7cd419a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:42.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.860+0000 7f7cdaa00640 1 --2- 172.21.15.53:0/3766479506 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7cd4104010 0x7f7cd419a280 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:42.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.860+0000 7f7cdaa00640 1 -- 172.21.15.53:0/3766479506 >> 172.21.15.53:0/3766479506 conn(0x7f7cd40ff580 msgr2=0x7f7cd41085d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:42.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.860+0000 7f7cdaa00640 1 -- 172.21.15.53:0/3766479506 shutdown_connections 2024-04-25T14:40:42.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:42.860+0000 7f7cdaa00640 1 -- 172.21.15.53:0/3766479506 wait complete. 2024-04-25T14:40:42.880 DEBUG:teuthology.orchestra.run.smithi053:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T14:40:43.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.076+0000 7f92f1a00640 1 -- 172.21.15.53:0/764360898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92ec105180 msgr2=0x7f92ec107570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:43.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.076+0000 7f92f1a00640 1 --2- 172.21.15.53:0/764360898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92ec105180 0x7f92ec107570 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f92d8009900 tx=0x7f92d802a400 comp rx=0 tx=0).stop 2024-04-25T14:40:43.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.076+0000 7f92f1a00640 1 -- 172.21.15.53:0/764360898 shutdown_connections 2024-04-25T14:40:43.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.076+0000 7f92f1a00640 1 --2- 172.21.15.53:0/764360898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92ec107ab0 0x7f92ec109f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:43.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.076+0000 7f92f1a00640 1 --2- 172.21.15.53:0/764360898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92ec105180 0x7f92ec107570 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:43.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.076+0000 7f92f1a00640 1 --2- 172.21.15.53:0/764360898 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92ec102850 0x7f92ec104c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:43.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.077+0000 7f92f1a00640 1 -- 172.21.15.53:0/764360898 >> 172.21.15.53:0/764360898 conn(0x7f92ec0fc240 msgr2=0x7f92ec0fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:43.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.077+0000 7f92f1a00640 1 -- 172.21.15.53:0/764360898 shutdown_connections 2024-04-25T14:40:43.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.077+0000 7f92f1a00640 1 -- 172.21.15.53:0/764360898 wait complete. 2024-04-25T14:40:43.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.078+0000 7f92f1a00640 1 Processor -- start 2024-04-25T14:40:43.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.078+0000 7f92f1a00640 1 -- start start 2024-04-25T14:40:43.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.079+0000 7f92f1a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92ec102850 0x7f92ec1021c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:43.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.079+0000 7f92f1a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92ec105180 0x7f92ec100800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:43.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.079+0000 7f92f1a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92ec107ab0 0x7f92ec100d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:43.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.079+0000 7f92f1a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f92ec10d940 con 0x7f92ec102850 2024-04-25T14:40:43.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.079+0000 7f92f1a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f92ec10d7c0 con 0x7f92ec107ab0 2024-04-25T14:40:43.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.079+0000 7f92f1a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f92ec10dac0 con 0x7f92ec105180 2024-04-25T14:40:43.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.079+0000 7f92eb400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92ec105180 0x7f92ec100800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:43.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.079+0000 7f92ebe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92ec102850 0x7f92ec1021c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:43.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.079+0000 7f92eb400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92ec105180 0x7f92ec100800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:37752/0 (socket says 172.21.15.53:37752) 2024-04-25T14:40:43.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.079+0000 7f92eb400640 1 -- 172.21.15.53:0/1827443315 learned_addr learned my addr 172.21.15.53:0/1827443315 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:43.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.079+0000 7f92f1000640 1 --2- 172.21.15.53:0/1827443315 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92ec107ab0 0x7f92ec100d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:43.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.079+0000 7f92eb400640 1 -- 172.21.15.53:0/1827443315 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92ec107ab0 msgr2=0x7f92ec100d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:43.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.079+0000 7f92eb400640 1 --2- 172.21.15.53:0/1827443315 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92ec107ab0 0x7f92ec100d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:43.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.079+0000 7f92eb400640 1 -- 172.21.15.53:0/1827443315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92ec102850 msgr2=0x7f92ec1021c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:43.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.079+0000 7f92eb400640 1 --2- 172.21.15.53:0/1827443315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92ec102850 0x7f92ec1021c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:43.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.079+0000 7f92eb400640 1 -- 172.21.15.53:0/1827443315 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92ec1015d0 con 0x7f92ec105180 2024-04-25T14:40:43.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.080+0000 7f92eb400640 1 --2- 172.21.15.53:0/1827443315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92ec105180 0x7f92ec100800 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f92d8031500 tx=0x7f92d8003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:43.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.080+0000 7f92e8c00640 1 -- 172.21.15.53:0/1827443315 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d803f070 con 0x7f92ec105180 2024-04-25T14:40:43.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.080+0000 7f92e8c00640 1 -- 172.21.15.53:0/1827443315 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92d8031c50 con 0x7f92ec105180 2024-04-25T14:40:43.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.080+0000 7f92e8c00640 1 -- 172.21.15.53:0/1827443315 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d8035030 con 0x7f92ec105180 2024-04-25T14:40:43.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.080+0000 7f92f1a00640 1 -- 172.21.15.53:0/1827443315 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92ec1a20d0 con 0x7f92ec105180 2024-04-25T14:40:43.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.080+0000 7f92f1a00640 1 -- 172.21.15.53:0/1827443315 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92ec1a2620 con 0x7f92ec105180 2024-04-25T14:40:43.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.081+0000 7f92f1a00640 1 -- 172.21.15.53:0/1827443315 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92ec0fdd30 con 0x7f92ec105180 2024-04-25T14:40:43.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.083+0000 7f92e8c00640 1 -- 172.21.15.53:0/1827443315 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) ==== 138766+0+0 (secure 0 0 0) 0x7f92d8042070 con 0x7f92ec105180 2024-04-25T14:40:43.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.085+0000 7f92e8c00640 1 --2- 172.21.15.53:0/1827443315 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f92d00a3d10 0x7f92d00a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:43.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.085+0000 7f92e8c00640 1 -- 172.21.15.53:0/1827443315 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8543+0+0 (secure 0 0 0) 0x7f92d80f1690 con 0x7f92ec105180 2024-04-25T14:40:43.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.085+0000 7f92ebe00640 1 --2- 172.21.15.53:0/1827443315 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f92d00a3d10 0x7f92d00a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:43.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.086+0000 7f92ebe00640 1 --2- 172.21.15.53:0/1827443315 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f92d00a3d10 0x7f92d00a61d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f92dc00bbe0 tx=0x7f92dc00b390 comp rx=0 tx=0).ready entity=mgr.15093 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:43.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.087+0000 7f92e8c00640 1 -- 172.21.15.53:0/1827443315 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f92d80bd670 con 0x7f92ec105180 2024-04-25T14:40:43.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.315+0000 7f92f1a00640 1 -- 172.21.15.53:0/1827443315 --> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f92ec062390 con 0x7f92d00a3d10 2024-04-25T14:40:43.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.324+0000 7f92e8c00640 1 -- 172.21.15.53:0/1827443315 <== mgr.15093 v2:172.21.15.53:6800/802683714 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f92ec062390 con 0x7f92d00a3d10 2024-04-25T14:40:43.325 INFO:teuthology.orchestra.run.smithi053.stdout:/ 1m 6m3h 2024-04-25T14:40:43.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.327+0000 7f92f1a00640 1 -- 172.21.15.53:0/1827443315 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f92d00a3d10 msgr2=0x7f92d00a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:43.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.327+0000 7f92f1a00640 1 --2- 172.21.15.53:0/1827443315 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f92d00a3d10 0x7f92d00a61d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f92dc00bbe0 tx=0x7f92dc00b390 comp rx=0 tx=0).stop 2024-04-25T14:40:43.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.327+0000 7f92f1a00640 1 -- 172.21.15.53:0/1827443315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92ec105180 msgr2=0x7f92ec100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:43.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.327+0000 7f92f1a00640 1 --2- 172.21.15.53:0/1827443315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92ec105180 0x7f92ec100800 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f92d8031500 tx=0x7f92d8003c10 comp rx=0 tx=0).stop 2024-04-25T14:40:43.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.327+0000 7f92f1a00640 1 -- 172.21.15.53:0/1827443315 shutdown_connections 2024-04-25T14:40:43.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.327+0000 7f92f1a00640 1 --2- 172.21.15.53:0/1827443315 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f92d00a3d10 0x7f92d00a61d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:43.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.327+0000 7f92f1a00640 1 --2- 172.21.15.53:0/1827443315 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92ec107ab0 0x7f92ec100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:43.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.327+0000 7f92f1a00640 1 --2- 172.21.15.53:0/1827443315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92ec105180 0x7f92ec100800 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:43.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.327+0000 7f92f1a00640 1 --2- 172.21.15.53:0/1827443315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92ec102850 0x7f92ec1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:43.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.327+0000 7f92f1a00640 1 -- 172.21.15.53:0/1827443315 >> 172.21.15.53:0/1827443315 conn(0x7f92ec0fc240 msgr2=0x7f92ec1032d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:43.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.327+0000 7f92f1a00640 1 -- 172.21.15.53:0/1827443315 shutdown_connections 2024-04-25T14:40:43.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:43.327+0000 7f92f1a00640 1 -- 172.21.15.53:0/1827443315 wait complete. 2024-04-25T14:40:43.350 DEBUG:teuthology.orchestra.run.smithi053:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-25T14:40:43.416 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-25T14:40:43.426 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-25T14:40:43.436 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi053.front.sepia.ceph.com 2024-04-25T14:40:43.436 DEBUG:teuthology.orchestra.run.smithi053:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-25T14:40:43.440 INFO:teuthology.run_tasks:Running task quiescer... 2024-04-25T14:40:43.452 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T14:40:43.452 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T14:40:43.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:43 smithi183 ceph-mon[32584]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:40:43.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:43 smithi183 ceph-mon[32584]: from='client.25277 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:40:43.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:43 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:43.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:43 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:43.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:43 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:43.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:43 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:43.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:43 smithi183 ceph-mon[32584]: mgrmap e24: x(active, since 2s), standbys: y, z 2024-04-25T14:40:43.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:43 smithi150 ceph-mon[32504]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:40:43.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:43 smithi150 ceph-mon[32504]: from='client.25277 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:40:43.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:43 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:43.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:43 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:43.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:43 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:43.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:43 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:43.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:43 smithi150 ceph-mon[32504]: mgrmap e24: x(active, since 2s), standbys: y, z 2024-04-25T14:40:43.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:43 smithi053 ceph-mon[27773]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:40:43.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:43 smithi053 ceph-mon[27773]: from='client.25277 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:40:43.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:43 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:43.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:43 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:43.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:43 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:43.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:43 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:43.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:43 smithi053 ceph-mon[27773]: mgrmap e24: x(active, since 2s), standbys: y, z 2024-04-25T14:40:43.882 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:40:44.784 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:44 smithi053 ceph-mon[27773]: from='client.25315 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:40:44.785 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:44 smithi053 ceph-mon[27773]: from='client.25321 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:40:44.785 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:44 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:44.785 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:44 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:44.785 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:44 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:44.785 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:44 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.785 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:44 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.785 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:44 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:44.785 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:44 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.785 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:44 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.785 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:44 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.785 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:44 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.785 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:44 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.786 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:44 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.786 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:44 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.786 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:44 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.786 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:44 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.786 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:44 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.786 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:44 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.786 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:44 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.786 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:44 smithi053 ceph-mon[27773]: Adjusting osd_memory_target on smithi150 to 176.2M 2024-04-25T14:40:44.786 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:44 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.786 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:44 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.786 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:44 smithi053 ceph-mon[27773]: Unable to set osd_memory_target on smithi150 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-25T14:40:44.786 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:44 smithi053 ceph-mon[27773]: Adjusting osd_memory_target on smithi183 to 176.2M 2024-04-25T14:40:44.786 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:44 smithi053 ceph-mon[27773]: Unable to set osd_memory_target on smithi183 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T14:40:44.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:44 smithi183 ceph-mon[32584]: from='client.25315 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:40:44.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:44 smithi183 ceph-mon[32584]: from='client.25321 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:40:44.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:44 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:44.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:44 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:44.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:44 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:44.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:44 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:44 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:44 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:44.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:44 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:44 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:44 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:44 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:44 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:44 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:44 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:44 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:44 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:44 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:44 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:44 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:44 smithi183 ceph-mon[32584]: Adjusting osd_memory_target on smithi150 to 176.2M 2024-04-25T14:40:44.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:44 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:44 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:44 smithi183 ceph-mon[32584]: Unable to set osd_memory_target on smithi150 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-25T14:40:44.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:44 smithi183 ceph-mon[32584]: Adjusting osd_memory_target on smithi183 to 176.2M 2024-04-25T14:40:44.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:44 smithi183 ceph-mon[32584]: Unable to set osd_memory_target on smithi183 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T14:40:44.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:44 smithi150 ceph-mon[32504]: from='client.25315 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:40:44.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:44 smithi150 ceph-mon[32504]: from='client.25321 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:40:44.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:44 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:44.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:44 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:44.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:44 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:44.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:44 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:44 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:44 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:44.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:44 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:44 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:44 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:44 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:44 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:44 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:44 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:44 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:44 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:44 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:44 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:44 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:44 smithi150 ceph-mon[32504]: Adjusting osd_memory_target on smithi150 to 176.2M 2024-04-25T14:40:44.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:44 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:44 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:44.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:44 smithi150 ceph-mon[32504]: Unable to set osd_memory_target on smithi150 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-25T14:40:44.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:44 smithi150 ceph-mon[32504]: Adjusting osd_memory_target on smithi183 to 176.2M 2024-04-25T14:40:44.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:44 smithi150 ceph-mon[32504]: Unable to set osd_memory_target on smithi183 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T14:40:45.600 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:45 smithi053 ceph-mon[27773]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:40:45.600 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:45 smithi053 ceph-mon[27773]: mgrmap e25: x(active, since 4s), standbys: y, z 2024-04-25T14:40:45.600 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:45 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:45.600 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:45 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:45.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:45 smithi183 ceph-mon[32584]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:40:45.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:45 smithi183 ceph-mon[32584]: mgrmap e25: x(active, since 4s), standbys: y, z 2024-04-25T14:40:45.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:45 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:45.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:45 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:45.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:45 smithi150 ceph-mon[32504]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:40:45.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:45 smithi150 ceph-mon[32504]: mgrmap e25: x(active, since 4s), standbys: y, z 2024-04-25T14:40:45.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:45 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:45.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:45 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:46.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.054+0000 7fa41f200700 1 -- 172.21.15.53:0/3349448107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa418075590 msgr2=0x7fa41810ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:46.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.054+0000 7fa41f200700 1 --2- 172.21.15.53:0/3349448107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa418075590 0x7fa41810ef00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa404009990 tx=0x7fa404009ca0 comp rx=0 tx=0).stop 2024-04-25T14:40:46.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.054+0000 7fa41f200700 1 -- 172.21.15.53:0/3349448107 shutdown_connections 2024-04-25T14:40:46.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.054+0000 7fa41f200700 1 --2- 172.21.15.53:0/3349448107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa41810f440 0x7fa418111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:46.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.054+0000 7fa41f200700 1 --2- 172.21.15.53:0/3349448107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa418074c70 0x7fa418075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:46.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.054+0000 7fa41f200700 1 --2- 172.21.15.53:0/3349448107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa418075590 0x7fa41810ef00 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:46.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.054+0000 7fa41f200700 1 -- 172.21.15.53:0/3349448107 >> 172.21.15.53:0/3349448107 conn(0x7fa418079bf0 msgr2=0x7fa41807a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:46.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.054+0000 7fa41f200700 1 -- 172.21.15.53:0/3349448107 shutdown_connections 2024-04-25T14:40:46.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.054+0000 7fa41f200700 1 -- 172.21.15.53:0/3349448107 wait complete. 2024-04-25T14:40:46.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.055+0000 7fa41f200700 1 Processor -- start 2024-04-25T14:40:46.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.055+0000 7fa41f200700 1 -- start start 2024-04-25T14:40:46.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.055+0000 7fa41f200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa418074c70 0x7fa41819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:46.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.056+0000 7fa41f200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa418075590 0x7fa41819b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:46.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.056+0000 7fa41f200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa41810f440 0x7fa41819fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:46.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.056+0000 7fa41f200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa418114ed0 con 0x7fa41810f440 2024-04-25T14:40:46.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.056+0000 7fa41f200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa418114d50 con 0x7fa418075590 2024-04-25T14:40:46.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.056+0000 7fa41f200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa418115050 con 0x7fa418074c70 2024-04-25T14:40:46.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.056+0000 7fa41d200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa41810f440 0x7fa41819fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:46.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.056+0000 7fa417e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa418074c70 0x7fa41819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:46.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.056+0000 7fa417400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa418075590 0x7fa41819b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:46.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.056+0000 7fa417e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa418074c70 0x7fa41819b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:37778/0 (socket says 172.21.15.53:37778) 2024-04-25T14:40:46.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.056+0000 7fa417e00700 1 -- 172.21.15.53:0/638949353 learned_addr learned my addr 172.21.15.53:0/638949353 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:46.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.056+0000 7fa417400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa418075590 0x7fa41819b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.53:50030/0 (socket says 172.21.15.53:50030) 2024-04-25T14:40:46.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.057+0000 7fa417400700 1 -- 172.21.15.53:0/638949353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa418074c70 msgr2=0x7fa41819b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:46.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.057+0000 7fa417400700 1 --2- 172.21.15.53:0/638949353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa418074c70 0x7fa41819b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:46.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.057+0000 7fa417400700 1 -- 172.21.15.53:0/638949353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa41810f440 msgr2=0x7fa41819fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:46.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.057+0000 7fa417400700 1 --2- 172.21.15.53:0/638949353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa41810f440 0x7fa41819fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:46.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.057+0000 7fa417400700 1 -- 172.21.15.53:0/638949353 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4181a0340 con 0x7fa418075590 2024-04-25T14:40:46.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.057+0000 7fa417400700 1 --2- 172.21.15.53:0/638949353 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa418075590 0x7fa41819b8c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa4040098e0 tx=0x7fa4040057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:46.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.057+0000 7fa414c00700 1 -- 172.21.15.53:0/638949353 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404027070 con 0x7fa418075590 2024-04-25T14:40:46.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.057+0000 7fa41f200700 1 -- 172.21.15.53:0/638949353 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4181a05d0 con 0x7fa418075590 2024-04-25T14:40:46.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.057+0000 7fa414c00700 1 -- 172.21.15.53:0/638949353 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa404022470 con 0x7fa418075590 2024-04-25T14:40:46.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.057+0000 7fa414c00700 1 -- 172.21.15.53:0/638949353 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404017650 con 0x7fa418075590 2024-04-25T14:40:46.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.058+0000 7fa41f200700 1 -- 172.21.15.53:0/638949353 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4181a7eb0 con 0x7fa418075590 2024-04-25T14:40:46.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.059+0000 7fa41f200700 1 -- 172.21.15.53:0/638949353 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa418076390 con 0x7fa418075590 2024-04-25T14:40:46.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.061+0000 7fa414c00700 1 -- 172.21.15.53:0/638949353 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7fa40402a070 con 0x7fa418075590 2024-04-25T14:40:46.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.062+0000 7fa414c00700 1 --2- 172.21.15.53:0/638949353 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fa3fc0a3e40 0x7fa3fc0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:46.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.063+0000 7fa414c00700 1 -- 172.21.15.53:0/638949353 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8543+0+0 (secure 0 0 0) 0x7fa4040d1780 con 0x7fa418075590 2024-04-25T14:40:46.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.063+0000 7fa417e00700 1 --2- 172.21.15.53:0/638949353 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fa3fc0a3e40 0x7fa3fc0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:46.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.063+0000 7fa417e00700 1 --2- 172.21.15.53:0/638949353 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fa3fc0a3e40 0x7fa3fc0a6300 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa408009d20 tx=0x7fa408009260 comp rx=0 tx=0).ready entity=mgr.15093 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:46.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.067+0000 7fa414c00700 1 -- 172.21.15.53:0/638949353 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa40405eff0 con 0x7fa418075590 2024-04-25T14:40:46.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.305+0000 7fa41f200700 1 -- 172.21.15.53:0/638949353 --> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa41819c690 con 0x7fa3fc0a3e40 2024-04-25T14:40:46.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.315+0000 7fa414c00700 1 -- 172.21.15.53:0/638949353 <== mgr.15093 v2:172.21.15.53:6800/802683714 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257781 (secure 0 0 0) 0x7fa41819c690 con 0x7fa3fc0a3e40 2024-04-25T14:40:46.321 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:40:46.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.320+0000 7fa41f200700 1 -- 172.21.15.53:0/638949353 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fa3fc0a3e40 msgr2=0x7fa3fc0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:46.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.320+0000 7fa41f200700 1 --2- 172.21.15.53:0/638949353 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fa3fc0a3e40 0x7fa3fc0a6300 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa408009d20 tx=0x7fa408009260 comp rx=0 tx=0).stop 2024-04-25T14:40:46.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.320+0000 7fa41f200700 1 -- 172.21.15.53:0/638949353 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa418075590 msgr2=0x7fa41819b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:46.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.320+0000 7fa41f200700 1 --2- 172.21.15.53:0/638949353 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa418075590 0x7fa41819b8c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa4040098e0 tx=0x7fa4040057c0 comp rx=0 tx=0).stop 2024-04-25T14:40:46.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.320+0000 7fa41f200700 1 -- 172.21.15.53:0/638949353 shutdown_connections 2024-04-25T14:40:46.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.320+0000 7fa41f200700 1 --2- 172.21.15.53:0/638949353 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fa3fc0a3e40 0x7fa3fc0a6300 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:46.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.320+0000 7fa41f200700 1 --2- 172.21.15.53:0/638949353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa41810f440 0x7fa41819fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:46.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.320+0000 7fa41f200700 1 --2- 172.21.15.53:0/638949353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa418074c70 0x7fa41819b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:46.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.320+0000 7fa41f200700 1 --2- 172.21.15.53:0/638949353 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa418075590 0x7fa41819b8c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:46.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.320+0000 7fa41f200700 1 -- 172.21.15.53:0/638949353 >> 172.21.15.53:0/638949353 conn(0x7fa418079bf0 msgr2=0x7fa4180ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:46.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.321+0000 7fa41f200700 1 -- 172.21.15.53:0/638949353 shutdown_connections 2024-04-25T14:40:46.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:46.321+0000 7fa41f200700 1 -- 172.21.15.53:0/638949353 wait complete. 2024-04-25T14:40:46.324 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-04-25T14:40:47.236 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-04-25T14:40:46.197615+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1189794,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1137,"num_read_kb":733,"num_write":477,"num_write_kb":4382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1181771,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":870,"ondisk_log_size":870,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381604,"kb_used_data":15776,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124560220,"statfs":{"total":1151940427776,"available":1151549665280,"internally_reserved":0,"allocated":16154624,"data_stored":9501845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19137,"internal_metadata":374453567},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.970860"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.651108+0000","last_change":"2024-04-25T14:38:10.435535+0000","last_active":"2024-04-25T14:40:40.651108+0000","last_peered":"2024-04-25T14:40:40.651108+0000","last_clean":"2024-04-25T14:40:40.651108+0000","last_became_active":"2024-04-25T14:38:10.435205+0000","last_became_peered":"2024-04-25T14:38:10.435205+0000","last_unstale":"2024-04-25T14:40:40.651108+0000","last_undegraded":"2024-04-25T14:40:40.651108+0000","last_fullsized":"2024-04-25T14:40:40.651108+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:12.301771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.991215+0000","last_change":"2024-04-25T14:38:10.428216+0000","last_active":"2024-04-25T14:40:40.991215+0000","last_peered":"2024-04-25T14:40:40.991215+0000","last_clean":"2024-04-25T14:40:40.991215+0000","last_became_active":"2024-04-25T14:38:10.427419+0000","last_became_peered":"2024-04-25T14:38:10.427419+0000","last_unstale":"2024-04-25T14:40:40.991215+0000","last_undegraded":"2024-04-25T14:40:40.991215+0000","last_fullsized":"2024-04-25T14:40:40.991215+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:42.580424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.991464+0000","last_change":"2024-04-25T14:38:10.427827+0000","last_active":"2024-04-25T14:40:40.991464+0000","last_peered":"2024-04-25T14:40:40.991464+0000","last_clean":"2024-04-25T14:40:40.991464+0000","last_became_active":"2024-04-25T14:38:10.427435+0000","last_became_peered":"2024-04-25T14:38:10.427435+0000","last_unstale":"2024-04-25T14:40:40.991464+0000","last_undegraded":"2024-04-25T14:40:40.991464+0000","last_fullsized":"2024-04-25T14:40:40.991464+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:15.539399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.481932+0000","last_change":"2024-04-25T14:38:10.423751+0000","last_active":"2024-04-25T14:40:40.481932+0000","last_peered":"2024-04-25T14:40:40.481932+0000","last_clean":"2024-04-25T14:40:40.481932+0000","last_became_active":"2024-04-25T14:38:10.423409+0000","last_became_peered":"2024-04-25T14:38:10.423409+0000","last_unstale":"2024-04-25T14:40:40.481932+0000","last_undegraded":"2024-04-25T14:40:40.481932+0000","last_fullsized":"2024-04-25T14:40:40.481932+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:45.403788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.304337+0000","last_change":"2024-04-25T14:38:10.430000+0000","last_active":"2024-04-25T14:40:40.304337+0000","last_peered":"2024-04-25T14:40:40.304337+0000","last_clean":"2024-04-25T14:40:40.304337+0000","last_became_active":"2024-04-25T14:38:10.429684+0000","last_became_peered":"2024-04-25T14:38:10.429684+0000","last_unstale":"2024-04-25T14:40:40.304337+0000","last_undegraded":"2024-04-25T14:40:40.304337+0000","last_fullsized":"2024-04-25T14:40:40.304337+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:18.421970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.179542+0000","last_change":"2024-04-25T14:39:15.984180+0000","last_active":"2024-04-25T14:40:40.179542+0000","last_peered":"2024-04-25T14:40:40.179542+0000","last_clean":"2024-04-25T14:40:40.179542+0000","last_became_active":"2024-04-25T14:39:15.983821+0000","last_became_peered":"2024-04-25T14:39:15.983821+0000","last_unstale":"2024-04-25T14:40:40.179542+0000","last_undegraded":"2024-04-25T14:40:40.179542+0000","last_fullsized":"2024-04-25T14:40:40.179542+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:54:47.617713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.482156+0000","last_change":"2024-04-25T14:38:10.420022+0000","last_active":"2024-04-25T14:40:40.482156+0000","last_peered":"2024-04-25T14:40:40.482156+0000","last_clean":"2024-04-25T14:40:40.482156+0000","last_became_active":"2024-04-25T14:38:10.419673+0000","last_became_peered":"2024-04-25T14:38:10.419673+0000","last_unstale":"2024-04-25T14:40:40.482156+0000","last_undegraded":"2024-04-25T14:40:40.482156+0000","last_fullsized":"2024-04-25T14:40:40.482156+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:09.177814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.281442+0000","last_change":"2024-04-25T14:40:16.499255+0000","last_active":"2024-04-25T14:40:40.281442+0000","last_peered":"2024-04-25T14:40:40.281442+0000","last_clean":"2024-04-25T14:40:40.281442+0000","last_became_active":"2024-04-25T14:40:16.497912+0000","last_became_peered":"2024-04-25T14:40:16.497912+0000","last_unstale":"2024-04-25T14:40:40.281442+0000","last_undegraded":"2024-04-25T14:40:40.281442+0000","last_fullsized":"2024-04-25T14:40:40.281442+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:10.595053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,10],"acting":[9,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.224740+0000","last_change":"2024-04-25T14:38:10.424086+0000","last_active":"2024-04-25T14:40:40.224740+0000","last_peered":"2024-04-25T14:40:40.224740+0000","last_clean":"2024-04-25T14:40:40.224740+0000","last_became_active":"2024-04-25T14:38:10.423769+0000","last_became_peered":"2024-04-25T14:38:10.423769+0000","last_unstale":"2024-04-25T14:40:40.224740+0000","last_undegraded":"2024-04-25T14:40:40.224740+0000","last_fullsized":"2024-04-25T14:40:40.224740+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:19:22.736053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.180126+0000","last_change":"2024-04-25T14:38:10.427049+0000","last_active":"2024-04-25T14:40:40.180126+0000","last_peered":"2024-04-25T14:40:40.180126+0000","last_clean":"2024-04-25T14:40:40.180126+0000","last_became_active":"2024-04-25T14:38:10.426729+0000","last_became_peered":"2024-04-25T14:38:10.426729+0000","last_unstale":"2024-04-25T14:40:40.180126+0000","last_undegraded":"2024-04-25T14:40:40.180126+0000","last_fullsized":"2024-04-25T14:40:40.180126+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:00.849650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.482804+0000","last_change":"2024-04-25T14:38:10.424976+0000","last_active":"2024-04-25T14:40:40.482804+0000","last_peered":"2024-04-25T14:40:40.482804+0000","last_clean":"2024-04-25T14:40:40.482804+0000","last_became_active":"2024-04-25T14:38:10.424585+0000","last_became_peered":"2024-04-25T14:38:10.424585+0000","last_unstale":"2024-04-25T14:40:40.482804+0000","last_undegraded":"2024-04-25T14:40:40.482804+0000","last_fullsized":"2024-04-25T14:40:40.482804+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:54.639471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.481912+0000","last_change":"2024-04-25T14:38:10.418022+0000","last_active":"2024-04-25T14:40:40.481912+0000","last_peered":"2024-04-25T14:40:40.481912+0000","last_clean":"2024-04-25T14:40:40.481912+0000","last_became_active":"2024-04-25T14:38:10.417735+0000","last_became_peered":"2024-04-25T14:38:10.417735+0000","last_unstale":"2024-04-25T14:40:40.481912+0000","last_undegraded":"2024-04-25T14:40:40.481912+0000","last_fullsized":"2024-04-25T14:40:40.481912+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:48.785521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.677401+0000","last_change":"2024-04-25T14:38:10.423464+0000","last_active":"2024-04-25T14:40:40.677401+0000","last_peered":"2024-04-25T14:40:40.677401+0000","last_clean":"2024-04-25T14:40:40.677401+0000","last_became_active":"2024-04-25T14:38:10.423105+0000","last_became_peered":"2024-04-25T14:38:10.423105+0000","last_unstale":"2024-04-25T14:40:40.677401+0000","last_undegraded":"2024-04-25T14:40:40.677401+0000","last_fullsized":"2024-04-25T14:40:40.677401+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:26.568198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.991106+0000","last_change":"2024-04-25T14:38:10.426870+0000","last_active":"2024-04-25T14:40:40.991106+0000","last_peered":"2024-04-25T14:40:40.991106+0000","last_clean":"2024-04-25T14:40:40.991106+0000","last_became_active":"2024-04-25T14:38:10.426506+0000","last_became_peered":"2024-04-25T14:38:10.426506+0000","last_unstale":"2024-04-25T14:40:40.991106+0000","last_undegraded":"2024-04-25T14:40:40.991106+0000","last_fullsized":"2024-04-25T14:40:40.991106+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:16:49.539221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.990805+0000","last_change":"2024-04-25T14:38:10.434618+0000","last_active":"2024-04-25T14:40:40.990805+0000","last_peered":"2024-04-25T14:40:40.990805+0000","last_clean":"2024-04-25T14:40:40.990805+0000","last_became_active":"2024-04-25T14:38:10.430661+0000","last_became_peered":"2024-04-25T14:38:10.430661+0000","last_unstale":"2024-04-25T14:40:40.990805+0000","last_undegraded":"2024-04-25T14:40:40.990805+0000","last_fullsized":"2024-04-25T14:40:40.990805+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:44.548788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.224125+0000","last_change":"2024-04-25T14:38:10.429898+0000","last_active":"2024-04-25T14:40:40.224125+0000","last_peered":"2024-04-25T14:40:40.224125+0000","last_clean":"2024-04-25T14:40:40.224125+0000","last_became_active":"2024-04-25T14:38:10.429557+0000","last_became_peered":"2024-04-25T14:38:10.429557+0000","last_unstale":"2024-04-25T14:40:40.224125+0000","last_undegraded":"2024-04-25T14:40:40.224125+0000","last_fullsized":"2024-04-25T14:40:40.224125+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:54.258526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.651158+0000","last_change":"2024-04-25T14:38:10.435134+0000","last_active":"2024-04-25T14:40:40.651158+0000","last_peered":"2024-04-25T14:40:40.651158+0000","last_clean":"2024-04-25T14:40:40.651158+0000","last_became_active":"2024-04-25T14:38:10.434775+0000","last_became_peered":"2024-04-25T14:38:10.434775+0000","last_unstale":"2024-04-25T14:40:40.651158+0000","last_undegraded":"2024-04-25T14:40:40.651158+0000","last_fullsized":"2024-04-25T14:40:40.651158+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:49.827881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.282019+0000","last_change":"2024-04-25T14:38:10.424929+0000","last_active":"2024-04-25T14:40:40.282019+0000","last_peered":"2024-04-25T14:40:40.282019+0000","last_clean":"2024-04-25T14:40:40.282019+0000","last_became_active":"2024-04-25T14:38:10.424399+0000","last_became_peered":"2024-04-25T14:38:10.424399+0000","last_unstale":"2024-04-25T14:40:40.282019+0000","last_undegraded":"2024-04-25T14:40:40.282019+0000","last_fullsized":"2024-04-25T14:40:40.282019+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:08.834176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.224851+0000","last_change":"2024-04-25T14:38:10.444004+0000","last_active":"2024-04-25T14:40:40.224851+0000","last_peered":"2024-04-25T14:40:40.224851+0000","last_clean":"2024-04-25T14:40:40.224851+0000","last_became_active":"2024-04-25T14:38:10.443669+0000","last_became_peered":"2024-04-25T14:38:10.443669+0000","last_unstale":"2024-04-25T14:40:40.224851+0000","last_undegraded":"2024-04-25T14:40:40.224851+0000","last_fullsized":"2024-04-25T14:40:40.224851+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:02.302057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.281993+0000","last_change":"2024-04-25T14:38:09.545010+0000","last_active":"2024-04-25T14:40:40.281993+0000","last_peered":"2024-04-25T14:40:40.281993+0000","last_clean":"2024-04-25T14:40:40.281993+0000","last_became_active":"2024-04-25T14:38:09.544622+0000","last_became_peered":"2024-04-25T14:38:09.544622+0000","last_unstale":"2024-04-25T14:40:40.281993+0000","last_undegraded":"2024-04-25T14:40:40.281993+0000","last_fullsized":"2024-04-25T14:40:40.281993+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:51.200242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.281678+0000","last_change":"2024-04-25T14:39:15.993912+0000","last_active":"2024-04-25T14:40:40.281678+0000","last_peered":"2024-04-25T14:40:40.281678+0000","last_clean":"2024-04-25T14:40:40.281678+0000","last_became_active":"2024-04-25T14:39:15.992511+0000","last_became_peered":"2024-04-25T14:39:15.992511+0000","last_unstale":"2024-04-25T14:40:40.281678+0000","last_undegraded":"2024-04-25T14:40:40.281678+0000","last_fullsized":"2024-04-25T14:40:40.281678+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:12:57.639396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,6],"acting":[9,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.184989+0000","last_change":"2024-04-25T14:38:09.555825+0000","last_active":"2024-04-25T14:40:40.184989+0000","last_peered":"2024-04-25T14:40:40.184989+0000","last_clean":"2024-04-25T14:40:40.184989+0000","last_became_active":"2024-04-25T14:38:09.555436+0000","last_became_peered":"2024-04-25T14:38:09.555436+0000","last_unstale":"2024-04-25T14:40:40.184989+0000","last_undegraded":"2024-04-25T14:40:40.184989+0000","last_fullsized":"2024-04-25T14:40:40.184989+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:45.097583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.991388+0000","last_change":"2024-04-25T14:38:10.432254+0000","last_active":"2024-04-25T14:40:40.991388+0000","last_peered":"2024-04-25T14:40:40.991388+0000","last_clean":"2024-04-25T14:40:40.991388+0000","last_became_active":"2024-04-25T14:38:10.431835+0000","last_became_peered":"2024-04-25T14:38:10.431835+0000","last_unstale":"2024-04-25T14:40:40.991388+0000","last_undegraded":"2024-04-25T14:40:40.991388+0000","last_fullsized":"2024-04-25T14:40:40.991388+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:23.089447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.304545+0000","last_change":"2024-04-25T14:38:09.550254+0000","last_active":"2024-04-25T14:40:40.304545+0000","last_peered":"2024-04-25T14:40:40.304545+0000","last_clean":"2024-04-25T14:40:40.304545+0000","last_became_active":"2024-04-25T14:38:09.549958+0000","last_became_peered":"2024-04-25T14:38:09.549958+0000","last_unstale":"2024-04-25T14:40:40.304545+0000","last_undegraded":"2024-04-25T14:40:40.304545+0000","last_fullsized":"2024-04-25T14:40:40.304545+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:56.161427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.224323+0000","last_change":"2024-04-25T14:38:10.425648+0000","last_active":"2024-04-25T14:40:40.224323+0000","last_peered":"2024-04-25T14:40:40.224323+0000","last_clean":"2024-04-25T14:40:40.224323+0000","last_became_active":"2024-04-25T14:38:10.425009+0000","last_became_peered":"2024-04-25T14:38:10.425009+0000","last_unstale":"2024-04-25T14:40:40.224323+0000","last_undegraded":"2024-04-25T14:40:40.224323+0000","last_fullsized":"2024-04-25T14:40:40.224323+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:46.696610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.304408+0000","last_change":"2024-04-25T14:38:09.544481+0000","last_active":"2024-04-25T14:40:40.304408+0000","last_peered":"2024-04-25T14:40:40.304408+0000","last_clean":"2024-04-25T14:40:40.304408+0000","last_became_active":"2024-04-25T14:38:09.544147+0000","last_became_peered":"2024-04-25T14:38:09.544147+0000","last_unstale":"2024-04-25T14:40:40.304408+0000","last_undegraded":"2024-04-25T14:40:40.304408+0000","last_fullsized":"2024-04-25T14:40:40.304408+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:35:22.971028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.180558+0000","last_change":"2024-04-25T14:38:10.426851+0000","last_active":"2024-04-25T14:40:40.180558+0000","last_peered":"2024-04-25T14:40:40.180558+0000","last_clean":"2024-04-25T14:40:40.180558+0000","last_became_active":"2024-04-25T14:38:10.426475+0000","last_became_peered":"2024-04-25T14:38:10.426475+0000","last_unstale":"2024-04-25T14:40:40.180558+0000","last_undegraded":"2024-04-25T14:40:40.180558+0000","last_fullsized":"2024-04-25T14:40:40.180558+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:28.213318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"93'8","reported_seq":197,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:41.104308+0000","last_change":"2024-04-25T14:39:15.979586+0000","last_active":"2024-04-25T14:40:41.104308+0000","last_peered":"2024-04-25T14:40:41.104308+0000","last_clean":"2024-04-25T14:40:41.104308+0000","last_became_active":"2024-04-25T14:39:15.974397+0000","last_became_peered":"2024-04-25T14:39:15.974397+0000","last_unstale":"2024-04-25T14:40:41.104308+0000","last_undegraded":"2024-04-25T14:40:41.104308+0000","last_fullsized":"2024-04-25T14:40:41.104308+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:02.152202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":149,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.304385+0000","last_change":"2024-04-25T14:38:10.425424+0000","last_active":"2024-04-25T14:40:40.304385+0000","last_peered":"2024-04-25T14:40:40.304385+0000","last_clean":"2024-04-25T14:40:40.304385+0000","last_became_active":"2024-04-25T14:38:10.425101+0000","last_became_peered":"2024-04-25T14:38:10.425101+0000","last_unstale":"2024-04-25T14:40:40.304385+0000","last_undegraded":"2024-04-25T14:40:40.304385+0000","last_fullsized":"2024-04-25T14:40:40.304385+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:52:29.871113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.179499+0000","last_change":"2024-04-25T14:38:09.552287+0000","last_active":"2024-04-25T14:40:40.179499+0000","last_peered":"2024-04-25T14:40:40.179499+0000","last_clean":"2024-04-25T14:40:40.179499+0000","last_became_active":"2024-04-25T14:38:09.551957+0000","last_became_peered":"2024-04-25T14:38:09.551957+0000","last_unstale":"2024-04-25T14:40:40.179499+0000","last_undegraded":"2024-04-25T14:40:40.179499+0000","last_fullsized":"2024-04-25T14:40:40.179499+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:22.586868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.180162+0000","last_change":"2024-04-25T14:39:15.990838+0000","last_active":"2024-04-25T14:40:40.180162+0000","last_peered":"2024-04-25T14:40:40.180162+0000","last_clean":"2024-04-25T14:40:40.180162+0000","last_became_active":"2024-04-25T14:39:15.990370+0000","last_became_peered":"2024-04-25T14:39:15.990370+0000","last_unstale":"2024-04-25T14:40:40.180162+0000","last_undegraded":"2024-04-25T14:40:40.180162+0000","last_fullsized":"2024-04-25T14:40:40.180162+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:50.676958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.676465+0000","last_change":"2024-04-25T14:38:09.552376+0000","last_active":"2024-04-25T14:40:40.676465+0000","last_peered":"2024-04-25T14:40:40.676465+0000","last_clean":"2024-04-25T14:40:40.676465+0000","last_became_active":"2024-04-25T14:38:09.548614+0000","last_became_peered":"2024-04-25T14:38:09.548614+0000","last_unstale":"2024-04-25T14:40:40.676465+0000","last_undegraded":"2024-04-25T14:40:40.676465+0000","last_fullsized":"2024-04-25T14:40:40.676465+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:27:27.133106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.181241+0000","last_change":"2024-04-25T14:38:10.434611+0000","last_active":"2024-04-25T14:40:40.181241+0000","last_peered":"2024-04-25T14:40:40.181241+0000","last_clean":"2024-04-25T14:40:40.181241+0000","last_became_active":"2024-04-25T14:38:10.434280+0000","last_became_peered":"2024-04-25T14:38:10.434280+0000","last_unstale":"2024-04-25T14:40:40.181241+0000","last_undegraded":"2024-04-25T14:40:40.181241+0000","last_fullsized":"2024-04-25T14:40:40.181241+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:12:32.275663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.179937+0000","last_change":"2024-04-25T14:38:09.539743+0000","last_active":"2024-04-25T14:40:40.179937+0000","last_peered":"2024-04-25T14:40:40.179937+0000","last_clean":"2024-04-25T14:40:40.179937+0000","last_became_active":"2024-04-25T14:38:09.539272+0000","last_became_peered":"2024-04-25T14:38:09.539272+0000","last_unstale":"2024-04-25T14:40:40.179937+0000","last_undegraded":"2024-04-25T14:40:40.179937+0000","last_fullsized":"2024-04-25T14:40:40.179937+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:29:23.159901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.282536+0000","last_change":"2024-04-25T14:38:10.444899+0000","last_active":"2024-04-25T14:40:40.282536+0000","last_peered":"2024-04-25T14:40:40.282536+0000","last_clean":"2024-04-25T14:40:40.282536+0000","last_became_active":"2024-04-25T14:38:10.444383+0000","last_became_peered":"2024-04-25T14:38:10.444383+0000","last_unstale":"2024-04-25T14:40:40.282536+0000","last_undegraded":"2024-04-25T14:40:40.282536+0000","last_fullsized":"2024-04-25T14:40:40.282536+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:21.351436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"93'5","reported_seq":177,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:41.103337+0000","last_change":"2024-04-25T14:38:09.550181+0000","last_active":"2024-04-25T14:40:41.103337+0000","last_peered":"2024-04-25T14:40:41.103337+0000","last_clean":"2024-04-25T14:40:41.103337+0000","last_became_active":"2024-04-25T14:38:09.549824+0000","last_became_peered":"2024-04-25T14:38:09.549824+0000","last_unstale":"2024-04-25T14:40:41.103337+0000","last_undegraded":"2024-04-25T14:40:41.103337+0000","last_fullsized":"2024-04-25T14:40:41.103337+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:59.220917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":146,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.224700+0000","last_change":"2024-04-25T14:38:10.421339+0000","last_active":"2024-04-25T14:40:40.224700+0000","last_peered":"2024-04-25T14:40:40.224700+0000","last_clean":"2024-04-25T14:40:40.224700+0000","last_became_active":"2024-04-25T14:38:10.421005+0000","last_became_peered":"2024-04-25T14:38:10.421005+0000","last_unstale":"2024-04-25T14:40:40.224700+0000","last_undegraded":"2024-04-25T14:40:40.224700+0000","last_fullsized":"2024-04-25T14:40:40.224700+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:51.949054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.482920+0000","last_change":"2024-04-25T14:38:09.548245+0000","last_active":"2024-04-25T14:40:40.482920+0000","last_peered":"2024-04-25T14:40:40.482920+0000","last_clean":"2024-04-25T14:40:40.482920+0000","last_became_active":"2024-04-25T14:38:09.547905+0000","last_became_peered":"2024-04-25T14:38:09.547905+0000","last_unstale":"2024-04-25T14:40:40.482920+0000","last_undegraded":"2024-04-25T14:40:40.482920+0000","last_fullsized":"2024-04-25T14:40:40.482920+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:56.798594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.991226+0000","last_change":"2024-04-25T14:38:10.431427+0000","last_active":"2024-04-25T14:40:40.991226+0000","last_peered":"2024-04-25T14:40:40.991226+0000","last_clean":"2024-04-25T14:40:40.991226+0000","last_became_active":"2024-04-25T14:38:10.428759+0000","last_became_peered":"2024-04-25T14:38:10.428759+0000","last_unstale":"2024-04-25T14:40:40.991226+0000","last_undegraded":"2024-04-25T14:40:40.991226+0000","last_fullsized":"2024-04-25T14:40:40.991226+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:15:27.341445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.180647+0000","last_change":"2024-04-25T14:38:09.556839+0000","last_active":"2024-04-25T14:40:40.180647+0000","last_peered":"2024-04-25T14:40:40.180647+0000","last_clean":"2024-04-25T14:40:40.180647+0000","last_became_active":"2024-04-25T14:38:09.556443+0000","last_became_peered":"2024-04-25T14:38:09.556443+0000","last_unstale":"2024-04-25T14:40:40.180647+0000","last_undegraded":"2024-04-25T14:40:40.180647+0000","last_fullsized":"2024-04-25T14:40:40.180647+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:46.621883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.224407+0000","last_change":"2024-04-25T14:38:10.414507+0000","last_active":"2024-04-25T14:40:40.224407+0000","last_peered":"2024-04-25T14:40:40.224407+0000","last_clean":"2024-04-25T14:40:40.224407+0000","last_became_active":"2024-04-25T14:38:10.414154+0000","last_became_peered":"2024-04-25T14:38:10.414154+0000","last_unstale":"2024-04-25T14:40:40.224407+0000","last_undegraded":"2024-04-25T14:40:40.224407+0000","last_fullsized":"2024-04-25T14:40:40.224407+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:12.208774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"93'36","reported_seq":365,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:41.104330+0000","last_change":"2024-04-25T14:38:09.549239+0000","last_active":"2024-04-25T14:40:41.104330+0000","last_peered":"2024-04-25T14:40:41.104330+0000","last_clean":"2024-04-25T14:40:41.104330+0000","last_became_active":"2024-04-25T14:38:09.548888+0000","last_became_peered":"2024-04-25T14:38:09.548888+0000","last_unstale":"2024-04-25T14:40:41.104330+0000","last_undegraded":"2024-04-25T14:40:41.104330+0000","last_fullsized":"2024-04-25T14:40:41.104330+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:36:46.536908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68610,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":303,"num_read_kb":219,"num_write":37,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.304392+0000","last_change":"2024-04-25T14:38:10.417510+0000","last_active":"2024-04-25T14:40:40.304392+0000","last_peered":"2024-04-25T14:40:40.304392+0000","last_clean":"2024-04-25T14:40:40.304392+0000","last_became_active":"2024-04-25T14:38:10.417079+0000","last_became_peered":"2024-04-25T14:38:10.417079+0000","last_unstale":"2024-04-25T14:40:40.304392+0000","last_undegraded":"2024-04-25T14:40:40.304392+0000","last_fullsized":"2024-04-25T14:40:40.304392+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:35.213199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.224809+0000","last_change":"2024-04-25T14:38:09.563650+0000","last_active":"2024-04-25T14:40:40.224809+0000","last_peered":"2024-04-25T14:40:40.224809+0000","last_clean":"2024-04-25T14:40:40.224809+0000","last_became_active":"2024-04-25T14:38:09.563347+0000","last_became_peered":"2024-04-25T14:38:09.563347+0000","last_unstale":"2024-04-25T14:40:40.224809+0000","last_undegraded":"2024-04-25T14:40:40.224809+0000","last_fullsized":"2024-04-25T14:40:40.224809+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:16.793191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.676501+0000","last_change":"2024-04-25T14:38:10.447504+0000","last_active":"2024-04-25T14:40:40.676501+0000","last_peered":"2024-04-25T14:40:40.676501+0000","last_clean":"2024-04-25T14:40:40.676501+0000","last_became_active":"2024-04-25T14:38:10.447157+0000","last_became_peered":"2024-04-25T14:38:10.447157+0000","last_unstale":"2024-04-25T14:40:40.676501+0000","last_undegraded":"2024-04-25T14:40:40.676501+0000","last_fullsized":"2024-04-25T14:40:40.676501+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:38.940320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.180450+0000","last_change":"2024-04-25T14:38:09.545214+0000","last_active":"2024-04-25T14:40:40.180450+0000","last_peered":"2024-04-25T14:40:40.180450+0000","last_clean":"2024-04-25T14:40:40.180450+0000","last_became_active":"2024-04-25T14:38:09.544903+0000","last_became_peered":"2024-04-25T14:38:09.544903+0000","last_unstale":"2024-04-25T14:40:40.180450+0000","last_undegraded":"2024-04-25T14:40:40.180450+0000","last_fullsized":"2024-04-25T14:40:40.180450+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:58:15.817036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.482678+0000","last_change":"2024-04-25T14:38:10.432017+0000","last_active":"2024-04-25T14:40:40.482678+0000","last_peered":"2024-04-25T14:40:40.482678+0000","last_clean":"2024-04-25T14:40:40.482678+0000","last_became_active":"2024-04-25T14:38:10.431596+0000","last_became_peered":"2024-04-25T14:38:10.431596+0000","last_unstale":"2024-04-25T14:40:40.482678+0000","last_undegraded":"2024-04-25T14:40:40.482678+0000","last_fullsized":"2024-04-25T14:40:40.482678+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:30.653642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"93'7","reported_seq":185,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.970143+0000","last_change":"2024-04-25T14:38:09.545477+0000","last_active":"2024-04-25T14:40:40.970143+0000","last_peered":"2024-04-25T14:40:40.970143+0000","last_clean":"2024-04-25T14:40:40.970143+0000","last_became_active":"2024-04-25T14:38:09.545093+0000","last_became_peered":"2024-04-25T14:38:09.545093+0000","last_unstale":"2024-04-25T14:40:40.970143+0000","last_undegraded":"2024-04-25T14:40:40.970143+0000","last_fullsized":"2024-04-25T14:40:40.970143+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:56.589231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1821,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":153,"num_read_kb":7,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.481697+0000","last_change":"2024-04-25T14:38:10.418914+0000","last_active":"2024-04-25T14:40:40.481697+0000","last_peered":"2024-04-25T14:40:40.481697+0000","last_clean":"2024-04-25T14:40:40.481697+0000","last_became_active":"2024-04-25T14:38:10.418600+0000","last_became_peered":"2024-04-25T14:38:10.418600+0000","last_unstale":"2024-04-25T14:40:40.481697+0000","last_undegraded":"2024-04-25T14:40:40.481697+0000","last_fullsized":"2024-04-25T14:40:40.481697+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:31.324035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.304080+0000","last_change":"2024-04-25T14:38:09.556574+0000","last_active":"2024-04-25T14:40:40.304080+0000","last_peered":"2024-04-25T14:40:40.304080+0000","last_clean":"2024-04-25T14:40:40.304080+0000","last_became_active":"2024-04-25T14:38:09.556071+0000","last_became_peered":"2024-04-25T14:38:09.556071+0000","last_unstale":"2024-04-25T14:40:40.304080+0000","last_undegraded":"2024-04-25T14:40:40.304080+0000","last_fullsized":"2024-04-25T14:40:40.304080+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:05.424593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.180084+0000","last_change":"2024-04-25T14:38:10.424591+0000","last_active":"2024-04-25T14:40:40.180084+0000","last_peered":"2024-04-25T14:40:40.180084+0000","last_clean":"2024-04-25T14:40:40.180084+0000","last_became_active":"2024-04-25T14:38:10.424258+0000","last_became_peered":"2024-04-25T14:38:10.424258+0000","last_unstale":"2024-04-25T14:40:40.180084+0000","last_undegraded":"2024-04-25T14:40:40.180084+0000","last_fullsized":"2024-04-25T14:40:40.180084+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:22.753595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"93'3","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.304606+0000","last_change":"2024-04-25T14:38:09.546073+0000","last_active":"2024-04-25T14:40:40.304606+0000","last_peered":"2024-04-25T14:40:40.304606+0000","last_clean":"2024-04-25T14:40:40.304606+0000","last_became_active":"2024-04-25T14:38:09.545783+0000","last_became_peered":"2024-04-25T14:38:09.545783+0000","last_unstale":"2024-04-25T14:40:40.304606+0000","last_undegraded":"2024-04-25T14:40:40.304606+0000","last_fullsized":"2024-04-25T14:40:40.304606+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:42.537431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.650953+0000","last_change":"2024-04-25T14:38:10.417230+0000","last_active":"2024-04-25T14:40:40.650953+0000","last_peered":"2024-04-25T14:40:40.650953+0000","last_clean":"2024-04-25T14:40:40.650953+0000","last_became_active":"2024-04-25T14:38:10.416891+0000","last_became_peered":"2024-04-25T14:38:10.416891+0000","last_unstale":"2024-04-25T14:40:40.650953+0000","last_undegraded":"2024-04-25T14:40:40.650953+0000","last_fullsized":"2024-04-25T14:40:40.650953+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:10:13.434518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.281801+0000","last_change":"2024-04-25T14:39:15.994906+0000","last_active":"2024-04-25T14:40:40.281801+0000","last_peered":"2024-04-25T14:40:40.281801+0000","last_clean":"2024-04-25T14:40:40.281801+0000","last_became_active":"2024-04-25T14:39:15.994365+0000","last_became_peered":"2024-04-25T14:39:15.994365+0000","last_unstale":"2024-04-25T14:40:40.281801+0000","last_undegraded":"2024-04-25T14:40:40.281801+0000","last_fullsized":"2024-04-25T14:40:40.281801+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:19.026241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.179623+0000","last_change":"2024-04-25T14:38:10.446846+0000","last_active":"2024-04-25T14:40:40.179623+0000","last_peered":"2024-04-25T14:40:40.179623+0000","last_clean":"2024-04-25T14:40:40.179623+0000","last_became_active":"2024-04-25T14:38:10.446498+0000","last_became_peered":"2024-04-25T14:38:10.446498+0000","last_unstale":"2024-04-25T14:40:40.179623+0000","last_undegraded":"2024-04-25T14:40:40.179623+0000","last_fullsized":"2024-04-25T14:40:40.179623+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:14:37.179198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.482585+0000","last_change":"2024-04-25T14:38:09.544283+0000","last_active":"2024-04-25T14:40:40.482585+0000","last_peered":"2024-04-25T14:40:40.482585+0000","last_clean":"2024-04-25T14:40:40.482585+0000","last_became_active":"2024-04-25T14:38:09.543135+0000","last_became_peered":"2024-04-25T14:38:09.543135+0000","last_unstale":"2024-04-25T14:40:40.482585+0000","last_undegraded":"2024-04-25T14:40:40.482585+0000","last_fullsized":"2024-04-25T14:40:40.482585+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:05.977062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.179523+0000","last_change":"2024-04-25T14:38:10.432863+0000","last_active":"2024-04-25T14:40:40.179523+0000","last_peered":"2024-04-25T14:40:40.179523+0000","last_clean":"2024-04-25T14:40:40.179523+0000","last_became_active":"2024-04-25T14:38:10.432493+0000","last_became_peered":"2024-04-25T14:38:10.432493+0000","last_unstale":"2024-04-25T14:40:40.179523+0000","last_undegraded":"2024-04-25T14:40:40.179523+0000","last_fullsized":"2024-04-25T14:40:40.179523+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:25:53.774764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.180156+0000","last_change":"2024-04-25T14:39:15.981979+0000","last_active":"2024-04-25T14:40:40.180156+0000","last_peered":"2024-04-25T14:40:40.180156+0000","last_clean":"2024-04-25T14:40:40.180156+0000","last_became_active":"2024-04-25T14:39:15.981539+0000","last_became_peered":"2024-04-25T14:39:15.981539+0000","last_unstale":"2024-04-25T14:40:40.180156+0000","last_undegraded":"2024-04-25T14:40:40.180156+0000","last_fullsized":"2024-04-25T14:40:40.180156+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:22.906826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":178,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.676315+0000","last_change":"2024-04-25T14:38:09.554305+0000","last_active":"2024-04-25T14:40:40.676315+0000","last_peered":"2024-04-25T14:40:40.676315+0000","last_clean":"2024-04-25T14:40:40.676315+0000","last_became_active":"2024-04-25T14:38:09.553984+0000","last_became_peered":"2024-04-25T14:38:09.553984+0000","last_unstale":"2024-04-25T14:40:40.676315+0000","last_undegraded":"2024-04-25T14:40:40.676315+0000","last_fullsized":"2024-04-25T14:40:40.676315+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:41.580867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":146,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.181490+0000","last_change":"2024-04-25T14:38:10.448290+0000","last_active":"2024-04-25T14:40:40.181490+0000","last_peered":"2024-04-25T14:40:40.181490+0000","last_clean":"2024-04-25T14:40:40.181490+0000","last_became_active":"2024-04-25T14:38:10.446147+0000","last_became_peered":"2024-04-25T14:38:10.446147+0000","last_unstale":"2024-04-25T14:40:40.181490+0000","last_undegraded":"2024-04-25T14:40:40.181490+0000","last_fullsized":"2024-04-25T14:40:40.181490+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:12.775551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.481561+0000","last_change":"2024-04-25T14:38:09.545209+0000","last_active":"2024-04-25T14:40:40.481561+0000","last_peered":"2024-04-25T14:40:40.481561+0000","last_clean":"2024-04-25T14:40:40.481561+0000","last_became_active":"2024-04-25T14:38:09.544861+0000","last_became_peered":"2024-04-25T14:38:09.544861+0000","last_unstale":"2024-04-25T14:40:40.481561+0000","last_undegraded":"2024-04-25T14:40:40.481561+0000","last_fullsized":"2024-04-25T14:40:40.481561+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:25.829162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":360,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.481274+0000","last_change":"2024-04-25T14:36:48.250081+0000","last_active":"2024-04-25T14:40:40.481274+0000","last_peered":"2024-04-25T14:40:40.481274+0000","last_clean":"2024-04-25T14:40:40.481274+0000","last_became_active":"2024-04-25T14:36:48.249520+0000","last_became_peered":"2024-04-25T14:36:48.249520+0000","last_unstale":"2024-04-25T14:40:40.481274+0000","last_undegraded":"2024-04-25T14:40:40.481274+0000","last_fullsized":"2024-04-25T14:40:40.481274+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:33:03.061016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:33:03.061016+0000","last_clean_scrub_stamp":"2024-04-25T14:33:03.061016+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:04:06.653326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.676039+0000","last_change":"2024-04-25T14:40:16.498939+0000","last_active":"2024-04-25T14:40:40.676039+0000","last_peered":"2024-04-25T14:40:40.676039+0000","last_clean":"2024-04-25T14:40:40.676039+0000","last_became_active":"2024-04-25T14:40:16.498128+0000","last_became_peered":"2024-04-25T14:40:16.498128+0000","last_unstale":"2024-04-25T14:40:40.676039+0000","last_undegraded":"2024-04-25T14:40:40.676039+0000","last_fullsized":"2024-04-25T14:40:40.676039+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:14.942944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.481930+0000","last_change":"2024-04-25T14:39:15.982120+0000","last_active":"2024-04-25T14:40:40.481930+0000","last_peered":"2024-04-25T14:40:40.481930+0000","last_clean":"2024-04-25T14:40:40.481930+0000","last_became_active":"2024-04-25T14:39:15.975343+0000","last_became_peered":"2024-04-25T14:39:15.975343+0000","last_unstale":"2024-04-25T14:40:40.481930+0000","last_undegraded":"2024-04-25T14:40:40.481930+0000","last_fullsized":"2024-04-25T14:40:40.481930+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:38.632052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.650887+0000","last_change":"2024-04-25T14:39:15.998744+0000","last_active":"2024-04-25T14:40:40.650887+0000","last_peered":"2024-04-25T14:40:40.650887+0000","last_clean":"2024-04-25T14:40:40.650887+0000","last_became_active":"2024-04-25T14:39:15.993416+0000","last_became_peered":"2024-04-25T14:39:15.993416+0000","last_unstale":"2024-04-25T14:40:40.650887+0000","last_undegraded":"2024-04-25T14:40:40.650887+0000","last_fullsized":"2024-04-25T14:40:40.650887+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:24.019707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.676812+0000","last_change":"2024-04-25T14:38:10.429554+0000","last_active":"2024-04-25T14:40:40.676812+0000","last_peered":"2024-04-25T14:40:40.676812+0000","last_clean":"2024-04-25T14:40:40.676812+0000","last_became_active":"2024-04-25T14:38:10.428944+0000","last_became_peered":"2024-04-25T14:38:10.428944+0000","last_unstale":"2024-04-25T14:40:40.676812+0000","last_undegraded":"2024-04-25T14:40:40.676812+0000","last_fullsized":"2024-04-25T14:40:40.676812+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:38.013028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"93'47","reported_seq":96,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.434928+0000","last_change":"2024-04-25T14:38:09.564849+0000","last_active":"2024-04-25T14:40:40.434928+0000","last_peered":"2024-04-25T14:40:40.434928+0000","last_clean":"2024-04-25T14:40:40.434928+0000","last_became_active":"2024-04-25T14:38:09.564354+0000","last_became_peered":"2024-04-25T14:38:09.564354+0000","last_unstale":"2024-04-25T14:40:40.434928+0000","last_undegraded":"2024-04-25T14:40:40.434928+0000","last_fullsized":"2024-04-25T14:40:40.434928+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:44.308366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.181997+0000","last_change":"2024-04-25T14:38:10.438178+0000","last_active":"2024-04-25T14:40:40.181997+0000","last_peered":"2024-04-25T14:40:40.181997+0000","last_clean":"2024-04-25T14:40:40.181997+0000","last_became_active":"2024-04-25T14:38:10.429936+0000","last_became_peered":"2024-04-25T14:38:10.429936+0000","last_unstale":"2024-04-25T14:40:40.181997+0000","last_undegraded":"2024-04-25T14:40:40.181997+0000","last_fullsized":"2024-04-25T14:40:40.181997+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:33.598747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.481640+0000","last_change":"2024-04-25T14:38:09.547680+0000","last_active":"2024-04-25T14:40:40.481640+0000","last_peered":"2024-04-25T14:40:40.481640+0000","last_clean":"2024-04-25T14:40:40.481640+0000","last_became_active":"2024-04-25T14:38:09.547337+0000","last_became_peered":"2024-04-25T14:38:09.547337+0000","last_unstale":"2024-04-25T14:40:40.481640+0000","last_undegraded":"2024-04-25T14:40:40.481640+0000","last_fullsized":"2024-04-25T14:40:40.481640+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:32.754439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.991670+0000","last_change":"2024-04-25T14:38:10.447115+0000","last_active":"2024-04-25T14:40:40.991670+0000","last_peered":"2024-04-25T14:40:40.991670+0000","last_clean":"2024-04-25T14:40:40.991670+0000","last_became_active":"2024-04-25T14:38:10.446681+0000","last_became_peered":"2024-04-25T14:38:10.446681+0000","last_unstale":"2024-04-25T14:40:40.991670+0000","last_undegraded":"2024-04-25T14:40:40.991670+0000","last_fullsized":"2024-04-25T14:40:40.991670+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:02:58.942053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.224061+0000","last_change":"2024-04-25T14:38:09.551523+0000","last_active":"2024-04-25T14:40:40.224061+0000","last_peered":"2024-04-25T14:40:40.224061+0000","last_clean":"2024-04-25T14:40:40.224061+0000","last_became_active":"2024-04-25T14:38:09.551213+0000","last_became_peered":"2024-04-25T14:38:09.551213+0000","last_unstale":"2024-04-25T14:40:40.224061+0000","last_undegraded":"2024-04-25T14:40:40.224061+0000","last_fullsized":"2024-04-25T14:40:40.224061+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:48.948571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.304011+0000","last_change":"2024-04-25T14:38:10.420032+0000","last_active":"2024-04-25T14:40:40.304011+0000","last_peered":"2024-04-25T14:40:40.304011+0000","last_clean":"2024-04-25T14:40:40.304011+0000","last_became_active":"2024-04-25T14:38:10.418349+0000","last_became_peered":"2024-04-25T14:38:10.418349+0000","last_unstale":"2024-04-25T14:40:40.304011+0000","last_undegraded":"2024-04-25T14:40:40.304011+0000","last_fullsized":"2024-04-25T14:40:40.304011+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:38:55.669414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.181847+0000","last_change":"2024-04-25T14:38:09.567097+0000","last_active":"2024-04-25T14:40:40.181847+0000","last_peered":"2024-04-25T14:40:40.181847+0000","last_clean":"2024-04-25T14:40:40.181847+0000","last_became_active":"2024-04-25T14:38:09.566852+0000","last_became_peered":"2024-04-25T14:38:09.566852+0000","last_unstale":"2024-04-25T14:40:40.181847+0000","last_undegraded":"2024-04-25T14:40:40.181847+0000","last_fullsized":"2024-04-25T14:40:40.181847+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:41:55.773005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.650673+0000","last_change":"2024-04-25T14:38:10.432691+0000","last_active":"2024-04-25T14:40:40.650673+0000","last_peered":"2024-04-25T14:40:40.650673+0000","last_clean":"2024-04-25T14:40:40.650673+0000","last_became_active":"2024-04-25T14:38:10.432321+0000","last_became_peered":"2024-04-25T14:38:10.432321+0000","last_unstale":"2024-04-25T14:40:40.650673+0000","last_undegraded":"2024-04-25T14:40:40.650673+0000","last_fullsized":"2024-04-25T14:40:40.650673+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:47:58.026331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.224545+0000","last_change":"2024-04-25T14:38:09.563692+0000","last_active":"2024-04-25T14:40:40.224545+0000","last_peered":"2024-04-25T14:40:40.224545+0000","last_clean":"2024-04-25T14:40:40.224545+0000","last_became_active":"2024-04-25T14:38:09.563345+0000","last_became_peered":"2024-04-25T14:38:09.563345+0000","last_unstale":"2024-04-25T14:40:40.224545+0000","last_undegraded":"2024-04-25T14:40:40.224545+0000","last_fullsized":"2024-04-25T14:40:40.224545+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:43.221429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.182980+0000","last_change":"2024-04-25T14:38:10.442519+0000","last_active":"2024-04-25T14:40:40.182980+0000","last_peered":"2024-04-25T14:40:40.182980+0000","last_clean":"2024-04-25T14:40:40.182980+0000","last_became_active":"2024-04-25T14:38:10.439045+0000","last_became_peered":"2024-04-25T14:38:10.439045+0000","last_unstale":"2024-04-25T14:40:40.182980+0000","last_undegraded":"2024-04-25T14:40:40.182980+0000","last_fullsized":"2024-04-25T14:40:40.182980+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:42.425106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.304120+0000","last_change":"2024-04-25T14:38:09.550363+0000","last_active":"2024-04-25T14:40:40.304120+0000","last_peered":"2024-04-25T14:40:40.304120+0000","last_clean":"2024-04-25T14:40:40.304120+0000","last_became_active":"2024-04-25T14:38:09.550044+0000","last_became_peered":"2024-04-25T14:38:09.550044+0000","last_unstale":"2024-04-25T14:40:40.304120+0000","last_undegraded":"2024-04-25T14:40:40.304120+0000","last_fullsized":"2024-04-25T14:40:40.304120+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:25.573353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.482361+0000","last_change":"2024-04-25T14:38:10.428736+0000","last_active":"2024-04-25T14:40:40.482361+0000","last_peered":"2024-04-25T14:40:40.482361+0000","last_clean":"2024-04-25T14:40:40.482361+0000","last_became_active":"2024-04-25T14:38:10.428410+0000","last_became_peered":"2024-04-25T14:38:10.428410+0000","last_unstale":"2024-04-25T14:40:40.482361+0000","last_undegraded":"2024-04-25T14:40:40.482361+0000","last_fullsized":"2024-04-25T14:40:40.482361+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:02:51.474220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.224711+0000","last_change":"2024-04-25T14:38:09.552800+0000","last_active":"2024-04-25T14:40:40.224711+0000","last_peered":"2024-04-25T14:40:40.224711+0000","last_clean":"2024-04-25T14:40:40.224711+0000","last_became_active":"2024-04-25T14:38:09.552460+0000","last_became_peered":"2024-04-25T14:38:09.552460+0000","last_unstale":"2024-04-25T14:40:40.224711+0000","last_undegraded":"2024-04-25T14:40:40.224711+0000","last_fullsized":"2024-04-25T14:40:40.224711+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:00.800770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.482907+0000","last_change":"2024-04-25T14:38:10.424359+0000","last_active":"2024-04-25T14:40:40.482907+0000","last_peered":"2024-04-25T14:40:40.482907+0000","last_clean":"2024-04-25T14:40:40.482907+0000","last_became_active":"2024-04-25T14:38:10.424020+0000","last_became_peered":"2024-04-25T14:38:10.424020+0000","last_unstale":"2024-04-25T14:40:40.482907+0000","last_undegraded":"2024-04-25T14:40:40.482907+0000","last_fullsized":"2024-04-25T14:40:40.482907+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:14.700772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"93'39","reported_seq":73,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.428697+0000","last_change":"2024-04-25T14:38:09.545637+0000","last_active":"2024-04-25T14:40:40.428697+0000","last_peered":"2024-04-25T14:40:40.428697+0000","last_clean":"2024-04-25T14:40:40.428697+0000","last_became_active":"2024-04-25T14:38:09.545319+0000","last_became_peered":"2024-04-25T14:38:09.545319+0000","last_unstale":"2024-04-25T14:40:40.428697+0000","last_undegraded":"2024-04-25T14:40:40.428697+0000","last_fullsized":"2024-04-25T14:40:40.428697+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:02.284715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.482427+0000","last_change":"2024-04-25T14:38:10.446928+0000","last_active":"2024-04-25T14:40:40.482427+0000","last_peered":"2024-04-25T14:40:40.482427+0000","last_clean":"2024-04-25T14:40:40.482427+0000","last_became_active":"2024-04-25T14:38:10.446518+0000","last_became_peered":"2024-04-25T14:38:10.446518+0000","last_unstale":"2024-04-25T14:40:40.482427+0000","last_undegraded":"2024-04-25T14:40:40.482427+0000","last_fullsized":"2024-04-25T14:40:40.482427+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:40.769059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.677657+0000","last_change":"2024-04-25T14:38:09.557381+0000","last_active":"2024-04-25T14:40:40.677657+0000","last_peered":"2024-04-25T14:40:40.677657+0000","last_clean":"2024-04-25T14:40:40.677657+0000","last_became_active":"2024-04-25T14:38:09.556504+0000","last_became_peered":"2024-04-25T14:38:09.556504+0000","last_unstale":"2024-04-25T14:40:40.677657+0000","last_undegraded":"2024-04-25T14:40:40.677657+0000","last_fullsized":"2024-04-25T14:40:40.677657+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:01:02.761249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.482978+0000","last_change":"2024-04-25T14:38:10.418104+0000","last_active":"2024-04-25T14:40:40.482978+0000","last_peered":"2024-04-25T14:40:40.482978+0000","last_clean":"2024-04-25T14:40:40.482978+0000","last_became_active":"2024-04-25T14:38:10.417774+0000","last_became_peered":"2024-04-25T14:38:10.417774+0000","last_unstale":"2024-04-25T14:40:40.482978+0000","last_undegraded":"2024-04-25T14:40:40.482978+0000","last_fullsized":"2024-04-25T14:40:40.482978+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:32:44.229899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.650780+0000","last_change":"2024-04-25T14:38:10.423546+0000","last_active":"2024-04-25T14:40:40.650780+0000","last_peered":"2024-04-25T14:40:40.650780+0000","last_clean":"2024-04-25T14:40:40.650780+0000","last_became_active":"2024-04-25T14:38:10.423216+0000","last_became_peered":"2024-04-25T14:38:10.423216+0000","last_unstale":"2024-04-25T14:40:40.650780+0000","last_undegraded":"2024-04-25T14:40:40.650780+0000","last_fullsized":"2024-04-25T14:40:40.650780+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:58.640669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.224609+0000","last_change":"2024-04-25T14:38:10.422504+0000","last_active":"2024-04-25T14:40:40.224609+0000","last_peered":"2024-04-25T14:40:40.224609+0000","last_clean":"2024-04-25T14:40:40.224609+0000","last_became_active":"2024-04-25T14:38:10.422095+0000","last_became_peered":"2024-04-25T14:38:10.422095+0000","last_unstale":"2024-04-25T14:40:40.224609+0000","last_undegraded":"2024-04-25T14:40:40.224609+0000","last_fullsized":"2024-04-25T14:40:40.224609+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:25.795962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.224467+0000","last_change":"2024-04-25T14:38:10.447446+0000","last_active":"2024-04-25T14:40:40.224467+0000","last_peered":"2024-04-25T14:40:40.224467+0000","last_clean":"2024-04-25T14:40:40.224467+0000","last_became_active":"2024-04-25T14:38:10.447056+0000","last_became_peered":"2024-04-25T14:38:10.447056+0000","last_unstale":"2024-04-25T14:40:40.224467+0000","last_undegraded":"2024-04-25T14:40:40.224467+0000","last_fullsized":"2024-04-25T14:40:40.224467+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:40.462340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.990840+0000","last_change":"2024-04-25T14:38:10.433444+0000","last_active":"2024-04-25T14:40:40.990840+0000","last_peered":"2024-04-25T14:40:40.990840+0000","last_clean":"2024-04-25T14:40:40.990840+0000","last_became_active":"2024-04-25T14:38:10.433153+0000","last_became_peered":"2024-04-25T14:38:10.433153+0000","last_unstale":"2024-04-25T14:40:40.990840+0000","last_undegraded":"2024-04-25T14:40:40.990840+0000","last_fullsized":"2024-04-25T14:40:40.990840+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:10.722650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.676722+0000","last_change":"2024-04-25T14:38:10.438322+0000","last_active":"2024-04-25T14:40:40.676722+0000","last_peered":"2024-04-25T14:40:40.676722+0000","last_clean":"2024-04-25T14:40:40.676722+0000","last_became_active":"2024-04-25T14:38:10.436103+0000","last_became_peered":"2024-04-25T14:38:10.436103+0000","last_unstale":"2024-04-25T14:40:40.676722+0000","last_undegraded":"2024-04-25T14:40:40.676722+0000","last_fullsized":"2024-04-25T14:40:40.676722+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:16.790202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.650995+0000","last_change":"2024-04-25T14:38:10.419710+0000","last_active":"2024-04-25T14:40:40.650995+0000","last_peered":"2024-04-25T14:40:40.650995+0000","last_clean":"2024-04-25T14:40:40.650995+0000","last_became_active":"2024-04-25T14:38:10.419406+0000","last_became_peered":"2024-04-25T14:38:10.419406+0000","last_unstale":"2024-04-25T14:40:40.650995+0000","last_undegraded":"2024-04-25T14:40:40.650995+0000","last_fullsized":"2024-04-25T14:40:40.650995+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:37.964698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.224748+0000","last_change":"2024-04-25T14:38:10.429510+0000","last_active":"2024-04-25T14:40:40.224748+0000","last_peered":"2024-04-25T14:40:40.224748+0000","last_clean":"2024-04-25T14:40:40.224748+0000","last_became_active":"2024-04-25T14:38:10.429219+0000","last_became_peered":"2024-04-25T14:38:10.429219+0000","last_unstale":"2024-04-25T14:40:40.224748+0000","last_undegraded":"2024-04-25T14:40:40.224748+0000","last_fullsized":"2024-04-25T14:40:40.224748+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:52:46.673126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.181877+0000","last_change":"2024-04-25T14:38:10.441487+0000","last_active":"2024-04-25T14:40:40.181877+0000","last_peered":"2024-04-25T14:40:40.181877+0000","last_clean":"2024-04-25T14:40:40.181877+0000","last_became_active":"2024-04-25T14:38:10.441111+0000","last_became_peered":"2024-04-25T14:38:10.441111+0000","last_unstale":"2024-04-25T14:40:40.181877+0000","last_undegraded":"2024-04-25T14:40:40.181877+0000","last_fullsized":"2024-04-25T14:40:40.181877+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:59.842687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.179847+0000","last_change":"2024-04-25T14:38:10.436511+0000","last_active":"2024-04-25T14:40:40.179847+0000","last_peered":"2024-04-25T14:40:40.179847+0000","last_clean":"2024-04-25T14:40:40.179847+0000","last_became_active":"2024-04-25T14:38:10.436171+0000","last_became_peered":"2024-04-25T14:38:10.436171+0000","last_unstale":"2024-04-25T14:40:40.179847+0000","last_undegraded":"2024-04-25T14:40:40.179847+0000","last_fullsized":"2024-04-25T14:40:40.179847+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:27:19.605907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.181672+0000","last_change":"2024-04-25T14:38:10.433624+0000","last_active":"2024-04-25T14:40:40.181672+0000","last_peered":"2024-04-25T14:40:40.181672+0000","last_clean":"2024-04-25T14:40:40.181672+0000","last_became_active":"2024-04-25T14:38:10.433258+0000","last_became_peered":"2024-04-25T14:38:10.433258+0000","last_unstale":"2024-04-25T14:40:40.181672+0000","last_undegraded":"2024-04-25T14:40:40.181672+0000","last_fullsized":"2024-04-25T14:40:40.181672+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:53.563110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.224234+0000","last_change":"2024-04-25T14:38:10.446924+0000","last_active":"2024-04-25T14:40:40.224234+0000","last_peered":"2024-04-25T14:40:40.224234+0000","last_clean":"2024-04-25T14:40:40.224234+0000","last_became_active":"2024-04-25T14:38:10.446553+0000","last_became_peered":"2024-04-25T14:38:10.446553+0000","last_unstale":"2024-04-25T14:40:40.224234+0000","last_undegraded":"2024-04-25T14:40:40.224234+0000","last_fullsized":"2024-04-25T14:40:40.224234+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:00.203796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.677384+0000","last_change":"2024-04-25T14:38:10.429952+0000","last_active":"2024-04-25T14:40:40.677384+0000","last_peered":"2024-04-25T14:40:40.677384+0000","last_clean":"2024-04-25T14:40:40.677384+0000","last_became_active":"2024-04-25T14:38:10.429626+0000","last_became_peered":"2024-04-25T14:38:10.429626+0000","last_unstale":"2024-04-25T14:40:40.677384+0000","last_undegraded":"2024-04-25T14:40:40.677384+0000","last_fullsized":"2024-04-25T14:40:40.677384+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:07.287113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.650742+0000","last_change":"2024-04-25T14:38:10.418292+0000","last_active":"2024-04-25T14:40:40.650742+0000","last_peered":"2024-04-25T14:40:40.650742+0000","last_clean":"2024-04-25T14:40:40.650742+0000","last_became_active":"2024-04-25T14:38:10.418005+0000","last_became_peered":"2024-04-25T14:38:10.418005+0000","last_unstale":"2024-04-25T14:40:40.650742+0000","last_undegraded":"2024-04-25T14:40:40.650742+0000","last_fullsized":"2024-04-25T14:40:40.650742+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:25.486993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599192,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1001,"num_read_kb":614,"num_write":294,"num_write_kb":1442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1035,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":1797576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":231,"ondisk_log_size":231,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285612,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1036288,"data_stored":495726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547249,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28844,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716308,"statfs":{"total":95995035648,"available":95965499392,"internally_reserved":0,"allocated":1871872,"data_stored":1288628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710774,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28424,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716728,"statfs":{"total":95995035648,"available":95965929472,"internally_reserved":0,"allocated":1441792,"data_stored":889156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972413,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32584,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712568,"statfs":{"total":95995035648,"available":95961669632,"internally_reserved":0,"allocated":1437696,"data_stored":889054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266753,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28624,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716528,"statfs":{"total":95995035648,"available":95965724672,"internally_reserved":0,"allocated":1642496,"data_stored":1089555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430277,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":486328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574945,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1105920,"data_stored":563732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738471,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1056768,"data_stored":496600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411422,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36436,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708716,"statfs":{"total":95995035648,"available":95957725184,"internally_reserved":0,"allocated":1122304,"data_stored":564460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149783,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28560,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716592,"statfs":{"total":95995035648,"available":95965790208,"internally_reserved":0,"allocated":1581056,"data_stored":1020456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888148,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1048576,"data_stored":498544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691917,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37084,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708068,"statfs":{"total":95995035648,"available":95957061632,"internally_reserved":0,"allocated":1789952,"data_stored":1219606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":69338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":133999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":203021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:40:47.240 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T14:40:47.511 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:47 smithi053 ceph-mon[27773]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:40:47.512 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:47 smithi053 ceph-mon[27773]: from='client.25301 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:40:47.512 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:47 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:47.512 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:47 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:47.512 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:47 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:47.750 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:40:47.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:47 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:47.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:47 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:47.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:47 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:47.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:47 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:47.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:47 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:47.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:47 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:47.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:47 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:47.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:47 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:40:47.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:47 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:40:47.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:47 smithi183 ceph-mon[32584]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:40:47.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:47 smithi183 ceph-mon[32584]: from='client.25301 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:40:47.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:47 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:47.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:47 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:47.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:47 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:47.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:47 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:47.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:47 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:47.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:47 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:47.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:47 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:47.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:47 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:47.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:47 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:47.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:47 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:47.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:47 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:40:47.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:47 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:40:47.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:47 smithi150 ceph-mon[32504]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:40:47.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:47 smithi150 ceph-mon[32504]: from='client.25301 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:40:47.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:47 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:47.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:47 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:47.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:47 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:47.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:47 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:47.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:47 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:47.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:47 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:47.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:47 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:47.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:47 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:47.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:47 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:47.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:47 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:40:47.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:47 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:40:47.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:47 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:40:48.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:48 smithi053 ceph-mon[27773]: Adjusting osd_memory_target on smithi053 to 177.5M 2024-04-25T14:40:48.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:48 smithi053 ceph-mon[27773]: Unable to set osd_memory_target on smithi053 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-25T14:40:48.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:48 smithi053 ceph-mon[27773]: Updating smithi053:/etc/ceph/ceph.conf 2024-04-25T14:40:48.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:48 smithi053 ceph-mon[27773]: Updating smithi150:/etc/ceph/ceph.conf 2024-04-25T14:40:48.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:48 smithi053 ceph-mon[27773]: Updating smithi183:/etc/ceph/ceph.conf 2024-04-25T14:40:48.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:48 smithi053 ceph-mon[27773]: Updating smithi183:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:40:48.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:48 smithi053 ceph-mon[27773]: Updating smithi150:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:40:48.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:48 smithi183 ceph-mon[32584]: Adjusting osd_memory_target on smithi053 to 177.5M 2024-04-25T14:40:48.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:48 smithi183 ceph-mon[32584]: Unable to set osd_memory_target on smithi053 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-25T14:40:48.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:48 smithi183 ceph-mon[32584]: Updating smithi053:/etc/ceph/ceph.conf 2024-04-25T14:40:48.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:48 smithi183 ceph-mon[32584]: Updating smithi150:/etc/ceph/ceph.conf 2024-04-25T14:40:48.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:48 smithi183 ceph-mon[32584]: Updating smithi183:/etc/ceph/ceph.conf 2024-04-25T14:40:48.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:48 smithi183 ceph-mon[32584]: Updating smithi183:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:40:48.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:48 smithi183 ceph-mon[32584]: Updating smithi150:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:40:48.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:48 smithi150 ceph-mon[32504]: Adjusting osd_memory_target on smithi053 to 177.5M 2024-04-25T14:40:48.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:48 smithi150 ceph-mon[32504]: Unable to set osd_memory_target on smithi053 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-25T14:40:48.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:48 smithi150 ceph-mon[32504]: Updating smithi053:/etc/ceph/ceph.conf 2024-04-25T14:40:48.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:48 smithi150 ceph-mon[32504]: Updating smithi150:/etc/ceph/ceph.conf 2024-04-25T14:40:48.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:48 smithi150 ceph-mon[32504]: Updating smithi183:/etc/ceph/ceph.conf 2024-04-25T14:40:48.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:48 smithi150 ceph-mon[32504]: Updating smithi183:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:40:48.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:48 smithi150 ceph-mon[32504]: Updating smithi150:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:40:49.919 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:49 smithi053 ceph-mon[27773]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:40:49.919 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:49 smithi053 ceph-mon[27773]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:40:49.919 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:49 smithi053 ceph-mon[27773]: Updating smithi053:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:40:49.919 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:49 smithi053 ceph-mon[27773]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 131 KiB/s wr, 76 op/s 2024-04-25T14:40:49.919 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:49 smithi053 ceph-mon[27773]: Updating smithi150:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T14:40:49.919 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:49 smithi053 ceph-mon[27773]: Updating smithi183:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T14:40:49.919 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:49 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:49.920 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:49 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:49.920 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:49 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:49.920 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:49 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:49.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:49 smithi183 ceph-mon[32584]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:40:49.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:49 smithi183 ceph-mon[32584]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:40:49.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:49 smithi183 ceph-mon[32584]: Updating smithi053:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:40:49.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:49 smithi183 ceph-mon[32584]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 131 KiB/s wr, 76 op/s 2024-04-25T14:40:49.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:49 smithi183 ceph-mon[32584]: Updating smithi150:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T14:40:49.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:49 smithi183 ceph-mon[32584]: Updating smithi183:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T14:40:49.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:49 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:49.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:49 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:49.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:49 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:49.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:49 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:49.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:49 smithi150 ceph-mon[32504]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:40:49.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:49 smithi150 ceph-mon[32504]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:40:49.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:49 smithi150 ceph-mon[32504]: Updating smithi053:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:40:49.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:49 smithi150 ceph-mon[32504]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 131 KiB/s wr, 76 op/s 2024-04-25T14:40:49.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:49 smithi150 ceph-mon[32504]: Updating smithi150:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T14:40:49.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:49 smithi150 ceph-mon[32504]: Updating smithi183:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T14:40:49.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:49 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:49.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:49 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:49.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:49 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:49.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:49 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:50.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.094+0000 7fba0d400700 1 -- 172.21.15.53:0/2433010347 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba08074c70 msgr2=0x7fba08075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:50.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.094+0000 7fba0d400700 1 --2- 172.21.15.53:0/2433010347 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba08074c70 0x7fba08075050 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8009990 tx=0x7fb9f8009ca0 comp rx=0 tx=0).stop 2024-04-25T14:40:50.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.094+0000 7fba0d400700 1 -- 172.21.15.53:0/2433010347 shutdown_connections 2024-04-25T14:40:50.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.094+0000 7fba0d400700 1 --2- 172.21.15.53:0/2433010347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba08113c10 0x7fba08116000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:50.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.094+0000 7fba0d400700 1 --2- 172.21.15.53:0/2433010347 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba08074c70 0x7fba08075050 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:50.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.094+0000 7fba0d400700 1 --2- 172.21.15.53:0/2433010347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba08075590 0x7fba081136d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:50.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.094+0000 7fba0d400700 1 -- 172.21.15.53:0/2433010347 >> 172.21.15.53:0/2433010347 conn(0x7fba08079ba0 msgr2=0x7fba08079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:50.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.094+0000 7fba0d400700 1 -- 172.21.15.53:0/2433010347 shutdown_connections 2024-04-25T14:40:50.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.094+0000 7fba0d400700 1 -- 172.21.15.53:0/2433010347 wait complete. 2024-04-25T14:40:50.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.095+0000 7fba0d400700 1 Processor -- start 2024-04-25T14:40:50.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.095+0000 7fba0d400700 1 -- start start 2024-04-25T14:40:50.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.096+0000 7fba0d400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba08074c70 0x7fba081a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:50.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.096+0000 7fba0d400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba08075590 0x7fba081a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:50.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.096+0000 7fba0d400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba08113c10 0x7fba0819f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:50.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.096+0000 7fba0d400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fba08119210 con 0x7fba08074c70 2024-04-25T14:40:50.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.096+0000 7fba0d400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fba08119090 con 0x7fba08113c10 2024-04-25T14:40:50.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.096+0000 7fba0d400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fba08119390 con 0x7fba08075590 2024-04-25T14:40:50.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.096+0000 7fba06800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba08074c70 0x7fba081a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:50.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.096+0000 7fba05e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba08075590 0x7fba081a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:50.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.096+0000 7fba07200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba08113c10 0x7fba0819f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:50.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.096+0000 7fba06800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba08074c70 0x7fba081a5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46136/0 (socket says 172.21.15.53:46136) 2024-04-25T14:40:50.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.096+0000 7fba06800700 1 -- 172.21.15.53:0/1369179632 learned_addr learned my addr 172.21.15.53:0/1369179632 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:50.099 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.097+0000 7fba05e00700 1 -- 172.21.15.53:0/1369179632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba08113c10 msgr2=0x7fba0819f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:50.099 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.097+0000 7fba05e00700 1 --2- 172.21.15.53:0/1369179632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba08113c10 0x7fba0819f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:50.099 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.097+0000 7fba05e00700 1 -- 172.21.15.53:0/1369179632 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba08074c70 msgr2=0x7fba081a5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:50.099 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.097+0000 7fba05e00700 1 --2- 172.21.15.53:0/1369179632 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba08074c70 0x7fba081a5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:50.099 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.097+0000 7fba05e00700 1 -- 172.21.15.53:0/1369179632 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba0819fd50 con 0x7fba08075590 2024-04-25T14:40:50.099 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.097+0000 7fba05e00700 1 --2- 172.21.15.53:0/1369179632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba08075590 0x7fba081a5970 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb9f400e9f0 tx=0x7fb9f400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:50.099 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.097+0000 7fb9ff400700 1 -- 172.21.15.53:0/1369179632 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f400cb70 con 0x7fba08075590 2024-04-25T14:40:50.099 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.098+0000 7fb9ff400700 1 -- 172.21.15.53:0/1369179632 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9f400ee90 con 0x7fba08075590 2024-04-25T14:40:50.099 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.098+0000 7fb9ff400700 1 -- 172.21.15.53:0/1369179632 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f4003d40 con 0x7fba08075590 2024-04-25T14:40:50.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.098+0000 7fba0d400700 1 -- 172.21.15.53:0/1369179632 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba081a0040 con 0x7fba08075590 2024-04-25T14:40:50.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.098+0000 7fba0d400700 1 -- 172.21.15.53:0/1369179632 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba081ac300 con 0x7fba08075590 2024-04-25T14:40:50.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.099+0000 7fba0d400700 1 -- 172.21.15.53:0/1369179632 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba08069730 con 0x7fba08075590 2024-04-25T14:40:50.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.101+0000 7fb9ff400700 1 -- 172.21.15.53:0/1369179632 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7fb9f4003ee0 con 0x7fba08075590 2024-04-25T14:40:50.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.103+0000 7fb9ff400700 1 --2- 172.21.15.53:0/1369179632 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fb9ec0a3ec0 0x7fb9ec0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:50.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.103+0000 7fb9ff400700 1 -- 172.21.15.53:0/1369179632 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8543+0+0 (secure 0 0 0) 0x7fb9f40d0cb0 con 0x7fba08075590 2024-04-25T14:40:50.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.103+0000 7fba06800700 1 --2- 172.21.15.53:0/1369179632 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fb9ec0a3ec0 0x7fb9ec0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:50.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.104+0000 7fba06800700 1 --2- 172.21.15.53:0/1369179632 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fb9ec0a3ec0 0x7fb9ec0a6380 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8005cd0 tx=0x7fb9f8005c60 comp rx=0 tx=0).ready entity=mgr.15093 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:50.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.105+0000 7fb9ff400700 1 -- 172.21.15.53:0/1369179632 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb9f4053ff0 con 0x7fba08075590 2024-04-25T14:40:50.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.342+0000 7fba0d400700 1 -- 172.21.15.53:0/1369179632 --> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fba08113150 con 0x7fb9ec0a3ec0 2024-04-25T14:40:50.352 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.351+0000 7fb9ff400700 1 -- 172.21.15.53:0/1369179632 <== mgr.15093 v2:172.21.15.53:6800/802683714 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257770 (secure 0 0 0) 0x7fba08113150 con 0x7fb9ec0a3ec0 2024-04-25T14:40:50.354 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:40:50.359 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.357+0000 7fba0d400700 1 -- 172.21.15.53:0/1369179632 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fb9ec0a3ec0 msgr2=0x7fb9ec0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:50.359 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.357+0000 7fba0d400700 1 --2- 172.21.15.53:0/1369179632 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fb9ec0a3ec0 0x7fb9ec0a6380 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8005cd0 tx=0x7fb9f8005c60 comp rx=0 tx=0).stop 2024-04-25T14:40:50.359 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.357+0000 7fba0d400700 1 -- 172.21.15.53:0/1369179632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba08075590 msgr2=0x7fba081a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:50.359 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.357+0000 7fba0d400700 1 --2- 172.21.15.53:0/1369179632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba08075590 0x7fba081a5970 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb9f400e9f0 tx=0x7fb9f400ed00 comp rx=0 tx=0).stop 2024-04-25T14:40:50.359 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.357+0000 7fba0d400700 1 -- 172.21.15.53:0/1369179632 shutdown_connections 2024-04-25T14:40:50.359 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.357+0000 7fba0d400700 1 --2- 172.21.15.53:0/1369179632 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fb9ec0a3ec0 0x7fb9ec0a6380 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:50.359 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.357+0000 7fba0d400700 1 --2- 172.21.15.53:0/1369179632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba08113c10 0x7fba0819f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:50.359 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.357+0000 7fba0d400700 1 --2- 172.21.15.53:0/1369179632 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba08074c70 0x7fba081a5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:50.359 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.357+0000 7fba0d400700 1 --2- 172.21.15.53:0/1369179632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba08075590 0x7fba081a5970 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:50.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.357+0000 7fba0d400700 1 -- 172.21.15.53:0/1369179632 >> 172.21.15.53:0/1369179632 conn(0x7fba08079ba0 msgr2=0x7fba08112a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:50.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.358+0000 7fba0d400700 1 -- 172.21.15.53:0/1369179632 shutdown_connections 2024-04-25T14:40:50.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:50.358+0000 7fba0d400700 1 -- 172.21.15.53:0/1369179632 wait complete. 2024-04-25T14:40:50.360 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-04-25T14:40:50.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:50 smithi053 ceph-mon[27773]: Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:40:50.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:50 smithi053 ceph-mon[27773]: Updating smithi053:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T14:40:50.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:50 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:50.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:50 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:50.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:50 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:50.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:50 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:40:50.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:50 smithi183 ceph-mon[32584]: Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:40:50.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:50 smithi183 ceph-mon[32584]: Updating smithi053:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T14:40:50.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:50 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:50.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:50 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:50.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:50 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:50.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:50 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:40:50.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:50 smithi150 ceph-mon[32504]: Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:40:50.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:50 smithi150 ceph-mon[32504]: Updating smithi053:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T14:40:50.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:50 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:50.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:50 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:50.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:50 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:50.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:50 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:40:51.352 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-04-25T14:40:50.199136+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1386426,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1521,"num_read_kb":1189,"num_write":623,"num_write_kb":5296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1181771,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":960,"ondisk_log_size":960,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":386708,"kb_used_data":16368,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124555116,"statfs":{"total":1151940427776,"available":1151544438784,"internally_reserved":0,"allocated":16760832,"data_stored":10100742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19137,"internal_metadata":378713407},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":384,"num_read_kb":456,"num_write":146,"num_write_kb":914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.972382"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.651108+0000","last_change":"2024-04-25T14:38:10.435535+0000","last_active":"2024-04-25T14:40:40.651108+0000","last_peered":"2024-04-25T14:40:40.651108+0000","last_clean":"2024-04-25T14:40:40.651108+0000","last_became_active":"2024-04-25T14:38:10.435205+0000","last_became_peered":"2024-04-25T14:38:10.435205+0000","last_unstale":"2024-04-25T14:40:40.651108+0000","last_undegraded":"2024-04-25T14:40:40.651108+0000","last_fullsized":"2024-04-25T14:40:40.651108+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:12.301771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.991215+0000","last_change":"2024-04-25T14:38:10.428216+0000","last_active":"2024-04-25T14:40:40.991215+0000","last_peered":"2024-04-25T14:40:40.991215+0000","last_clean":"2024-04-25T14:40:40.991215+0000","last_became_active":"2024-04-25T14:38:10.427419+0000","last_became_peered":"2024-04-25T14:38:10.427419+0000","last_unstale":"2024-04-25T14:40:40.991215+0000","last_undegraded":"2024-04-25T14:40:40.991215+0000","last_fullsized":"2024-04-25T14:40:40.991215+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:42.580424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.991464+0000","last_change":"2024-04-25T14:38:10.427827+0000","last_active":"2024-04-25T14:40:40.991464+0000","last_peered":"2024-04-25T14:40:40.991464+0000","last_clean":"2024-04-25T14:40:40.991464+0000","last_became_active":"2024-04-25T14:38:10.427435+0000","last_became_peered":"2024-04-25T14:38:10.427435+0000","last_unstale":"2024-04-25T14:40:40.991464+0000","last_undegraded":"2024-04-25T14:40:40.991464+0000","last_fullsized":"2024-04-25T14:40:40.991464+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:15.539399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.481932+0000","last_change":"2024-04-25T14:38:10.423751+0000","last_active":"2024-04-25T14:40:40.481932+0000","last_peered":"2024-04-25T14:40:40.481932+0000","last_clean":"2024-04-25T14:40:40.481932+0000","last_became_active":"2024-04-25T14:38:10.423409+0000","last_became_peered":"2024-04-25T14:38:10.423409+0000","last_unstale":"2024-04-25T14:40:40.481932+0000","last_undegraded":"2024-04-25T14:40:40.481932+0000","last_fullsized":"2024-04-25T14:40:40.481932+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:45.403788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.304337+0000","last_change":"2024-04-25T14:38:10.430000+0000","last_active":"2024-04-25T14:40:40.304337+0000","last_peered":"2024-04-25T14:40:40.304337+0000","last_clean":"2024-04-25T14:40:40.304337+0000","last_became_active":"2024-04-25T14:38:10.429684+0000","last_became_peered":"2024-04-25T14:38:10.429684+0000","last_unstale":"2024-04-25T14:40:40.304337+0000","last_undegraded":"2024-04-25T14:40:40.304337+0000","last_fullsized":"2024-04-25T14:40:40.304337+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:18.421970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.179542+0000","last_change":"2024-04-25T14:39:15.984180+0000","last_active":"2024-04-25T14:40:40.179542+0000","last_peered":"2024-04-25T14:40:40.179542+0000","last_clean":"2024-04-25T14:40:40.179542+0000","last_became_active":"2024-04-25T14:39:15.983821+0000","last_became_peered":"2024-04-25T14:39:15.983821+0000","last_unstale":"2024-04-25T14:40:40.179542+0000","last_undegraded":"2024-04-25T14:40:40.179542+0000","last_fullsized":"2024-04-25T14:40:40.179542+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:54:47.617713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.482156+0000","last_change":"2024-04-25T14:38:10.420022+0000","last_active":"2024-04-25T14:40:40.482156+0000","last_peered":"2024-04-25T14:40:40.482156+0000","last_clean":"2024-04-25T14:40:40.482156+0000","last_became_active":"2024-04-25T14:38:10.419673+0000","last_became_peered":"2024-04-25T14:38:10.419673+0000","last_unstale":"2024-04-25T14:40:40.482156+0000","last_undegraded":"2024-04-25T14:40:40.482156+0000","last_fullsized":"2024-04-25T14:40:40.482156+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:09.177814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.281442+0000","last_change":"2024-04-25T14:40:16.499255+0000","last_active":"2024-04-25T14:40:40.281442+0000","last_peered":"2024-04-25T14:40:40.281442+0000","last_clean":"2024-04-25T14:40:40.281442+0000","last_became_active":"2024-04-25T14:40:16.497912+0000","last_became_peered":"2024-04-25T14:40:16.497912+0000","last_unstale":"2024-04-25T14:40:40.281442+0000","last_undegraded":"2024-04-25T14:40:40.281442+0000","last_fullsized":"2024-04-25T14:40:40.281442+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:10.595053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,10],"acting":[9,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.224740+0000","last_change":"2024-04-25T14:38:10.424086+0000","last_active":"2024-04-25T14:40:40.224740+0000","last_peered":"2024-04-25T14:40:40.224740+0000","last_clean":"2024-04-25T14:40:40.224740+0000","last_became_active":"2024-04-25T14:38:10.423769+0000","last_became_peered":"2024-04-25T14:38:10.423769+0000","last_unstale":"2024-04-25T14:40:40.224740+0000","last_undegraded":"2024-04-25T14:40:40.224740+0000","last_fullsized":"2024-04-25T14:40:40.224740+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:19:22.736053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.180126+0000","last_change":"2024-04-25T14:38:10.427049+0000","last_active":"2024-04-25T14:40:40.180126+0000","last_peered":"2024-04-25T14:40:40.180126+0000","last_clean":"2024-04-25T14:40:40.180126+0000","last_became_active":"2024-04-25T14:38:10.426729+0000","last_became_peered":"2024-04-25T14:38:10.426729+0000","last_unstale":"2024-04-25T14:40:40.180126+0000","last_undegraded":"2024-04-25T14:40:40.180126+0000","last_fullsized":"2024-04-25T14:40:40.180126+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:00.849650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.482804+0000","last_change":"2024-04-25T14:38:10.424976+0000","last_active":"2024-04-25T14:40:40.482804+0000","last_peered":"2024-04-25T14:40:40.482804+0000","last_clean":"2024-04-25T14:40:40.482804+0000","last_became_active":"2024-04-25T14:38:10.424585+0000","last_became_peered":"2024-04-25T14:38:10.424585+0000","last_unstale":"2024-04-25T14:40:40.482804+0000","last_undegraded":"2024-04-25T14:40:40.482804+0000","last_fullsized":"2024-04-25T14:40:40.482804+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:54.639471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.481912+0000","last_change":"2024-04-25T14:38:10.418022+0000","last_active":"2024-04-25T14:40:40.481912+0000","last_peered":"2024-04-25T14:40:40.481912+0000","last_clean":"2024-04-25T14:40:40.481912+0000","last_became_active":"2024-04-25T14:38:10.417735+0000","last_became_peered":"2024-04-25T14:38:10.417735+0000","last_unstale":"2024-04-25T14:40:40.481912+0000","last_undegraded":"2024-04-25T14:40:40.481912+0000","last_fullsized":"2024-04-25T14:40:40.481912+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:48.785521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.677401+0000","last_change":"2024-04-25T14:38:10.423464+0000","last_active":"2024-04-25T14:40:40.677401+0000","last_peered":"2024-04-25T14:40:40.677401+0000","last_clean":"2024-04-25T14:40:40.677401+0000","last_became_active":"2024-04-25T14:38:10.423105+0000","last_became_peered":"2024-04-25T14:38:10.423105+0000","last_unstale":"2024-04-25T14:40:40.677401+0000","last_undegraded":"2024-04-25T14:40:40.677401+0000","last_fullsized":"2024-04-25T14:40:40.677401+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:26.568198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.991106+0000","last_change":"2024-04-25T14:38:10.426870+0000","last_active":"2024-04-25T14:40:40.991106+0000","last_peered":"2024-04-25T14:40:40.991106+0000","last_clean":"2024-04-25T14:40:40.991106+0000","last_became_active":"2024-04-25T14:38:10.426506+0000","last_became_peered":"2024-04-25T14:38:10.426506+0000","last_unstale":"2024-04-25T14:40:40.991106+0000","last_undegraded":"2024-04-25T14:40:40.991106+0000","last_fullsized":"2024-04-25T14:40:40.991106+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:16:49.539221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.990805+0000","last_change":"2024-04-25T14:38:10.434618+0000","last_active":"2024-04-25T14:40:40.990805+0000","last_peered":"2024-04-25T14:40:40.990805+0000","last_clean":"2024-04-25T14:40:40.990805+0000","last_became_active":"2024-04-25T14:38:10.430661+0000","last_became_peered":"2024-04-25T14:38:10.430661+0000","last_unstale":"2024-04-25T14:40:40.990805+0000","last_undegraded":"2024-04-25T14:40:40.990805+0000","last_fullsized":"2024-04-25T14:40:40.990805+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:44.548788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.224125+0000","last_change":"2024-04-25T14:38:10.429898+0000","last_active":"2024-04-25T14:40:40.224125+0000","last_peered":"2024-04-25T14:40:40.224125+0000","last_clean":"2024-04-25T14:40:40.224125+0000","last_became_active":"2024-04-25T14:38:10.429557+0000","last_became_peered":"2024-04-25T14:38:10.429557+0000","last_unstale":"2024-04-25T14:40:40.224125+0000","last_undegraded":"2024-04-25T14:40:40.224125+0000","last_fullsized":"2024-04-25T14:40:40.224125+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:54.258526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.651158+0000","last_change":"2024-04-25T14:38:10.435134+0000","last_active":"2024-04-25T14:40:40.651158+0000","last_peered":"2024-04-25T14:40:40.651158+0000","last_clean":"2024-04-25T14:40:40.651158+0000","last_became_active":"2024-04-25T14:38:10.434775+0000","last_became_peered":"2024-04-25T14:38:10.434775+0000","last_unstale":"2024-04-25T14:40:40.651158+0000","last_undegraded":"2024-04-25T14:40:40.651158+0000","last_fullsized":"2024-04-25T14:40:40.651158+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:49.827881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.282019+0000","last_change":"2024-04-25T14:38:10.424929+0000","last_active":"2024-04-25T14:40:40.282019+0000","last_peered":"2024-04-25T14:40:40.282019+0000","last_clean":"2024-04-25T14:40:40.282019+0000","last_became_active":"2024-04-25T14:38:10.424399+0000","last_became_peered":"2024-04-25T14:38:10.424399+0000","last_unstale":"2024-04-25T14:40:40.282019+0000","last_undegraded":"2024-04-25T14:40:40.282019+0000","last_fullsized":"2024-04-25T14:40:40.282019+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:08.834176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.224851+0000","last_change":"2024-04-25T14:38:10.444004+0000","last_active":"2024-04-25T14:40:40.224851+0000","last_peered":"2024-04-25T14:40:40.224851+0000","last_clean":"2024-04-25T14:40:40.224851+0000","last_became_active":"2024-04-25T14:38:10.443669+0000","last_became_peered":"2024-04-25T14:38:10.443669+0000","last_unstale":"2024-04-25T14:40:40.224851+0000","last_undegraded":"2024-04-25T14:40:40.224851+0000","last_fullsized":"2024-04-25T14:40:40.224851+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:02.302057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.281993+0000","last_change":"2024-04-25T14:38:09.545010+0000","last_active":"2024-04-25T14:40:40.281993+0000","last_peered":"2024-04-25T14:40:40.281993+0000","last_clean":"2024-04-25T14:40:40.281993+0000","last_became_active":"2024-04-25T14:38:09.544622+0000","last_became_peered":"2024-04-25T14:38:09.544622+0000","last_unstale":"2024-04-25T14:40:40.281993+0000","last_undegraded":"2024-04-25T14:40:40.281993+0000","last_fullsized":"2024-04-25T14:40:40.281993+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:51.200242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.281678+0000","last_change":"2024-04-25T14:39:15.993912+0000","last_active":"2024-04-25T14:40:40.281678+0000","last_peered":"2024-04-25T14:40:40.281678+0000","last_clean":"2024-04-25T14:40:40.281678+0000","last_became_active":"2024-04-25T14:39:15.992511+0000","last_became_peered":"2024-04-25T14:39:15.992511+0000","last_unstale":"2024-04-25T14:40:40.281678+0000","last_undegraded":"2024-04-25T14:40:40.281678+0000","last_fullsized":"2024-04-25T14:40:40.281678+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:12:57.639396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,6],"acting":[9,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.184989+0000","last_change":"2024-04-25T14:38:09.555825+0000","last_active":"2024-04-25T14:40:40.184989+0000","last_peered":"2024-04-25T14:40:40.184989+0000","last_clean":"2024-04-25T14:40:40.184989+0000","last_became_active":"2024-04-25T14:38:09.555436+0000","last_became_peered":"2024-04-25T14:38:09.555436+0000","last_unstale":"2024-04-25T14:40:40.184989+0000","last_undegraded":"2024-04-25T14:40:40.184989+0000","last_fullsized":"2024-04-25T14:40:40.184989+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:45.097583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.991388+0000","last_change":"2024-04-25T14:38:10.432254+0000","last_active":"2024-04-25T14:40:40.991388+0000","last_peered":"2024-04-25T14:40:40.991388+0000","last_clean":"2024-04-25T14:40:40.991388+0000","last_became_active":"2024-04-25T14:38:10.431835+0000","last_became_peered":"2024-04-25T14:38:10.431835+0000","last_unstale":"2024-04-25T14:40:40.991388+0000","last_undegraded":"2024-04-25T14:40:40.991388+0000","last_fullsized":"2024-04-25T14:40:40.991388+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:23.089447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.304545+0000","last_change":"2024-04-25T14:38:09.550254+0000","last_active":"2024-04-25T14:40:40.304545+0000","last_peered":"2024-04-25T14:40:40.304545+0000","last_clean":"2024-04-25T14:40:40.304545+0000","last_became_active":"2024-04-25T14:38:09.549958+0000","last_became_peered":"2024-04-25T14:38:09.549958+0000","last_unstale":"2024-04-25T14:40:40.304545+0000","last_undegraded":"2024-04-25T14:40:40.304545+0000","last_fullsized":"2024-04-25T14:40:40.304545+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:56.161427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.224323+0000","last_change":"2024-04-25T14:38:10.425648+0000","last_active":"2024-04-25T14:40:40.224323+0000","last_peered":"2024-04-25T14:40:40.224323+0000","last_clean":"2024-04-25T14:40:40.224323+0000","last_became_active":"2024-04-25T14:38:10.425009+0000","last_became_peered":"2024-04-25T14:38:10.425009+0000","last_unstale":"2024-04-25T14:40:40.224323+0000","last_undegraded":"2024-04-25T14:40:40.224323+0000","last_fullsized":"2024-04-25T14:40:40.224323+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:46.696610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.304408+0000","last_change":"2024-04-25T14:38:09.544481+0000","last_active":"2024-04-25T14:40:40.304408+0000","last_peered":"2024-04-25T14:40:40.304408+0000","last_clean":"2024-04-25T14:40:40.304408+0000","last_became_active":"2024-04-25T14:38:09.544147+0000","last_became_peered":"2024-04-25T14:38:09.544147+0000","last_unstale":"2024-04-25T14:40:40.304408+0000","last_undegraded":"2024-04-25T14:40:40.304408+0000","last_fullsized":"2024-04-25T14:40:40.304408+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:35:22.971028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.180558+0000","last_change":"2024-04-25T14:38:10.426851+0000","last_active":"2024-04-25T14:40:40.180558+0000","last_peered":"2024-04-25T14:40:40.180558+0000","last_clean":"2024-04-25T14:40:40.180558+0000","last_became_active":"2024-04-25T14:38:10.426475+0000","last_became_peered":"2024-04-25T14:38:10.426475+0000","last_unstale":"2024-04-25T14:40:40.180558+0000","last_undegraded":"2024-04-25T14:40:40.180558+0000","last_fullsized":"2024-04-25T14:40:40.180558+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:28.213318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"93'8","reported_seq":202,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:45.109559+0000","last_change":"2024-04-25T14:39:15.979586+0000","last_active":"2024-04-25T14:40:45.109559+0000","last_peered":"2024-04-25T14:40:45.109559+0000","last_clean":"2024-04-25T14:40:45.109559+0000","last_became_active":"2024-04-25T14:39:15.974397+0000","last_became_peered":"2024-04-25T14:39:15.974397+0000","last_unstale":"2024-04-25T14:40:45.109559+0000","last_undegraded":"2024-04-25T14:40:45.109559+0000","last_fullsized":"2024-04-25T14:40:45.109559+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:02.152202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":154,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.304385+0000","last_change":"2024-04-25T14:38:10.425424+0000","last_active":"2024-04-25T14:40:40.304385+0000","last_peered":"2024-04-25T14:40:40.304385+0000","last_clean":"2024-04-25T14:40:40.304385+0000","last_became_active":"2024-04-25T14:38:10.425101+0000","last_became_peered":"2024-04-25T14:38:10.425101+0000","last_unstale":"2024-04-25T14:40:40.304385+0000","last_undegraded":"2024-04-25T14:40:40.304385+0000","last_fullsized":"2024-04-25T14:40:40.304385+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:52:29.871113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.179499+0000","last_change":"2024-04-25T14:38:09.552287+0000","last_active":"2024-04-25T14:40:40.179499+0000","last_peered":"2024-04-25T14:40:40.179499+0000","last_clean":"2024-04-25T14:40:40.179499+0000","last_became_active":"2024-04-25T14:38:09.551957+0000","last_became_peered":"2024-04-25T14:38:09.551957+0000","last_unstale":"2024-04-25T14:40:40.179499+0000","last_undegraded":"2024-04-25T14:40:40.179499+0000","last_fullsized":"2024-04-25T14:40:40.179499+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:22.586868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.180162+0000","last_change":"2024-04-25T14:39:15.990838+0000","last_active":"2024-04-25T14:40:40.180162+0000","last_peered":"2024-04-25T14:40:40.180162+0000","last_clean":"2024-04-25T14:40:40.180162+0000","last_became_active":"2024-04-25T14:39:15.990370+0000","last_became_peered":"2024-04-25T14:39:15.990370+0000","last_unstale":"2024-04-25T14:40:40.180162+0000","last_undegraded":"2024-04-25T14:40:40.180162+0000","last_fullsized":"2024-04-25T14:40:40.180162+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:50.676958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.676465+0000","last_change":"2024-04-25T14:38:09.552376+0000","last_active":"2024-04-25T14:40:40.676465+0000","last_peered":"2024-04-25T14:40:40.676465+0000","last_clean":"2024-04-25T14:40:40.676465+0000","last_became_active":"2024-04-25T14:38:09.548614+0000","last_became_peered":"2024-04-25T14:38:09.548614+0000","last_unstale":"2024-04-25T14:40:40.676465+0000","last_undegraded":"2024-04-25T14:40:40.676465+0000","last_fullsized":"2024-04-25T14:40:40.676465+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:27:27.133106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.181241+0000","last_change":"2024-04-25T14:38:10.434611+0000","last_active":"2024-04-25T14:40:40.181241+0000","last_peered":"2024-04-25T14:40:40.181241+0000","last_clean":"2024-04-25T14:40:40.181241+0000","last_became_active":"2024-04-25T14:38:10.434280+0000","last_became_peered":"2024-04-25T14:38:10.434280+0000","last_unstale":"2024-04-25T14:40:40.181241+0000","last_undegraded":"2024-04-25T14:40:40.181241+0000","last_fullsized":"2024-04-25T14:40:40.181241+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:12:32.275663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.179937+0000","last_change":"2024-04-25T14:38:09.539743+0000","last_active":"2024-04-25T14:40:40.179937+0000","last_peered":"2024-04-25T14:40:40.179937+0000","last_clean":"2024-04-25T14:40:40.179937+0000","last_became_active":"2024-04-25T14:38:09.539272+0000","last_became_peered":"2024-04-25T14:38:09.539272+0000","last_unstale":"2024-04-25T14:40:40.179937+0000","last_undegraded":"2024-04-25T14:40:40.179937+0000","last_fullsized":"2024-04-25T14:40:40.179937+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:29:23.159901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.282536+0000","last_change":"2024-04-25T14:38:10.444899+0000","last_active":"2024-04-25T14:40:40.282536+0000","last_peered":"2024-04-25T14:40:40.282536+0000","last_clean":"2024-04-25T14:40:40.282536+0000","last_became_active":"2024-04-25T14:38:10.444383+0000","last_became_peered":"2024-04-25T14:38:10.444383+0000","last_unstale":"2024-04-25T14:40:40.282536+0000","last_undegraded":"2024-04-25T14:40:40.282536+0000","last_fullsized":"2024-04-25T14:40:40.282536+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:21.351436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"93'5","reported_seq":182,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:45.108956+0000","last_change":"2024-04-25T14:38:09.550181+0000","last_active":"2024-04-25T14:40:45.108956+0000","last_peered":"2024-04-25T14:40:45.108956+0000","last_clean":"2024-04-25T14:40:45.108956+0000","last_became_active":"2024-04-25T14:38:09.549824+0000","last_became_peered":"2024-04-25T14:38:09.549824+0000","last_unstale":"2024-04-25T14:40:45.108956+0000","last_undegraded":"2024-04-25T14:40:45.108956+0000","last_fullsized":"2024-04-25T14:40:45.108956+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:59.220917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":151,"num_read_kb":151,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.224700+0000","last_change":"2024-04-25T14:38:10.421339+0000","last_active":"2024-04-25T14:40:40.224700+0000","last_peered":"2024-04-25T14:40:40.224700+0000","last_clean":"2024-04-25T14:40:40.224700+0000","last_became_active":"2024-04-25T14:38:10.421005+0000","last_became_peered":"2024-04-25T14:38:10.421005+0000","last_unstale":"2024-04-25T14:40:40.224700+0000","last_undegraded":"2024-04-25T14:40:40.224700+0000","last_fullsized":"2024-04-25T14:40:40.224700+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:51.949054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.482920+0000","last_change":"2024-04-25T14:38:09.548245+0000","last_active":"2024-04-25T14:40:40.482920+0000","last_peered":"2024-04-25T14:40:40.482920+0000","last_clean":"2024-04-25T14:40:40.482920+0000","last_became_active":"2024-04-25T14:38:09.547905+0000","last_became_peered":"2024-04-25T14:38:09.547905+0000","last_unstale":"2024-04-25T14:40:40.482920+0000","last_undegraded":"2024-04-25T14:40:40.482920+0000","last_fullsized":"2024-04-25T14:40:40.482920+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:56.798594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.991226+0000","last_change":"2024-04-25T14:38:10.431427+0000","last_active":"2024-04-25T14:40:40.991226+0000","last_peered":"2024-04-25T14:40:40.991226+0000","last_clean":"2024-04-25T14:40:40.991226+0000","last_became_active":"2024-04-25T14:38:10.428759+0000","last_became_peered":"2024-04-25T14:38:10.428759+0000","last_unstale":"2024-04-25T14:40:40.991226+0000","last_undegraded":"2024-04-25T14:40:40.991226+0000","last_fullsized":"2024-04-25T14:40:40.991226+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:15:27.341445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.180647+0000","last_change":"2024-04-25T14:38:09.556839+0000","last_active":"2024-04-25T14:40:40.180647+0000","last_peered":"2024-04-25T14:40:40.180647+0000","last_clean":"2024-04-25T14:40:40.180647+0000","last_became_active":"2024-04-25T14:38:09.556443+0000","last_became_peered":"2024-04-25T14:38:09.556443+0000","last_unstale":"2024-04-25T14:40:40.180647+0000","last_undegraded":"2024-04-25T14:40:40.180647+0000","last_fullsized":"2024-04-25T14:40:40.180647+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:46.621883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.224407+0000","last_change":"2024-04-25T14:38:10.414507+0000","last_active":"2024-04-25T14:40:40.224407+0000","last_peered":"2024-04-25T14:40:40.224407+0000","last_clean":"2024-04-25T14:40:40.224407+0000","last_became_active":"2024-04-25T14:38:10.414154+0000","last_became_peered":"2024-04-25T14:38:10.414154+0000","last_unstale":"2024-04-25T14:40:40.224407+0000","last_undegraded":"2024-04-25T14:40:40.224407+0000","last_fullsized":"2024-04-25T14:40:40.224407+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:12.208774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"93'36","reported_seq":375,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:46.110755+0000","last_change":"2024-04-25T14:38:09.549239+0000","last_active":"2024-04-25T14:40:46.110755+0000","last_peered":"2024-04-25T14:40:46.110755+0000","last_clean":"2024-04-25T14:40:46.110755+0000","last_became_active":"2024-04-25T14:38:09.548888+0000","last_became_peered":"2024-04-25T14:38:09.548888+0000","last_unstale":"2024-04-25T14:40:46.110755+0000","last_undegraded":"2024-04-25T14:40:46.110755+0000","last_fullsized":"2024-04-25T14:40:46.110755+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:36:46.536908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68610,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":313,"num_read_kb":224,"num_write":37,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.304392+0000","last_change":"2024-04-25T14:38:10.417510+0000","last_active":"2024-04-25T14:40:40.304392+0000","last_peered":"2024-04-25T14:40:40.304392+0000","last_clean":"2024-04-25T14:40:40.304392+0000","last_became_active":"2024-04-25T14:38:10.417079+0000","last_became_peered":"2024-04-25T14:38:10.417079+0000","last_unstale":"2024-04-25T14:40:40.304392+0000","last_undegraded":"2024-04-25T14:40:40.304392+0000","last_fullsized":"2024-04-25T14:40:40.304392+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:35.213199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.224809+0000","last_change":"2024-04-25T14:38:09.563650+0000","last_active":"2024-04-25T14:40:40.224809+0000","last_peered":"2024-04-25T14:40:40.224809+0000","last_clean":"2024-04-25T14:40:40.224809+0000","last_became_active":"2024-04-25T14:38:09.563347+0000","last_became_peered":"2024-04-25T14:38:09.563347+0000","last_unstale":"2024-04-25T14:40:40.224809+0000","last_undegraded":"2024-04-25T14:40:40.224809+0000","last_fullsized":"2024-04-25T14:40:40.224809+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:16.793191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.676501+0000","last_change":"2024-04-25T14:38:10.447504+0000","last_active":"2024-04-25T14:40:40.676501+0000","last_peered":"2024-04-25T14:40:40.676501+0000","last_clean":"2024-04-25T14:40:40.676501+0000","last_became_active":"2024-04-25T14:38:10.447157+0000","last_became_peered":"2024-04-25T14:38:10.447157+0000","last_unstale":"2024-04-25T14:40:40.676501+0000","last_undegraded":"2024-04-25T14:40:40.676501+0000","last_fullsized":"2024-04-25T14:40:40.676501+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:38.940320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.180450+0000","last_change":"2024-04-25T14:38:09.545214+0000","last_active":"2024-04-25T14:40:40.180450+0000","last_peered":"2024-04-25T14:40:40.180450+0000","last_clean":"2024-04-25T14:40:40.180450+0000","last_became_active":"2024-04-25T14:38:09.544903+0000","last_became_peered":"2024-04-25T14:38:09.544903+0000","last_unstale":"2024-04-25T14:40:40.180450+0000","last_undegraded":"2024-04-25T14:40:40.180450+0000","last_fullsized":"2024-04-25T14:40:40.180450+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:58:15.817036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.482678+0000","last_change":"2024-04-25T14:38:10.432017+0000","last_active":"2024-04-25T14:40:40.482678+0000","last_peered":"2024-04-25T14:40:40.482678+0000","last_clean":"2024-04-25T14:40:40.482678+0000","last_became_active":"2024-04-25T14:38:10.431596+0000","last_became_peered":"2024-04-25T14:38:10.431596+0000","last_unstale":"2024-04-25T14:40:40.482678+0000","last_undegraded":"2024-04-25T14:40:40.482678+0000","last_fullsized":"2024-04-25T14:40:40.482678+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:30.653642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"93'7","reported_seq":190,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:44.978291+0000","last_change":"2024-04-25T14:38:09.545477+0000","last_active":"2024-04-25T14:40:44.978291+0000","last_peered":"2024-04-25T14:40:44.978291+0000","last_clean":"2024-04-25T14:40:44.978291+0000","last_became_active":"2024-04-25T14:38:09.545093+0000","last_became_peered":"2024-04-25T14:38:09.545093+0000","last_unstale":"2024-04-25T14:40:44.978291+0000","last_undegraded":"2024-04-25T14:40:44.978291+0000","last_fullsized":"2024-04-25T14:40:44.978291+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:56.589231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1821,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":158,"num_read_kb":7,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.481697+0000","last_change":"2024-04-25T14:38:10.418914+0000","last_active":"2024-04-25T14:40:40.481697+0000","last_peered":"2024-04-25T14:40:40.481697+0000","last_clean":"2024-04-25T14:40:40.481697+0000","last_became_active":"2024-04-25T14:38:10.418600+0000","last_became_peered":"2024-04-25T14:38:10.418600+0000","last_unstale":"2024-04-25T14:40:40.481697+0000","last_undegraded":"2024-04-25T14:40:40.481697+0000","last_fullsized":"2024-04-25T14:40:40.481697+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:31.324035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.304080+0000","last_change":"2024-04-25T14:38:09.556574+0000","last_active":"2024-04-25T14:40:40.304080+0000","last_peered":"2024-04-25T14:40:40.304080+0000","last_clean":"2024-04-25T14:40:40.304080+0000","last_became_active":"2024-04-25T14:38:09.556071+0000","last_became_peered":"2024-04-25T14:38:09.556071+0000","last_unstale":"2024-04-25T14:40:40.304080+0000","last_undegraded":"2024-04-25T14:40:40.304080+0000","last_fullsized":"2024-04-25T14:40:40.304080+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:05.424593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.180084+0000","last_change":"2024-04-25T14:38:10.424591+0000","last_active":"2024-04-25T14:40:40.180084+0000","last_peered":"2024-04-25T14:40:40.180084+0000","last_clean":"2024-04-25T14:40:40.180084+0000","last_became_active":"2024-04-25T14:38:10.424258+0000","last_became_peered":"2024-04-25T14:38:10.424258+0000","last_unstale":"2024-04-25T14:40:40.180084+0000","last_undegraded":"2024-04-25T14:40:40.180084+0000","last_fullsized":"2024-04-25T14:40:40.180084+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:22.753595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"93'3","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.304606+0000","last_change":"2024-04-25T14:38:09.546073+0000","last_active":"2024-04-25T14:40:40.304606+0000","last_peered":"2024-04-25T14:40:40.304606+0000","last_clean":"2024-04-25T14:40:40.304606+0000","last_became_active":"2024-04-25T14:38:09.545783+0000","last_became_peered":"2024-04-25T14:38:09.545783+0000","last_unstale":"2024-04-25T14:40:40.304606+0000","last_undegraded":"2024-04-25T14:40:40.304606+0000","last_fullsized":"2024-04-25T14:40:40.304606+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:42.537431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.650953+0000","last_change":"2024-04-25T14:38:10.417230+0000","last_active":"2024-04-25T14:40:40.650953+0000","last_peered":"2024-04-25T14:40:40.650953+0000","last_clean":"2024-04-25T14:40:40.650953+0000","last_became_active":"2024-04-25T14:38:10.416891+0000","last_became_peered":"2024-04-25T14:38:10.416891+0000","last_unstale":"2024-04-25T14:40:40.650953+0000","last_undegraded":"2024-04-25T14:40:40.650953+0000","last_fullsized":"2024-04-25T14:40:40.650953+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:10:13.434518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.281801+0000","last_change":"2024-04-25T14:39:15.994906+0000","last_active":"2024-04-25T14:40:40.281801+0000","last_peered":"2024-04-25T14:40:40.281801+0000","last_clean":"2024-04-25T14:40:40.281801+0000","last_became_active":"2024-04-25T14:39:15.994365+0000","last_became_peered":"2024-04-25T14:39:15.994365+0000","last_unstale":"2024-04-25T14:40:40.281801+0000","last_undegraded":"2024-04-25T14:40:40.281801+0000","last_fullsized":"2024-04-25T14:40:40.281801+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:19.026241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.179623+0000","last_change":"2024-04-25T14:38:10.446846+0000","last_active":"2024-04-25T14:40:40.179623+0000","last_peered":"2024-04-25T14:40:40.179623+0000","last_clean":"2024-04-25T14:40:40.179623+0000","last_became_active":"2024-04-25T14:38:10.446498+0000","last_became_peered":"2024-04-25T14:38:10.446498+0000","last_unstale":"2024-04-25T14:40:40.179623+0000","last_undegraded":"2024-04-25T14:40:40.179623+0000","last_fullsized":"2024-04-25T14:40:40.179623+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:14:37.179198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.482585+0000","last_change":"2024-04-25T14:38:09.544283+0000","last_active":"2024-04-25T14:40:40.482585+0000","last_peered":"2024-04-25T14:40:40.482585+0000","last_clean":"2024-04-25T14:40:40.482585+0000","last_became_active":"2024-04-25T14:38:09.543135+0000","last_became_peered":"2024-04-25T14:38:09.543135+0000","last_unstale":"2024-04-25T14:40:40.482585+0000","last_undegraded":"2024-04-25T14:40:40.482585+0000","last_fullsized":"2024-04-25T14:40:40.482585+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:05.977062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.179523+0000","last_change":"2024-04-25T14:38:10.432863+0000","last_active":"2024-04-25T14:40:40.179523+0000","last_peered":"2024-04-25T14:40:40.179523+0000","last_clean":"2024-04-25T14:40:40.179523+0000","last_became_active":"2024-04-25T14:38:10.432493+0000","last_became_peered":"2024-04-25T14:38:10.432493+0000","last_unstale":"2024-04-25T14:40:40.179523+0000","last_undegraded":"2024-04-25T14:40:40.179523+0000","last_fullsized":"2024-04-25T14:40:40.179523+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:25:53.774764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.180156+0000","last_change":"2024-04-25T14:39:15.981979+0000","last_active":"2024-04-25T14:40:40.180156+0000","last_peered":"2024-04-25T14:40:40.180156+0000","last_clean":"2024-04-25T14:40:40.180156+0000","last_became_active":"2024-04-25T14:39:15.981539+0000","last_became_peered":"2024-04-25T14:39:15.981539+0000","last_unstale":"2024-04-25T14:40:40.180156+0000","last_undegraded":"2024-04-25T14:40:40.180156+0000","last_fullsized":"2024-04-25T14:40:40.180156+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:22.906826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":183,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:44.977521+0000","last_change":"2024-04-25T14:38:09.554305+0000","last_active":"2024-04-25T14:40:44.977521+0000","last_peered":"2024-04-25T14:40:44.977521+0000","last_clean":"2024-04-25T14:40:44.977521+0000","last_became_active":"2024-04-25T14:38:09.553984+0000","last_became_peered":"2024-04-25T14:38:09.553984+0000","last_unstale":"2024-04-25T14:40:44.977521+0000","last_undegraded":"2024-04-25T14:40:44.977521+0000","last_fullsized":"2024-04-25T14:40:44.977521+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:41.580867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":151,"num_read_kb":151,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.181490+0000","last_change":"2024-04-25T14:38:10.448290+0000","last_active":"2024-04-25T14:40:40.181490+0000","last_peered":"2024-04-25T14:40:40.181490+0000","last_clean":"2024-04-25T14:40:40.181490+0000","last_became_active":"2024-04-25T14:38:10.446147+0000","last_became_peered":"2024-04-25T14:38:10.446147+0000","last_unstale":"2024-04-25T14:40:40.181490+0000","last_undegraded":"2024-04-25T14:40:40.181490+0000","last_fullsized":"2024-04-25T14:40:40.181490+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:12.775551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.481561+0000","last_change":"2024-04-25T14:38:09.545209+0000","last_active":"2024-04-25T14:40:40.481561+0000","last_peered":"2024-04-25T14:40:40.481561+0000","last_clean":"2024-04-25T14:40:40.481561+0000","last_became_active":"2024-04-25T14:38:09.544861+0000","last_became_peered":"2024-04-25T14:38:09.544861+0000","last_unstale":"2024-04-25T14:40:40.481561+0000","last_undegraded":"2024-04-25T14:40:40.481561+0000","last_fullsized":"2024-04-25T14:40:40.481561+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:25.829162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"93'148","reported_seq":404,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:42.316942+0000","last_change":"2024-04-25T14:36:48.250081+0000","last_active":"2024-04-25T14:40:42.316942+0000","last_peered":"2024-04-25T14:40:42.316942+0000","last_clean":"2024-04-25T14:40:42.316942+0000","last_became_active":"2024-04-25T14:36:48.249520+0000","last_became_peered":"2024-04-25T14:36:48.249520+0000","last_unstale":"2024-04-25T14:40:42.316942+0000","last_undegraded":"2024-04-25T14:40:42.316942+0000","last_fullsized":"2024-04-25T14:40:42.316942+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:33:03.061016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:33:03.061016+0000","last_clean_scrub_stamp":"2024-04-25T14:33:03.061016+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:04:06.653326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.676039+0000","last_change":"2024-04-25T14:40:16.498939+0000","last_active":"2024-04-25T14:40:40.676039+0000","last_peered":"2024-04-25T14:40:40.676039+0000","last_clean":"2024-04-25T14:40:40.676039+0000","last_became_active":"2024-04-25T14:40:16.498128+0000","last_became_peered":"2024-04-25T14:40:16.498128+0000","last_unstale":"2024-04-25T14:40:40.676039+0000","last_undegraded":"2024-04-25T14:40:40.676039+0000","last_fullsized":"2024-04-25T14:40:40.676039+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:14.942944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.481930+0000","last_change":"2024-04-25T14:39:15.982120+0000","last_active":"2024-04-25T14:40:40.481930+0000","last_peered":"2024-04-25T14:40:40.481930+0000","last_clean":"2024-04-25T14:40:40.481930+0000","last_became_active":"2024-04-25T14:39:15.975343+0000","last_became_peered":"2024-04-25T14:39:15.975343+0000","last_unstale":"2024-04-25T14:40:40.481930+0000","last_undegraded":"2024-04-25T14:40:40.481930+0000","last_fullsized":"2024-04-25T14:40:40.481930+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:38.632052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.650887+0000","last_change":"2024-04-25T14:39:15.998744+0000","last_active":"2024-04-25T14:40:40.650887+0000","last_peered":"2024-04-25T14:40:40.650887+0000","last_clean":"2024-04-25T14:40:40.650887+0000","last_became_active":"2024-04-25T14:39:15.993416+0000","last_became_peered":"2024-04-25T14:39:15.993416+0000","last_unstale":"2024-04-25T14:40:40.650887+0000","last_undegraded":"2024-04-25T14:40:40.650887+0000","last_fullsized":"2024-04-25T14:40:40.650887+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:24.019707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.676812+0000","last_change":"2024-04-25T14:38:10.429554+0000","last_active":"2024-04-25T14:40:40.676812+0000","last_peered":"2024-04-25T14:40:40.676812+0000","last_clean":"2024-04-25T14:40:40.676812+0000","last_became_active":"2024-04-25T14:38:10.428944+0000","last_became_peered":"2024-04-25T14:38:10.428944+0000","last_unstale":"2024-04-25T14:40:40.676812+0000","last_undegraded":"2024-04-25T14:40:40.676812+0000","last_fullsized":"2024-04-25T14:40:40.676812+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:38.013028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"93'100","reported_seq":218,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:43.323542+0000","last_change":"2024-04-25T14:38:09.564849+0000","last_active":"2024-04-25T14:40:43.323542+0000","last_peered":"2024-04-25T14:40:43.323542+0000","last_clean":"2024-04-25T14:40:43.323542+0000","last_became_active":"2024-04-25T14:38:09.564354+0000","last_became_peered":"2024-04-25T14:38:09.564354+0000","last_unstale":"2024-04-25T14:40:43.323542+0000","last_undegraded":"2024-04-25T14:40:43.323542+0000","last_fullsized":"2024-04-25T14:40:43.323542+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:44.308366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":183,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.181997+0000","last_change":"2024-04-25T14:38:10.438178+0000","last_active":"2024-04-25T14:40:40.181997+0000","last_peered":"2024-04-25T14:40:40.181997+0000","last_clean":"2024-04-25T14:40:40.181997+0000","last_became_active":"2024-04-25T14:38:10.429936+0000","last_became_peered":"2024-04-25T14:38:10.429936+0000","last_unstale":"2024-04-25T14:40:40.181997+0000","last_undegraded":"2024-04-25T14:40:40.181997+0000","last_fullsized":"2024-04-25T14:40:40.181997+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:33.598747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.481640+0000","last_change":"2024-04-25T14:38:09.547680+0000","last_active":"2024-04-25T14:40:40.481640+0000","last_peered":"2024-04-25T14:40:40.481640+0000","last_clean":"2024-04-25T14:40:40.481640+0000","last_became_active":"2024-04-25T14:38:09.547337+0000","last_became_peered":"2024-04-25T14:38:09.547337+0000","last_unstale":"2024-04-25T14:40:40.481640+0000","last_undegraded":"2024-04-25T14:40:40.481640+0000","last_fullsized":"2024-04-25T14:40:40.481640+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:32.754439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.991670+0000","last_change":"2024-04-25T14:38:10.447115+0000","last_active":"2024-04-25T14:40:40.991670+0000","last_peered":"2024-04-25T14:40:40.991670+0000","last_clean":"2024-04-25T14:40:40.991670+0000","last_became_active":"2024-04-25T14:38:10.446681+0000","last_became_peered":"2024-04-25T14:38:10.446681+0000","last_unstale":"2024-04-25T14:40:40.991670+0000","last_undegraded":"2024-04-25T14:40:40.991670+0000","last_fullsized":"2024-04-25T14:40:40.991670+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:02:58.942053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.224061+0000","last_change":"2024-04-25T14:38:09.551523+0000","last_active":"2024-04-25T14:40:40.224061+0000","last_peered":"2024-04-25T14:40:40.224061+0000","last_clean":"2024-04-25T14:40:40.224061+0000","last_became_active":"2024-04-25T14:38:09.551213+0000","last_became_peered":"2024-04-25T14:38:09.551213+0000","last_unstale":"2024-04-25T14:40:40.224061+0000","last_undegraded":"2024-04-25T14:40:40.224061+0000","last_fullsized":"2024-04-25T14:40:40.224061+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:48.948571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.304011+0000","last_change":"2024-04-25T14:38:10.420032+0000","last_active":"2024-04-25T14:40:40.304011+0000","last_peered":"2024-04-25T14:40:40.304011+0000","last_clean":"2024-04-25T14:40:40.304011+0000","last_became_active":"2024-04-25T14:38:10.418349+0000","last_became_peered":"2024-04-25T14:38:10.418349+0000","last_unstale":"2024-04-25T14:40:40.304011+0000","last_undegraded":"2024-04-25T14:40:40.304011+0000","last_fullsized":"2024-04-25T14:40:40.304011+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:38:55.669414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.181847+0000","last_change":"2024-04-25T14:38:09.567097+0000","last_active":"2024-04-25T14:40:40.181847+0000","last_peered":"2024-04-25T14:40:40.181847+0000","last_clean":"2024-04-25T14:40:40.181847+0000","last_became_active":"2024-04-25T14:38:09.566852+0000","last_became_peered":"2024-04-25T14:38:09.566852+0000","last_unstale":"2024-04-25T14:40:40.181847+0000","last_undegraded":"2024-04-25T14:40:40.181847+0000","last_fullsized":"2024-04-25T14:40:40.181847+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:41:55.773005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.650673+0000","last_change":"2024-04-25T14:38:10.432691+0000","last_active":"2024-04-25T14:40:40.650673+0000","last_peered":"2024-04-25T14:40:40.650673+0000","last_clean":"2024-04-25T14:40:40.650673+0000","last_became_active":"2024-04-25T14:38:10.432321+0000","last_became_peered":"2024-04-25T14:38:10.432321+0000","last_unstale":"2024-04-25T14:40:40.650673+0000","last_undegraded":"2024-04-25T14:40:40.650673+0000","last_fullsized":"2024-04-25T14:40:40.650673+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:47:58.026331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.224545+0000","last_change":"2024-04-25T14:38:09.563692+0000","last_active":"2024-04-25T14:40:40.224545+0000","last_peered":"2024-04-25T14:40:40.224545+0000","last_clean":"2024-04-25T14:40:40.224545+0000","last_became_active":"2024-04-25T14:38:09.563345+0000","last_became_peered":"2024-04-25T14:38:09.563345+0000","last_unstale":"2024-04-25T14:40:40.224545+0000","last_undegraded":"2024-04-25T14:40:40.224545+0000","last_fullsized":"2024-04-25T14:40:40.224545+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:43.221429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.182980+0000","last_change":"2024-04-25T14:38:10.442519+0000","last_active":"2024-04-25T14:40:40.182980+0000","last_peered":"2024-04-25T14:40:40.182980+0000","last_clean":"2024-04-25T14:40:40.182980+0000","last_became_active":"2024-04-25T14:38:10.439045+0000","last_became_peered":"2024-04-25T14:38:10.439045+0000","last_unstale":"2024-04-25T14:40:40.182980+0000","last_undegraded":"2024-04-25T14:40:40.182980+0000","last_fullsized":"2024-04-25T14:40:40.182980+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:42.425106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.304120+0000","last_change":"2024-04-25T14:38:09.550363+0000","last_active":"2024-04-25T14:40:40.304120+0000","last_peered":"2024-04-25T14:40:40.304120+0000","last_clean":"2024-04-25T14:40:40.304120+0000","last_became_active":"2024-04-25T14:38:09.550044+0000","last_became_peered":"2024-04-25T14:38:09.550044+0000","last_unstale":"2024-04-25T14:40:40.304120+0000","last_undegraded":"2024-04-25T14:40:40.304120+0000","last_fullsized":"2024-04-25T14:40:40.304120+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:25.573353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.482361+0000","last_change":"2024-04-25T14:38:10.428736+0000","last_active":"2024-04-25T14:40:40.482361+0000","last_peered":"2024-04-25T14:40:40.482361+0000","last_clean":"2024-04-25T14:40:40.482361+0000","last_became_active":"2024-04-25T14:38:10.428410+0000","last_became_peered":"2024-04-25T14:38:10.428410+0000","last_unstale":"2024-04-25T14:40:40.482361+0000","last_undegraded":"2024-04-25T14:40:40.482361+0000","last_fullsized":"2024-04-25T14:40:40.482361+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:02:51.474220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.224711+0000","last_change":"2024-04-25T14:38:09.552800+0000","last_active":"2024-04-25T14:40:40.224711+0000","last_peered":"2024-04-25T14:40:40.224711+0000","last_clean":"2024-04-25T14:40:40.224711+0000","last_became_active":"2024-04-25T14:38:09.552460+0000","last_became_peered":"2024-04-25T14:38:09.552460+0000","last_unstale":"2024-04-25T14:40:40.224711+0000","last_undegraded":"2024-04-25T14:40:40.224711+0000","last_fullsized":"2024-04-25T14:40:40.224711+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:00.800770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.482907+0000","last_change":"2024-04-25T14:38:10.424359+0000","last_active":"2024-04-25T14:40:40.482907+0000","last_peered":"2024-04-25T14:40:40.482907+0000","last_clean":"2024-04-25T14:40:40.482907+0000","last_became_active":"2024-04-25T14:38:10.424020+0000","last_became_peered":"2024-04-25T14:38:10.424020+0000","last_unstale":"2024-04-25T14:40:40.482907+0000","last_undegraded":"2024-04-25T14:40:40.482907+0000","last_fullsized":"2024-04-25T14:40:40.482907+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:14.700772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"93'65","reported_seq":122,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:42.853177+0000","last_change":"2024-04-25T14:38:09.545637+0000","last_active":"2024-04-25T14:40:42.853177+0000","last_peered":"2024-04-25T14:40:42.853177+0000","last_clean":"2024-04-25T14:40:42.853177+0000","last_became_active":"2024-04-25T14:38:09.545319+0000","last_became_peered":"2024-04-25T14:38:09.545319+0000","last_unstale":"2024-04-25T14:40:42.853177+0000","last_undegraded":"2024-04-25T14:40:42.853177+0000","last_fullsized":"2024-04-25T14:40:42.853177+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:02.284715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.482427+0000","last_change":"2024-04-25T14:38:10.446928+0000","last_active":"2024-04-25T14:40:40.482427+0000","last_peered":"2024-04-25T14:40:40.482427+0000","last_clean":"2024-04-25T14:40:40.482427+0000","last_became_active":"2024-04-25T14:38:10.446518+0000","last_became_peered":"2024-04-25T14:38:10.446518+0000","last_unstale":"2024-04-25T14:40:40.482427+0000","last_undegraded":"2024-04-25T14:40:40.482427+0000","last_fullsized":"2024-04-25T14:40:40.482427+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:40.769059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.677657+0000","last_change":"2024-04-25T14:38:09.557381+0000","last_active":"2024-04-25T14:40:40.677657+0000","last_peered":"2024-04-25T14:40:40.677657+0000","last_clean":"2024-04-25T14:40:40.677657+0000","last_became_active":"2024-04-25T14:38:09.556504+0000","last_became_peered":"2024-04-25T14:38:09.556504+0000","last_unstale":"2024-04-25T14:40:40.677657+0000","last_undegraded":"2024-04-25T14:40:40.677657+0000","last_fullsized":"2024-04-25T14:40:40.677657+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:01:02.761249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.482978+0000","last_change":"2024-04-25T14:38:10.418104+0000","last_active":"2024-04-25T14:40:40.482978+0000","last_peered":"2024-04-25T14:40:40.482978+0000","last_clean":"2024-04-25T14:40:40.482978+0000","last_became_active":"2024-04-25T14:38:10.417774+0000","last_became_peered":"2024-04-25T14:38:10.417774+0000","last_unstale":"2024-04-25T14:40:40.482978+0000","last_undegraded":"2024-04-25T14:40:40.482978+0000","last_fullsized":"2024-04-25T14:40:40.482978+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:32:44.229899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.650780+0000","last_change":"2024-04-25T14:38:10.423546+0000","last_active":"2024-04-25T14:40:40.650780+0000","last_peered":"2024-04-25T14:40:40.650780+0000","last_clean":"2024-04-25T14:40:40.650780+0000","last_became_active":"2024-04-25T14:38:10.423216+0000","last_became_peered":"2024-04-25T14:38:10.423216+0000","last_unstale":"2024-04-25T14:40:40.650780+0000","last_undegraded":"2024-04-25T14:40:40.650780+0000","last_fullsized":"2024-04-25T14:40:40.650780+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:58.640669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.224609+0000","last_change":"2024-04-25T14:38:10.422504+0000","last_active":"2024-04-25T14:40:40.224609+0000","last_peered":"2024-04-25T14:40:40.224609+0000","last_clean":"2024-04-25T14:40:40.224609+0000","last_became_active":"2024-04-25T14:38:10.422095+0000","last_became_peered":"2024-04-25T14:38:10.422095+0000","last_unstale":"2024-04-25T14:40:40.224609+0000","last_undegraded":"2024-04-25T14:40:40.224609+0000","last_fullsized":"2024-04-25T14:40:40.224609+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:25.795962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.224467+0000","last_change":"2024-04-25T14:38:10.447446+0000","last_active":"2024-04-25T14:40:40.224467+0000","last_peered":"2024-04-25T14:40:40.224467+0000","last_clean":"2024-04-25T14:40:40.224467+0000","last_became_active":"2024-04-25T14:38:10.447056+0000","last_became_peered":"2024-04-25T14:38:10.447056+0000","last_unstale":"2024-04-25T14:40:40.224467+0000","last_undegraded":"2024-04-25T14:40:40.224467+0000","last_fullsized":"2024-04-25T14:40:40.224467+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:40.462340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.990840+0000","last_change":"2024-04-25T14:38:10.433444+0000","last_active":"2024-04-25T14:40:40.990840+0000","last_peered":"2024-04-25T14:40:40.990840+0000","last_clean":"2024-04-25T14:40:40.990840+0000","last_became_active":"2024-04-25T14:38:10.433153+0000","last_became_peered":"2024-04-25T14:38:10.433153+0000","last_unstale":"2024-04-25T14:40:40.990840+0000","last_undegraded":"2024-04-25T14:40:40.990840+0000","last_fullsized":"2024-04-25T14:40:40.990840+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:10.722650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.676722+0000","last_change":"2024-04-25T14:38:10.438322+0000","last_active":"2024-04-25T14:40:40.676722+0000","last_peered":"2024-04-25T14:40:40.676722+0000","last_clean":"2024-04-25T14:40:40.676722+0000","last_became_active":"2024-04-25T14:38:10.436103+0000","last_became_peered":"2024-04-25T14:38:10.436103+0000","last_unstale":"2024-04-25T14:40:40.676722+0000","last_undegraded":"2024-04-25T14:40:40.676722+0000","last_fullsized":"2024-04-25T14:40:40.676722+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:16.790202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.650995+0000","last_change":"2024-04-25T14:38:10.419710+0000","last_active":"2024-04-25T14:40:40.650995+0000","last_peered":"2024-04-25T14:40:40.650995+0000","last_clean":"2024-04-25T14:40:40.650995+0000","last_became_active":"2024-04-25T14:38:10.419406+0000","last_became_peered":"2024-04-25T14:38:10.419406+0000","last_unstale":"2024-04-25T14:40:40.650995+0000","last_undegraded":"2024-04-25T14:40:40.650995+0000","last_fullsized":"2024-04-25T14:40:40.650995+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:37.964698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.224748+0000","last_change":"2024-04-25T14:38:10.429510+0000","last_active":"2024-04-25T14:40:40.224748+0000","last_peered":"2024-04-25T14:40:40.224748+0000","last_clean":"2024-04-25T14:40:40.224748+0000","last_became_active":"2024-04-25T14:38:10.429219+0000","last_became_peered":"2024-04-25T14:38:10.429219+0000","last_unstale":"2024-04-25T14:40:40.224748+0000","last_undegraded":"2024-04-25T14:40:40.224748+0000","last_fullsized":"2024-04-25T14:40:40.224748+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:52:46.673126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.181877+0000","last_change":"2024-04-25T14:38:10.441487+0000","last_active":"2024-04-25T14:40:40.181877+0000","last_peered":"2024-04-25T14:40:40.181877+0000","last_clean":"2024-04-25T14:40:40.181877+0000","last_became_active":"2024-04-25T14:38:10.441111+0000","last_became_peered":"2024-04-25T14:38:10.441111+0000","last_unstale":"2024-04-25T14:40:40.181877+0000","last_undegraded":"2024-04-25T14:40:40.181877+0000","last_fullsized":"2024-04-25T14:40:40.181877+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:59.842687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.179847+0000","last_change":"2024-04-25T14:38:10.436511+0000","last_active":"2024-04-25T14:40:40.179847+0000","last_peered":"2024-04-25T14:40:40.179847+0000","last_clean":"2024-04-25T14:40:40.179847+0000","last_became_active":"2024-04-25T14:38:10.436171+0000","last_became_peered":"2024-04-25T14:38:10.436171+0000","last_unstale":"2024-04-25T14:40:40.179847+0000","last_undegraded":"2024-04-25T14:40:40.179847+0000","last_fullsized":"2024-04-25T14:40:40.179847+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:27:19.605907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.181672+0000","last_change":"2024-04-25T14:38:10.433624+0000","last_active":"2024-04-25T14:40:40.181672+0000","last_peered":"2024-04-25T14:40:40.181672+0000","last_clean":"2024-04-25T14:40:40.181672+0000","last_became_active":"2024-04-25T14:38:10.433258+0000","last_became_peered":"2024-04-25T14:38:10.433258+0000","last_unstale":"2024-04-25T14:40:40.181672+0000","last_undegraded":"2024-04-25T14:40:40.181672+0000","last_fullsized":"2024-04-25T14:40:40.181672+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:53.563110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.224234+0000","last_change":"2024-04-25T14:38:10.446924+0000","last_active":"2024-04-25T14:40:40.224234+0000","last_peered":"2024-04-25T14:40:40.224234+0000","last_clean":"2024-04-25T14:40:40.224234+0000","last_became_active":"2024-04-25T14:38:10.446553+0000","last_became_peered":"2024-04-25T14:38:10.446553+0000","last_unstale":"2024-04-25T14:40:40.224234+0000","last_undegraded":"2024-04-25T14:40:40.224234+0000","last_fullsized":"2024-04-25T14:40:40.224234+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:00.203796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.677384+0000","last_change":"2024-04-25T14:38:10.429952+0000","last_active":"2024-04-25T14:40:40.677384+0000","last_peered":"2024-04-25T14:40:40.677384+0000","last_clean":"2024-04-25T14:40:40.677384+0000","last_became_active":"2024-04-25T14:38:10.429626+0000","last_became_peered":"2024-04-25T14:38:10.429626+0000","last_unstale":"2024-04-25T14:40:40.677384+0000","last_undegraded":"2024-04-25T14:40:40.677384+0000","last_fullsized":"2024-04-25T14:40:40.677384+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:07.287113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.650742+0000","last_change":"2024-04-25T14:38:10.418292+0000","last_active":"2024-04-25T14:40:40.650742+0000","last_peered":"2024-04-25T14:40:40.650742+0000","last_clean":"2024-04-25T14:40:40.650742+0000","last_became_active":"2024-04-25T14:38:10.418005+0000","last_became_peered":"2024-04-25T14:38:10.418005+0000","last_unstale":"2024-04-25T14:40:40.650742+0000","last_undegraded":"2024-04-25T14:40:40.650742+0000","last_fullsized":"2024-04-25T14:40:40.650742+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:25.486993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":795824,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1284,"num_read_kb":859,"num_write":420,"num_write_kb":2356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1035,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2617344,"data_stored":2387472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":310,"ondisk_log_size":310,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285613,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1036288,"data_stored":495726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547250,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29156,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93715996,"statfs":{"total":95995035648,"available":95965179904,"internally_reserved":0,"allocated":2068480,"data_stored":1485260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710775,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28424,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716728,"statfs":{"total":95995035648,"available":95965929472,"internally_reserved":0,"allocated":1441792,"data_stored":889156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972414,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36740,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708412,"statfs":{"total":95995035648,"available":95957413888,"internally_reserved":0,"allocated":1437696,"data_stored":889054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266754,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28624,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716528,"statfs":{"total":95995035648,"available":95965724672,"internally_reserved":0,"allocated":1642496,"data_stored":1089555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430278,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":495329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574946,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1105920,"data_stored":563732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738472,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1056768,"data_stored":496600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411423,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36436,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708716,"statfs":{"total":95995035648,"available":95957725184,"internally_reserved":0,"allocated":1122304,"data_stored":564460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149784,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28872,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716280,"statfs":{"total":95995035648,"available":95965470720,"internally_reserved":0,"allocated":1777664,"data_stored":1217088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888149,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1048576,"data_stored":498544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691918,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37396,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707756,"statfs":{"total":95995035648,"available":95956742144,"internally_reserved":0,"allocated":1986560,"data_stored":1416238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":69338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":741376,"data_stored":721732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":360448,"data_stored":330631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":399653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:40:51.357 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T14:40:51.357 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T14:40:51.861 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:51 smithi150 ceph-mon[32504]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 102 KiB/s wr, 59 op/s 2024-04-25T14:40:51.862 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:51 smithi150 ceph-mon[32504]: from='client.25333 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:40:51.862 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:51 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:51.862 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:51 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:51.862 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:51 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:40:51.862 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:51 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:40:51.862 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:51 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:51.876 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:40:51.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:51 smithi183 ceph-mon[32584]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 102 KiB/s wr, 59 op/s 2024-04-25T14:40:51.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:51 smithi183 ceph-mon[32584]: from='client.25333 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:40:51.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:51 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:51.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:51 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:51.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:51 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:40:51.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:51 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:40:51.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:51 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:52.024 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:51 smithi053 ceph-mon[27773]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 102 KiB/s wr, 59 op/s 2024-04-25T14:40:52.024 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:51 smithi053 ceph-mon[27773]: from='client.25333 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:40:52.024 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:51 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:52.024 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:51 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:52.024 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:51 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:40:52.024 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:51 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:40:52.024 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:51 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:40:53.841 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:53 smithi053 ceph-mon[27773]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 83 KiB/s wr, 51 op/s 2024-04-25T14:40:53.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:53 smithi183 ceph-mon[32584]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 83 KiB/s wr, 51 op/s 2024-04-25T14:40:53.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:53 smithi150 ceph-mon[32504]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 83 KiB/s wr, 51 op/s 2024-04-25T14:40:54.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.222+0000 7fa35da00700 1 -- 172.21.15.53:0/3009362838 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa35806a750 msgr2=0x7fa35806ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:54.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.222+0000 7fa35da00700 1 --2- 172.21.15.53:0/3009362838 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa35806a750 0x7fa35806ab30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa34800b0a0 tx=0x7fa34800b3b0 comp rx=0 tx=0).stop 2024-04-25T14:40:54.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.222+0000 7fa35da00700 1 -- 172.21.15.53:0/3009362838 shutdown_connections 2024-04-25T14:40:54.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.222+0000 7fa35da00700 1 --2- 172.21.15.53:0/3009362838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa35810f4f0 0x7fa3581118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:54.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.222+0000 7fa35da00700 1 --2- 172.21.15.53:0/3009362838 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa35806a750 0x7fa35806ab30 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:54.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.222+0000 7fa35da00700 1 --2- 172.21.15.53:0/3009362838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa35806b070 0x7fa35810efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:54.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.222+0000 7fa35da00700 1 -- 172.21.15.53:0/3009362838 >> 172.21.15.53:0/3009362838 conn(0x7fa3580788f0 msgr2=0x7fa358078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:54.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.222+0000 7fa35da00700 1 -- 172.21.15.53:0/3009362838 shutdown_connections 2024-04-25T14:40:54.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.222+0000 7fa35da00700 1 -- 172.21.15.53:0/3009362838 wait complete. 2024-04-25T14:40:54.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.224+0000 7fa35da00700 1 Processor -- start 2024-04-25T14:40:54.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.224+0000 7fa35da00700 1 -- start start 2024-04-25T14:40:54.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.224+0000 7fa35da00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa35806a750 0x7fa35819b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:54.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.224+0000 7fa35da00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa35806b070 0x7fa35819b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:54.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.224+0000 7fa35da00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa35810f4f0 0x7fa35819fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:54.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.225+0000 7fa35da00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa358114e50 con 0x7fa35806b070 2024-04-25T14:40:54.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.225+0000 7fa35da00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa358114cd0 con 0x7fa35806a750 2024-04-25T14:40:54.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.225+0000 7fa35da00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa358114fd0 con 0x7fa35810f4f0 2024-04-25T14:40:54.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.225+0000 7fa356a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa35806a750 0x7fa35819b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:54.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.225+0000 7fa356000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa35806b070 0x7fa35819b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:54.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.225+0000 7fa356a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa35806a750 0x7fa35819b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.53:47638/0 (socket says 172.21.15.53:47638) 2024-04-25T14:40:54.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.225+0000 7fa356a00700 1 -- 172.21.15.53:0/590810667 learned_addr learned my addr 172.21.15.53:0/590810667 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:54.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.225+0000 7fa357400700 1 --2- 172.21.15.53:0/590810667 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa35810f4f0 0x7fa35819fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:54.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.225+0000 7fa356a00700 1 -- 172.21.15.53:0/590810667 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa35810f4f0 msgr2=0x7fa35819fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:54.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.225+0000 7fa356a00700 1 --2- 172.21.15.53:0/590810667 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa35810f4f0 0x7fa35819fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:54.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.225+0000 7fa356a00700 1 -- 172.21.15.53:0/590810667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa35806b070 msgr2=0x7fa35819b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:54.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.225+0000 7fa356a00700 1 --2- 172.21.15.53:0/590810667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa35806b070 0x7fa35819b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:54.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.225+0000 7fa356a00700 1 -- 172.21.15.53:0/590810667 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3581a0370 con 0x7fa35806a750 2024-04-25T14:40:54.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.225+0000 7fa357400700 1 --2- 172.21.15.53:0/590810667 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa35810f4f0 0x7fa35819fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:40:54.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.226+0000 7fa356a00700 1 --2- 172.21.15.53:0/590810667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa35806a750 0x7fa35819b360 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa34800d010 tx=0x7fa348004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:54.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.226+0000 7fa34f400700 1 -- 172.21.15.53:0/590810667 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa348021070 con 0x7fa35806a750 2024-04-25T14:40:54.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.226+0000 7fa34f400700 1 -- 172.21.15.53:0/590810667 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa348007c40 con 0x7fa35806a750 2024-04-25T14:40:54.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.226+0000 7fa35da00700 1 -- 172.21.15.53:0/590810667 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3581a0600 con 0x7fa35806a750 2024-04-25T14:40:54.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.226+0000 7fa34f400700 1 -- 172.21.15.53:0/590810667 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3480046a0 con 0x7fa35806a750 2024-04-25T14:40:54.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.226+0000 7fa35da00700 1 -- 172.21.15.53:0/590810667 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3581a7ee0 con 0x7fa35806a750 2024-04-25T14:40:54.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.227+0000 7fa35da00700 1 -- 172.21.15.53:0/590810667 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa358069730 con 0x7fa35806a750 2024-04-25T14:40:54.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.229+0000 7fa34f400700 1 -- 172.21.15.53:0/590810667 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7fa348003a00 con 0x7fa35806a750 2024-04-25T14:40:54.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.231+0000 7fa34f400700 1 --2- 172.21.15.53:0/590810667 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fa33c0a3ec0 0x7fa33c0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:54.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.231+0000 7fa34f400700 1 -- 172.21.15.53:0/590810667 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8543+0+0 (secure 0 0 0) 0x7fa34800f020 con 0x7fa35806a750 2024-04-25T14:40:54.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.231+0000 7fa356000700 1 --2- 172.21.15.53:0/590810667 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fa33c0a3ec0 0x7fa33c0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:54.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.232+0000 7fa356000700 1 --2- 172.21.15.53:0/590810667 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fa33c0a3ec0 0x7fa33c0a6380 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa344009680 tx=0x7fa344009260 comp rx=0 tx=0).ready entity=mgr.15093 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:54.234 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.235+0000 7fa34f400700 1 -- 172.21.15.53:0/590810667 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa34802a020 con 0x7fa35806a750 2024-04-25T14:40:54.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.460+0000 7fa35da00700 1 -- 172.21.15.53:0/590810667 --> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa35819c6c0 con 0x7fa33c0a3ec0 2024-04-25T14:40:54.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.485+0000 7fa34f400700 1 -- 172.21.15.53:0/590810667 <== mgr.15093 v2:172.21.15.53:6800/802683714 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257772 (secure 0 0 0) 0x7fa35819c6c0 con 0x7fa33c0a3ec0 2024-04-25T14:40:54.490 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:40:54.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.491+0000 7fa35da00700 1 -- 172.21.15.53:0/590810667 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fa33c0a3ec0 msgr2=0x7fa33c0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:54.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.491+0000 7fa35da00700 1 --2- 172.21.15.53:0/590810667 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fa33c0a3ec0 0x7fa33c0a6380 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa344009680 tx=0x7fa344009260 comp rx=0 tx=0).stop 2024-04-25T14:40:54.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.491+0000 7fa35da00700 1 -- 172.21.15.53:0/590810667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa35806a750 msgr2=0x7fa35819b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:54.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.491+0000 7fa35da00700 1 --2- 172.21.15.53:0/590810667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa35806a750 0x7fa35819b360 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa34800d010 tx=0x7fa348004d00 comp rx=0 tx=0).stop 2024-04-25T14:40:54.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.491+0000 7fa35da00700 1 -- 172.21.15.53:0/590810667 shutdown_connections 2024-04-25T14:40:54.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.491+0000 7fa35da00700 1 --2- 172.21.15.53:0/590810667 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fa33c0a3ec0 0x7fa33c0a6380 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:54.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.491+0000 7fa35da00700 1 --2- 172.21.15.53:0/590810667 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa35810f4f0 0x7fa35819fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:54.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.491+0000 7fa35da00700 1 --2- 172.21.15.53:0/590810667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa35806a750 0x7fa35819b360 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:54.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.491+0000 7fa35da00700 1 --2- 172.21.15.53:0/590810667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa35806b070 0x7fa35819b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:54.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.491+0000 7fa35da00700 1 -- 172.21.15.53:0/590810667 >> 172.21.15.53:0/590810667 conn(0x7fa3580788f0 msgr2=0x7fa3581109c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:54.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.491+0000 7fa35da00700 1 -- 172.21.15.53:0/590810667 shutdown_connections 2024-04-25T14:40:54.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:54.491+0000 7fa35da00700 1 -- 172.21.15.53:0/590810667 wait complete. 2024-04-25T14:40:54.493 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-04-25T14:40:55.309 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-04-25T14:40:54.200994+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1386426,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1551,"num_read_kb":1204,"num_write":623,"num_write_kb":5296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1181771,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":960,"ondisk_log_size":960,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":386708,"kb_used_data":16368,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124555116,"statfs":{"total":1151940427776,"available":1151544438784,"internally_reserved":0,"allocated":16760832,"data_stored":10100742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19137,"internal_metadata":378713407},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":414,"num_read_kb":471,"num_write":146,"num_write_kb":914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004532"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.651108+0000","last_change":"2024-04-25T14:38:10.435535+0000","last_active":"2024-04-25T14:40:40.651108+0000","last_peered":"2024-04-25T14:40:40.651108+0000","last_clean":"2024-04-25T14:40:40.651108+0000","last_became_active":"2024-04-25T14:38:10.435205+0000","last_became_peered":"2024-04-25T14:38:10.435205+0000","last_unstale":"2024-04-25T14:40:40.651108+0000","last_undegraded":"2024-04-25T14:40:40.651108+0000","last_fullsized":"2024-04-25T14:40:40.651108+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:12.301771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.991215+0000","last_change":"2024-04-25T14:38:10.428216+0000","last_active":"2024-04-25T14:40:40.991215+0000","last_peered":"2024-04-25T14:40:40.991215+0000","last_clean":"2024-04-25T14:40:40.991215+0000","last_became_active":"2024-04-25T14:38:10.427419+0000","last_became_peered":"2024-04-25T14:38:10.427419+0000","last_unstale":"2024-04-25T14:40:40.991215+0000","last_undegraded":"2024-04-25T14:40:40.991215+0000","last_fullsized":"2024-04-25T14:40:40.991215+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:42.580424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.991464+0000","last_change":"2024-04-25T14:38:10.427827+0000","last_active":"2024-04-25T14:40:40.991464+0000","last_peered":"2024-04-25T14:40:40.991464+0000","last_clean":"2024-04-25T14:40:40.991464+0000","last_became_active":"2024-04-25T14:38:10.427435+0000","last_became_peered":"2024-04-25T14:38:10.427435+0000","last_unstale":"2024-04-25T14:40:40.991464+0000","last_undegraded":"2024-04-25T14:40:40.991464+0000","last_fullsized":"2024-04-25T14:40:40.991464+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:15.539399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.481932+0000","last_change":"2024-04-25T14:38:10.423751+0000","last_active":"2024-04-25T14:40:40.481932+0000","last_peered":"2024-04-25T14:40:40.481932+0000","last_clean":"2024-04-25T14:40:40.481932+0000","last_became_active":"2024-04-25T14:38:10.423409+0000","last_became_peered":"2024-04-25T14:38:10.423409+0000","last_unstale":"2024-04-25T14:40:40.481932+0000","last_undegraded":"2024-04-25T14:40:40.481932+0000","last_fullsized":"2024-04-25T14:40:40.481932+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:45.403788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.304337+0000","last_change":"2024-04-25T14:38:10.430000+0000","last_active":"2024-04-25T14:40:40.304337+0000","last_peered":"2024-04-25T14:40:40.304337+0000","last_clean":"2024-04-25T14:40:40.304337+0000","last_became_active":"2024-04-25T14:38:10.429684+0000","last_became_peered":"2024-04-25T14:38:10.429684+0000","last_unstale":"2024-04-25T14:40:40.304337+0000","last_undegraded":"2024-04-25T14:40:40.304337+0000","last_fullsized":"2024-04-25T14:40:40.304337+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:18.421970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.179542+0000","last_change":"2024-04-25T14:39:15.984180+0000","last_active":"2024-04-25T14:40:40.179542+0000","last_peered":"2024-04-25T14:40:40.179542+0000","last_clean":"2024-04-25T14:40:40.179542+0000","last_became_active":"2024-04-25T14:39:15.983821+0000","last_became_peered":"2024-04-25T14:39:15.983821+0000","last_unstale":"2024-04-25T14:40:40.179542+0000","last_undegraded":"2024-04-25T14:40:40.179542+0000","last_fullsized":"2024-04-25T14:40:40.179542+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:54:47.617713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.482156+0000","last_change":"2024-04-25T14:38:10.420022+0000","last_active":"2024-04-25T14:40:40.482156+0000","last_peered":"2024-04-25T14:40:40.482156+0000","last_clean":"2024-04-25T14:40:40.482156+0000","last_became_active":"2024-04-25T14:38:10.419673+0000","last_became_peered":"2024-04-25T14:38:10.419673+0000","last_unstale":"2024-04-25T14:40:40.482156+0000","last_undegraded":"2024-04-25T14:40:40.482156+0000","last_fullsized":"2024-04-25T14:40:40.482156+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:09.177814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.281442+0000","last_change":"2024-04-25T14:40:16.499255+0000","last_active":"2024-04-25T14:40:40.281442+0000","last_peered":"2024-04-25T14:40:40.281442+0000","last_clean":"2024-04-25T14:40:40.281442+0000","last_became_active":"2024-04-25T14:40:16.497912+0000","last_became_peered":"2024-04-25T14:40:16.497912+0000","last_unstale":"2024-04-25T14:40:40.281442+0000","last_undegraded":"2024-04-25T14:40:40.281442+0000","last_fullsized":"2024-04-25T14:40:40.281442+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:10.595053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,10],"acting":[9,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.224740+0000","last_change":"2024-04-25T14:38:10.424086+0000","last_active":"2024-04-25T14:40:40.224740+0000","last_peered":"2024-04-25T14:40:40.224740+0000","last_clean":"2024-04-25T14:40:40.224740+0000","last_became_active":"2024-04-25T14:38:10.423769+0000","last_became_peered":"2024-04-25T14:38:10.423769+0000","last_unstale":"2024-04-25T14:40:40.224740+0000","last_undegraded":"2024-04-25T14:40:40.224740+0000","last_fullsized":"2024-04-25T14:40:40.224740+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:19:22.736053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.180126+0000","last_change":"2024-04-25T14:38:10.427049+0000","last_active":"2024-04-25T14:40:40.180126+0000","last_peered":"2024-04-25T14:40:40.180126+0000","last_clean":"2024-04-25T14:40:40.180126+0000","last_became_active":"2024-04-25T14:38:10.426729+0000","last_became_peered":"2024-04-25T14:38:10.426729+0000","last_unstale":"2024-04-25T14:40:40.180126+0000","last_undegraded":"2024-04-25T14:40:40.180126+0000","last_fullsized":"2024-04-25T14:40:40.180126+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:00.849650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.482804+0000","last_change":"2024-04-25T14:38:10.424976+0000","last_active":"2024-04-25T14:40:40.482804+0000","last_peered":"2024-04-25T14:40:40.482804+0000","last_clean":"2024-04-25T14:40:40.482804+0000","last_became_active":"2024-04-25T14:38:10.424585+0000","last_became_peered":"2024-04-25T14:38:10.424585+0000","last_unstale":"2024-04-25T14:40:40.482804+0000","last_undegraded":"2024-04-25T14:40:40.482804+0000","last_fullsized":"2024-04-25T14:40:40.482804+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:54.639471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.481912+0000","last_change":"2024-04-25T14:38:10.418022+0000","last_active":"2024-04-25T14:40:40.481912+0000","last_peered":"2024-04-25T14:40:40.481912+0000","last_clean":"2024-04-25T14:40:40.481912+0000","last_became_active":"2024-04-25T14:38:10.417735+0000","last_became_peered":"2024-04-25T14:38:10.417735+0000","last_unstale":"2024-04-25T14:40:40.481912+0000","last_undegraded":"2024-04-25T14:40:40.481912+0000","last_fullsized":"2024-04-25T14:40:40.481912+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:48.785521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.677401+0000","last_change":"2024-04-25T14:38:10.423464+0000","last_active":"2024-04-25T14:40:40.677401+0000","last_peered":"2024-04-25T14:40:40.677401+0000","last_clean":"2024-04-25T14:40:40.677401+0000","last_became_active":"2024-04-25T14:38:10.423105+0000","last_became_peered":"2024-04-25T14:38:10.423105+0000","last_unstale":"2024-04-25T14:40:40.677401+0000","last_undegraded":"2024-04-25T14:40:40.677401+0000","last_fullsized":"2024-04-25T14:40:40.677401+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:26.568198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.991106+0000","last_change":"2024-04-25T14:38:10.426870+0000","last_active":"2024-04-25T14:40:40.991106+0000","last_peered":"2024-04-25T14:40:40.991106+0000","last_clean":"2024-04-25T14:40:40.991106+0000","last_became_active":"2024-04-25T14:38:10.426506+0000","last_became_peered":"2024-04-25T14:38:10.426506+0000","last_unstale":"2024-04-25T14:40:40.991106+0000","last_undegraded":"2024-04-25T14:40:40.991106+0000","last_fullsized":"2024-04-25T14:40:40.991106+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:16:49.539221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.990805+0000","last_change":"2024-04-25T14:38:10.434618+0000","last_active":"2024-04-25T14:40:40.990805+0000","last_peered":"2024-04-25T14:40:40.990805+0000","last_clean":"2024-04-25T14:40:40.990805+0000","last_became_active":"2024-04-25T14:38:10.430661+0000","last_became_peered":"2024-04-25T14:38:10.430661+0000","last_unstale":"2024-04-25T14:40:40.990805+0000","last_undegraded":"2024-04-25T14:40:40.990805+0000","last_fullsized":"2024-04-25T14:40:40.990805+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:44.548788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.224125+0000","last_change":"2024-04-25T14:38:10.429898+0000","last_active":"2024-04-25T14:40:40.224125+0000","last_peered":"2024-04-25T14:40:40.224125+0000","last_clean":"2024-04-25T14:40:40.224125+0000","last_became_active":"2024-04-25T14:38:10.429557+0000","last_became_peered":"2024-04-25T14:38:10.429557+0000","last_unstale":"2024-04-25T14:40:40.224125+0000","last_undegraded":"2024-04-25T14:40:40.224125+0000","last_fullsized":"2024-04-25T14:40:40.224125+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:54.258526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.651158+0000","last_change":"2024-04-25T14:38:10.435134+0000","last_active":"2024-04-25T14:40:40.651158+0000","last_peered":"2024-04-25T14:40:40.651158+0000","last_clean":"2024-04-25T14:40:40.651158+0000","last_became_active":"2024-04-25T14:38:10.434775+0000","last_became_peered":"2024-04-25T14:38:10.434775+0000","last_unstale":"2024-04-25T14:40:40.651158+0000","last_undegraded":"2024-04-25T14:40:40.651158+0000","last_fullsized":"2024-04-25T14:40:40.651158+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:49.827881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.282019+0000","last_change":"2024-04-25T14:38:10.424929+0000","last_active":"2024-04-25T14:40:40.282019+0000","last_peered":"2024-04-25T14:40:40.282019+0000","last_clean":"2024-04-25T14:40:40.282019+0000","last_became_active":"2024-04-25T14:38:10.424399+0000","last_became_peered":"2024-04-25T14:38:10.424399+0000","last_unstale":"2024-04-25T14:40:40.282019+0000","last_undegraded":"2024-04-25T14:40:40.282019+0000","last_fullsized":"2024-04-25T14:40:40.282019+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:08.834176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.224851+0000","last_change":"2024-04-25T14:38:10.444004+0000","last_active":"2024-04-25T14:40:40.224851+0000","last_peered":"2024-04-25T14:40:40.224851+0000","last_clean":"2024-04-25T14:40:40.224851+0000","last_became_active":"2024-04-25T14:38:10.443669+0000","last_became_peered":"2024-04-25T14:38:10.443669+0000","last_unstale":"2024-04-25T14:40:40.224851+0000","last_undegraded":"2024-04-25T14:40:40.224851+0000","last_fullsized":"2024-04-25T14:40:40.224851+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:02.302057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.281993+0000","last_change":"2024-04-25T14:38:09.545010+0000","last_active":"2024-04-25T14:40:40.281993+0000","last_peered":"2024-04-25T14:40:40.281993+0000","last_clean":"2024-04-25T14:40:40.281993+0000","last_became_active":"2024-04-25T14:38:09.544622+0000","last_became_peered":"2024-04-25T14:38:09.544622+0000","last_unstale":"2024-04-25T14:40:40.281993+0000","last_undegraded":"2024-04-25T14:40:40.281993+0000","last_fullsized":"2024-04-25T14:40:40.281993+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:51.200242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.281678+0000","last_change":"2024-04-25T14:39:15.993912+0000","last_active":"2024-04-25T14:40:40.281678+0000","last_peered":"2024-04-25T14:40:40.281678+0000","last_clean":"2024-04-25T14:40:40.281678+0000","last_became_active":"2024-04-25T14:39:15.992511+0000","last_became_peered":"2024-04-25T14:39:15.992511+0000","last_unstale":"2024-04-25T14:40:40.281678+0000","last_undegraded":"2024-04-25T14:40:40.281678+0000","last_fullsized":"2024-04-25T14:40:40.281678+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:12:57.639396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,6],"acting":[9,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.184989+0000","last_change":"2024-04-25T14:38:09.555825+0000","last_active":"2024-04-25T14:40:40.184989+0000","last_peered":"2024-04-25T14:40:40.184989+0000","last_clean":"2024-04-25T14:40:40.184989+0000","last_became_active":"2024-04-25T14:38:09.555436+0000","last_became_peered":"2024-04-25T14:38:09.555436+0000","last_unstale":"2024-04-25T14:40:40.184989+0000","last_undegraded":"2024-04-25T14:40:40.184989+0000","last_fullsized":"2024-04-25T14:40:40.184989+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:45.097583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.991388+0000","last_change":"2024-04-25T14:38:10.432254+0000","last_active":"2024-04-25T14:40:40.991388+0000","last_peered":"2024-04-25T14:40:40.991388+0000","last_clean":"2024-04-25T14:40:40.991388+0000","last_became_active":"2024-04-25T14:38:10.431835+0000","last_became_peered":"2024-04-25T14:38:10.431835+0000","last_unstale":"2024-04-25T14:40:40.991388+0000","last_undegraded":"2024-04-25T14:40:40.991388+0000","last_fullsized":"2024-04-25T14:40:40.991388+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:23.089447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.304545+0000","last_change":"2024-04-25T14:38:09.550254+0000","last_active":"2024-04-25T14:40:40.304545+0000","last_peered":"2024-04-25T14:40:40.304545+0000","last_clean":"2024-04-25T14:40:40.304545+0000","last_became_active":"2024-04-25T14:38:09.549958+0000","last_became_peered":"2024-04-25T14:38:09.549958+0000","last_unstale":"2024-04-25T14:40:40.304545+0000","last_undegraded":"2024-04-25T14:40:40.304545+0000","last_fullsized":"2024-04-25T14:40:40.304545+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:56.161427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.224323+0000","last_change":"2024-04-25T14:38:10.425648+0000","last_active":"2024-04-25T14:40:40.224323+0000","last_peered":"2024-04-25T14:40:40.224323+0000","last_clean":"2024-04-25T14:40:40.224323+0000","last_became_active":"2024-04-25T14:38:10.425009+0000","last_became_peered":"2024-04-25T14:38:10.425009+0000","last_unstale":"2024-04-25T14:40:40.224323+0000","last_undegraded":"2024-04-25T14:40:40.224323+0000","last_fullsized":"2024-04-25T14:40:40.224323+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:46.696610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.304408+0000","last_change":"2024-04-25T14:38:09.544481+0000","last_active":"2024-04-25T14:40:40.304408+0000","last_peered":"2024-04-25T14:40:40.304408+0000","last_clean":"2024-04-25T14:40:40.304408+0000","last_became_active":"2024-04-25T14:38:09.544147+0000","last_became_peered":"2024-04-25T14:38:09.544147+0000","last_unstale":"2024-04-25T14:40:40.304408+0000","last_undegraded":"2024-04-25T14:40:40.304408+0000","last_fullsized":"2024-04-25T14:40:40.304408+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:35:22.971028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.180558+0000","last_change":"2024-04-25T14:38:10.426851+0000","last_active":"2024-04-25T14:40:40.180558+0000","last_peered":"2024-04-25T14:40:40.180558+0000","last_clean":"2024-04-25T14:40:40.180558+0000","last_became_active":"2024-04-25T14:38:10.426475+0000","last_became_peered":"2024-04-25T14:38:10.426475+0000","last_unstale":"2024-04-25T14:40:40.180558+0000","last_undegraded":"2024-04-25T14:40:40.180558+0000","last_fullsized":"2024-04-25T14:40:40.180558+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:28.213318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"93'8","reported_seq":207,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:50.116696+0000","last_change":"2024-04-25T14:39:15.979586+0000","last_active":"2024-04-25T14:40:50.116696+0000","last_peered":"2024-04-25T14:40:50.116696+0000","last_clean":"2024-04-25T14:40:50.116696+0000","last_became_active":"2024-04-25T14:39:15.974397+0000","last_became_peered":"2024-04-25T14:39:15.974397+0000","last_unstale":"2024-04-25T14:40:50.116696+0000","last_undegraded":"2024-04-25T14:40:50.116696+0000","last_fullsized":"2024-04-25T14:40:50.116696+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:02.152202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":159,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.304385+0000","last_change":"2024-04-25T14:38:10.425424+0000","last_active":"2024-04-25T14:40:40.304385+0000","last_peered":"2024-04-25T14:40:40.304385+0000","last_clean":"2024-04-25T14:40:40.304385+0000","last_became_active":"2024-04-25T14:38:10.425101+0000","last_became_peered":"2024-04-25T14:38:10.425101+0000","last_unstale":"2024-04-25T14:40:40.304385+0000","last_undegraded":"2024-04-25T14:40:40.304385+0000","last_fullsized":"2024-04-25T14:40:40.304385+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:52:29.871113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.179499+0000","last_change":"2024-04-25T14:38:09.552287+0000","last_active":"2024-04-25T14:40:40.179499+0000","last_peered":"2024-04-25T14:40:40.179499+0000","last_clean":"2024-04-25T14:40:40.179499+0000","last_became_active":"2024-04-25T14:38:09.551957+0000","last_became_peered":"2024-04-25T14:38:09.551957+0000","last_unstale":"2024-04-25T14:40:40.179499+0000","last_undegraded":"2024-04-25T14:40:40.179499+0000","last_fullsized":"2024-04-25T14:40:40.179499+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:22.586868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.180162+0000","last_change":"2024-04-25T14:39:15.990838+0000","last_active":"2024-04-25T14:40:40.180162+0000","last_peered":"2024-04-25T14:40:40.180162+0000","last_clean":"2024-04-25T14:40:40.180162+0000","last_became_active":"2024-04-25T14:39:15.990370+0000","last_became_peered":"2024-04-25T14:39:15.990370+0000","last_unstale":"2024-04-25T14:40:40.180162+0000","last_undegraded":"2024-04-25T14:40:40.180162+0000","last_fullsized":"2024-04-25T14:40:40.180162+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:50.676958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.676465+0000","last_change":"2024-04-25T14:38:09.552376+0000","last_active":"2024-04-25T14:40:40.676465+0000","last_peered":"2024-04-25T14:40:40.676465+0000","last_clean":"2024-04-25T14:40:40.676465+0000","last_became_active":"2024-04-25T14:38:09.548614+0000","last_became_peered":"2024-04-25T14:38:09.548614+0000","last_unstale":"2024-04-25T14:40:40.676465+0000","last_undegraded":"2024-04-25T14:40:40.676465+0000","last_fullsized":"2024-04-25T14:40:40.676465+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:27:27.133106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.181241+0000","last_change":"2024-04-25T14:38:10.434611+0000","last_active":"2024-04-25T14:40:40.181241+0000","last_peered":"2024-04-25T14:40:40.181241+0000","last_clean":"2024-04-25T14:40:40.181241+0000","last_became_active":"2024-04-25T14:38:10.434280+0000","last_became_peered":"2024-04-25T14:38:10.434280+0000","last_unstale":"2024-04-25T14:40:40.181241+0000","last_undegraded":"2024-04-25T14:40:40.181241+0000","last_fullsized":"2024-04-25T14:40:40.181241+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:12:32.275663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.179937+0000","last_change":"2024-04-25T14:38:09.539743+0000","last_active":"2024-04-25T14:40:40.179937+0000","last_peered":"2024-04-25T14:40:40.179937+0000","last_clean":"2024-04-25T14:40:40.179937+0000","last_became_active":"2024-04-25T14:38:09.539272+0000","last_became_peered":"2024-04-25T14:38:09.539272+0000","last_unstale":"2024-04-25T14:40:40.179937+0000","last_undegraded":"2024-04-25T14:40:40.179937+0000","last_fullsized":"2024-04-25T14:40:40.179937+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:29:23.159901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.282536+0000","last_change":"2024-04-25T14:38:10.444899+0000","last_active":"2024-04-25T14:40:40.282536+0000","last_peered":"2024-04-25T14:40:40.282536+0000","last_clean":"2024-04-25T14:40:40.282536+0000","last_became_active":"2024-04-25T14:38:10.444383+0000","last_became_peered":"2024-04-25T14:38:10.444383+0000","last_unstale":"2024-04-25T14:40:40.282536+0000","last_undegraded":"2024-04-25T14:40:40.282536+0000","last_fullsized":"2024-04-25T14:40:40.282536+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:21.351436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"93'5","reported_seq":187,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:50.116205+0000","last_change":"2024-04-25T14:38:09.550181+0000","last_active":"2024-04-25T14:40:50.116205+0000","last_peered":"2024-04-25T14:40:50.116205+0000","last_clean":"2024-04-25T14:40:50.116205+0000","last_became_active":"2024-04-25T14:38:09.549824+0000","last_became_peered":"2024-04-25T14:38:09.549824+0000","last_unstale":"2024-04-25T14:40:50.116205+0000","last_undegraded":"2024-04-25T14:40:50.116205+0000","last_fullsized":"2024-04-25T14:40:50.116205+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:59.220917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":156,"num_read_kb":156,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.224700+0000","last_change":"2024-04-25T14:38:10.421339+0000","last_active":"2024-04-25T14:40:40.224700+0000","last_peered":"2024-04-25T14:40:40.224700+0000","last_clean":"2024-04-25T14:40:40.224700+0000","last_became_active":"2024-04-25T14:38:10.421005+0000","last_became_peered":"2024-04-25T14:38:10.421005+0000","last_unstale":"2024-04-25T14:40:40.224700+0000","last_undegraded":"2024-04-25T14:40:40.224700+0000","last_fullsized":"2024-04-25T14:40:40.224700+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:51.949054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.482920+0000","last_change":"2024-04-25T14:38:09.548245+0000","last_active":"2024-04-25T14:40:40.482920+0000","last_peered":"2024-04-25T14:40:40.482920+0000","last_clean":"2024-04-25T14:40:40.482920+0000","last_became_active":"2024-04-25T14:38:09.547905+0000","last_became_peered":"2024-04-25T14:38:09.547905+0000","last_unstale":"2024-04-25T14:40:40.482920+0000","last_undegraded":"2024-04-25T14:40:40.482920+0000","last_fullsized":"2024-04-25T14:40:40.482920+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:56.798594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.991226+0000","last_change":"2024-04-25T14:38:10.431427+0000","last_active":"2024-04-25T14:40:40.991226+0000","last_peered":"2024-04-25T14:40:40.991226+0000","last_clean":"2024-04-25T14:40:40.991226+0000","last_became_active":"2024-04-25T14:38:10.428759+0000","last_became_peered":"2024-04-25T14:38:10.428759+0000","last_unstale":"2024-04-25T14:40:40.991226+0000","last_undegraded":"2024-04-25T14:40:40.991226+0000","last_fullsized":"2024-04-25T14:40:40.991226+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:15:27.341445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.180647+0000","last_change":"2024-04-25T14:38:09.556839+0000","last_active":"2024-04-25T14:40:40.180647+0000","last_peered":"2024-04-25T14:40:40.180647+0000","last_clean":"2024-04-25T14:40:40.180647+0000","last_became_active":"2024-04-25T14:38:09.556443+0000","last_became_peered":"2024-04-25T14:38:09.556443+0000","last_unstale":"2024-04-25T14:40:40.180647+0000","last_undegraded":"2024-04-25T14:40:40.180647+0000","last_fullsized":"2024-04-25T14:40:40.180647+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:46.621883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.224407+0000","last_change":"2024-04-25T14:38:10.414507+0000","last_active":"2024-04-25T14:40:40.224407+0000","last_peered":"2024-04-25T14:40:40.224407+0000","last_clean":"2024-04-25T14:40:40.224407+0000","last_became_active":"2024-04-25T14:38:10.414154+0000","last_became_peered":"2024-04-25T14:38:10.414154+0000","last_unstale":"2024-04-25T14:40:40.224407+0000","last_undegraded":"2024-04-25T14:40:40.224407+0000","last_fullsized":"2024-04-25T14:40:40.224407+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:12.208774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"93'36","reported_seq":385,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:51.117622+0000","last_change":"2024-04-25T14:38:09.549239+0000","last_active":"2024-04-25T14:40:51.117622+0000","last_peered":"2024-04-25T14:40:51.117622+0000","last_clean":"2024-04-25T14:40:51.117622+0000","last_became_active":"2024-04-25T14:38:09.548888+0000","last_became_peered":"2024-04-25T14:38:09.548888+0000","last_unstale":"2024-04-25T14:40:51.117622+0000","last_undegraded":"2024-04-25T14:40:51.117622+0000","last_fullsized":"2024-04-25T14:40:51.117622+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:36:46.536908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68610,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":323,"num_read_kb":229,"num_write":37,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.304392+0000","last_change":"2024-04-25T14:38:10.417510+0000","last_active":"2024-04-25T14:40:40.304392+0000","last_peered":"2024-04-25T14:40:40.304392+0000","last_clean":"2024-04-25T14:40:40.304392+0000","last_became_active":"2024-04-25T14:38:10.417079+0000","last_became_peered":"2024-04-25T14:38:10.417079+0000","last_unstale":"2024-04-25T14:40:40.304392+0000","last_undegraded":"2024-04-25T14:40:40.304392+0000","last_fullsized":"2024-04-25T14:40:40.304392+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:35.213199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.224809+0000","last_change":"2024-04-25T14:38:09.563650+0000","last_active":"2024-04-25T14:40:40.224809+0000","last_peered":"2024-04-25T14:40:40.224809+0000","last_clean":"2024-04-25T14:40:40.224809+0000","last_became_active":"2024-04-25T14:38:09.563347+0000","last_became_peered":"2024-04-25T14:38:09.563347+0000","last_unstale":"2024-04-25T14:40:40.224809+0000","last_undegraded":"2024-04-25T14:40:40.224809+0000","last_fullsized":"2024-04-25T14:40:40.224809+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:16.793191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.676501+0000","last_change":"2024-04-25T14:38:10.447504+0000","last_active":"2024-04-25T14:40:40.676501+0000","last_peered":"2024-04-25T14:40:40.676501+0000","last_clean":"2024-04-25T14:40:40.676501+0000","last_became_active":"2024-04-25T14:38:10.447157+0000","last_became_peered":"2024-04-25T14:38:10.447157+0000","last_unstale":"2024-04-25T14:40:40.676501+0000","last_undegraded":"2024-04-25T14:40:40.676501+0000","last_fullsized":"2024-04-25T14:40:40.676501+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:38.940320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.180450+0000","last_change":"2024-04-25T14:38:09.545214+0000","last_active":"2024-04-25T14:40:40.180450+0000","last_peered":"2024-04-25T14:40:40.180450+0000","last_clean":"2024-04-25T14:40:40.180450+0000","last_became_active":"2024-04-25T14:38:09.544903+0000","last_became_peered":"2024-04-25T14:38:09.544903+0000","last_unstale":"2024-04-25T14:40:40.180450+0000","last_undegraded":"2024-04-25T14:40:40.180450+0000","last_fullsized":"2024-04-25T14:40:40.180450+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:58:15.817036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.482678+0000","last_change":"2024-04-25T14:38:10.432017+0000","last_active":"2024-04-25T14:40:40.482678+0000","last_peered":"2024-04-25T14:40:40.482678+0000","last_clean":"2024-04-25T14:40:40.482678+0000","last_became_active":"2024-04-25T14:38:10.431596+0000","last_became_peered":"2024-04-25T14:38:10.431596+0000","last_unstale":"2024-04-25T14:40:40.482678+0000","last_undegraded":"2024-04-25T14:40:40.482678+0000","last_fullsized":"2024-04-25T14:40:40.482678+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:30.653642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"93'7","reported_seq":195,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:49.988459+0000","last_change":"2024-04-25T14:38:09.545477+0000","last_active":"2024-04-25T14:40:49.988459+0000","last_peered":"2024-04-25T14:40:49.988459+0000","last_clean":"2024-04-25T14:40:49.988459+0000","last_became_active":"2024-04-25T14:38:09.545093+0000","last_became_peered":"2024-04-25T14:38:09.545093+0000","last_unstale":"2024-04-25T14:40:49.988459+0000","last_undegraded":"2024-04-25T14:40:49.988459+0000","last_fullsized":"2024-04-25T14:40:49.988459+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:56.589231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1821,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":163,"num_read_kb":7,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.481697+0000","last_change":"2024-04-25T14:38:10.418914+0000","last_active":"2024-04-25T14:40:40.481697+0000","last_peered":"2024-04-25T14:40:40.481697+0000","last_clean":"2024-04-25T14:40:40.481697+0000","last_became_active":"2024-04-25T14:38:10.418600+0000","last_became_peered":"2024-04-25T14:38:10.418600+0000","last_unstale":"2024-04-25T14:40:40.481697+0000","last_undegraded":"2024-04-25T14:40:40.481697+0000","last_fullsized":"2024-04-25T14:40:40.481697+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:31.324035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.304080+0000","last_change":"2024-04-25T14:38:09.556574+0000","last_active":"2024-04-25T14:40:40.304080+0000","last_peered":"2024-04-25T14:40:40.304080+0000","last_clean":"2024-04-25T14:40:40.304080+0000","last_became_active":"2024-04-25T14:38:09.556071+0000","last_became_peered":"2024-04-25T14:38:09.556071+0000","last_unstale":"2024-04-25T14:40:40.304080+0000","last_undegraded":"2024-04-25T14:40:40.304080+0000","last_fullsized":"2024-04-25T14:40:40.304080+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:05.424593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.180084+0000","last_change":"2024-04-25T14:38:10.424591+0000","last_active":"2024-04-25T14:40:40.180084+0000","last_peered":"2024-04-25T14:40:40.180084+0000","last_clean":"2024-04-25T14:40:40.180084+0000","last_became_active":"2024-04-25T14:38:10.424258+0000","last_became_peered":"2024-04-25T14:38:10.424258+0000","last_unstale":"2024-04-25T14:40:40.180084+0000","last_undegraded":"2024-04-25T14:40:40.180084+0000","last_fullsized":"2024-04-25T14:40:40.180084+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:22.753595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"93'3","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.304606+0000","last_change":"2024-04-25T14:38:09.546073+0000","last_active":"2024-04-25T14:40:40.304606+0000","last_peered":"2024-04-25T14:40:40.304606+0000","last_clean":"2024-04-25T14:40:40.304606+0000","last_became_active":"2024-04-25T14:38:09.545783+0000","last_became_peered":"2024-04-25T14:38:09.545783+0000","last_unstale":"2024-04-25T14:40:40.304606+0000","last_undegraded":"2024-04-25T14:40:40.304606+0000","last_fullsized":"2024-04-25T14:40:40.304606+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:42.537431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.650953+0000","last_change":"2024-04-25T14:38:10.417230+0000","last_active":"2024-04-25T14:40:40.650953+0000","last_peered":"2024-04-25T14:40:40.650953+0000","last_clean":"2024-04-25T14:40:40.650953+0000","last_became_active":"2024-04-25T14:38:10.416891+0000","last_became_peered":"2024-04-25T14:38:10.416891+0000","last_unstale":"2024-04-25T14:40:40.650953+0000","last_undegraded":"2024-04-25T14:40:40.650953+0000","last_fullsized":"2024-04-25T14:40:40.650953+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:10:13.434518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.281801+0000","last_change":"2024-04-25T14:39:15.994906+0000","last_active":"2024-04-25T14:40:40.281801+0000","last_peered":"2024-04-25T14:40:40.281801+0000","last_clean":"2024-04-25T14:40:40.281801+0000","last_became_active":"2024-04-25T14:39:15.994365+0000","last_became_peered":"2024-04-25T14:39:15.994365+0000","last_unstale":"2024-04-25T14:40:40.281801+0000","last_undegraded":"2024-04-25T14:40:40.281801+0000","last_fullsized":"2024-04-25T14:40:40.281801+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:19.026241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.179623+0000","last_change":"2024-04-25T14:38:10.446846+0000","last_active":"2024-04-25T14:40:40.179623+0000","last_peered":"2024-04-25T14:40:40.179623+0000","last_clean":"2024-04-25T14:40:40.179623+0000","last_became_active":"2024-04-25T14:38:10.446498+0000","last_became_peered":"2024-04-25T14:38:10.446498+0000","last_unstale":"2024-04-25T14:40:40.179623+0000","last_undegraded":"2024-04-25T14:40:40.179623+0000","last_fullsized":"2024-04-25T14:40:40.179623+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:14:37.179198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.482585+0000","last_change":"2024-04-25T14:38:09.544283+0000","last_active":"2024-04-25T14:40:40.482585+0000","last_peered":"2024-04-25T14:40:40.482585+0000","last_clean":"2024-04-25T14:40:40.482585+0000","last_became_active":"2024-04-25T14:38:09.543135+0000","last_became_peered":"2024-04-25T14:38:09.543135+0000","last_unstale":"2024-04-25T14:40:40.482585+0000","last_undegraded":"2024-04-25T14:40:40.482585+0000","last_fullsized":"2024-04-25T14:40:40.482585+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:05.977062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.179523+0000","last_change":"2024-04-25T14:38:10.432863+0000","last_active":"2024-04-25T14:40:40.179523+0000","last_peered":"2024-04-25T14:40:40.179523+0000","last_clean":"2024-04-25T14:40:40.179523+0000","last_became_active":"2024-04-25T14:38:10.432493+0000","last_became_peered":"2024-04-25T14:38:10.432493+0000","last_unstale":"2024-04-25T14:40:40.179523+0000","last_undegraded":"2024-04-25T14:40:40.179523+0000","last_fullsized":"2024-04-25T14:40:40.179523+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:25:53.774764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.180156+0000","last_change":"2024-04-25T14:39:15.981979+0000","last_active":"2024-04-25T14:40:40.180156+0000","last_peered":"2024-04-25T14:40:40.180156+0000","last_clean":"2024-04-25T14:40:40.180156+0000","last_became_active":"2024-04-25T14:39:15.981539+0000","last_became_peered":"2024-04-25T14:39:15.981539+0000","last_unstale":"2024-04-25T14:40:40.180156+0000","last_undegraded":"2024-04-25T14:40:40.180156+0000","last_fullsized":"2024-04-25T14:40:40.180156+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:22.906826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":188,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:49.987406+0000","last_change":"2024-04-25T14:38:09.554305+0000","last_active":"2024-04-25T14:40:49.987406+0000","last_peered":"2024-04-25T14:40:49.987406+0000","last_clean":"2024-04-25T14:40:49.987406+0000","last_became_active":"2024-04-25T14:38:09.553984+0000","last_became_peered":"2024-04-25T14:38:09.553984+0000","last_unstale":"2024-04-25T14:40:49.987406+0000","last_undegraded":"2024-04-25T14:40:49.987406+0000","last_fullsized":"2024-04-25T14:40:49.987406+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:41.580867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":156,"num_read_kb":156,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.181490+0000","last_change":"2024-04-25T14:38:10.448290+0000","last_active":"2024-04-25T14:40:40.181490+0000","last_peered":"2024-04-25T14:40:40.181490+0000","last_clean":"2024-04-25T14:40:40.181490+0000","last_became_active":"2024-04-25T14:38:10.446147+0000","last_became_peered":"2024-04-25T14:38:10.446147+0000","last_unstale":"2024-04-25T14:40:40.181490+0000","last_undegraded":"2024-04-25T14:40:40.181490+0000","last_fullsized":"2024-04-25T14:40:40.181490+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:12.775551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.481561+0000","last_change":"2024-04-25T14:38:09.545209+0000","last_active":"2024-04-25T14:40:40.481561+0000","last_peered":"2024-04-25T14:40:40.481561+0000","last_clean":"2024-04-25T14:40:40.481561+0000","last_became_active":"2024-04-25T14:38:09.544861+0000","last_became_peered":"2024-04-25T14:38:09.544861+0000","last_unstale":"2024-04-25T14:40:40.481561+0000","last_undegraded":"2024-04-25T14:40:40.481561+0000","last_fullsized":"2024-04-25T14:40:40.481561+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:25.829162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"93'148","reported_seq":404,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:42.316942+0000","last_change":"2024-04-25T14:36:48.250081+0000","last_active":"2024-04-25T14:40:42.316942+0000","last_peered":"2024-04-25T14:40:42.316942+0000","last_clean":"2024-04-25T14:40:42.316942+0000","last_became_active":"2024-04-25T14:36:48.249520+0000","last_became_peered":"2024-04-25T14:36:48.249520+0000","last_unstale":"2024-04-25T14:40:42.316942+0000","last_undegraded":"2024-04-25T14:40:42.316942+0000","last_fullsized":"2024-04-25T14:40:42.316942+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:33:03.061016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:33:03.061016+0000","last_clean_scrub_stamp":"2024-04-25T14:33:03.061016+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:04:06.653326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.676039+0000","last_change":"2024-04-25T14:40:16.498939+0000","last_active":"2024-04-25T14:40:40.676039+0000","last_peered":"2024-04-25T14:40:40.676039+0000","last_clean":"2024-04-25T14:40:40.676039+0000","last_became_active":"2024-04-25T14:40:16.498128+0000","last_became_peered":"2024-04-25T14:40:16.498128+0000","last_unstale":"2024-04-25T14:40:40.676039+0000","last_undegraded":"2024-04-25T14:40:40.676039+0000","last_fullsized":"2024-04-25T14:40:40.676039+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:14.942944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.481930+0000","last_change":"2024-04-25T14:39:15.982120+0000","last_active":"2024-04-25T14:40:40.481930+0000","last_peered":"2024-04-25T14:40:40.481930+0000","last_clean":"2024-04-25T14:40:40.481930+0000","last_became_active":"2024-04-25T14:39:15.975343+0000","last_became_peered":"2024-04-25T14:39:15.975343+0000","last_unstale":"2024-04-25T14:40:40.481930+0000","last_undegraded":"2024-04-25T14:40:40.481930+0000","last_fullsized":"2024-04-25T14:40:40.481930+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:38.632052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.650887+0000","last_change":"2024-04-25T14:39:15.998744+0000","last_active":"2024-04-25T14:40:40.650887+0000","last_peered":"2024-04-25T14:40:40.650887+0000","last_clean":"2024-04-25T14:40:40.650887+0000","last_became_active":"2024-04-25T14:39:15.993416+0000","last_became_peered":"2024-04-25T14:39:15.993416+0000","last_unstale":"2024-04-25T14:40:40.650887+0000","last_undegraded":"2024-04-25T14:40:40.650887+0000","last_fullsized":"2024-04-25T14:40:40.650887+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:24.019707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.676812+0000","last_change":"2024-04-25T14:38:10.429554+0000","last_active":"2024-04-25T14:40:40.676812+0000","last_peered":"2024-04-25T14:40:40.676812+0000","last_clean":"2024-04-25T14:40:40.676812+0000","last_became_active":"2024-04-25T14:38:10.428944+0000","last_became_peered":"2024-04-25T14:38:10.428944+0000","last_unstale":"2024-04-25T14:40:40.676812+0000","last_undegraded":"2024-04-25T14:40:40.676812+0000","last_fullsized":"2024-04-25T14:40:40.676812+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:38.013028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"93'100","reported_seq":218,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:43.323542+0000","last_change":"2024-04-25T14:38:09.564849+0000","last_active":"2024-04-25T14:40:43.323542+0000","last_peered":"2024-04-25T14:40:43.323542+0000","last_clean":"2024-04-25T14:40:43.323542+0000","last_became_active":"2024-04-25T14:38:09.564354+0000","last_became_peered":"2024-04-25T14:38:09.564354+0000","last_unstale":"2024-04-25T14:40:43.323542+0000","last_undegraded":"2024-04-25T14:40:43.323542+0000","last_fullsized":"2024-04-25T14:40:43.323542+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:44.308366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":183,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.181997+0000","last_change":"2024-04-25T14:38:10.438178+0000","last_active":"2024-04-25T14:40:40.181997+0000","last_peered":"2024-04-25T14:40:40.181997+0000","last_clean":"2024-04-25T14:40:40.181997+0000","last_became_active":"2024-04-25T14:38:10.429936+0000","last_became_peered":"2024-04-25T14:38:10.429936+0000","last_unstale":"2024-04-25T14:40:40.181997+0000","last_undegraded":"2024-04-25T14:40:40.181997+0000","last_fullsized":"2024-04-25T14:40:40.181997+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:33.598747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.481640+0000","last_change":"2024-04-25T14:38:09.547680+0000","last_active":"2024-04-25T14:40:40.481640+0000","last_peered":"2024-04-25T14:40:40.481640+0000","last_clean":"2024-04-25T14:40:40.481640+0000","last_became_active":"2024-04-25T14:38:09.547337+0000","last_became_peered":"2024-04-25T14:38:09.547337+0000","last_unstale":"2024-04-25T14:40:40.481640+0000","last_undegraded":"2024-04-25T14:40:40.481640+0000","last_fullsized":"2024-04-25T14:40:40.481640+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:32.754439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.991670+0000","last_change":"2024-04-25T14:38:10.447115+0000","last_active":"2024-04-25T14:40:40.991670+0000","last_peered":"2024-04-25T14:40:40.991670+0000","last_clean":"2024-04-25T14:40:40.991670+0000","last_became_active":"2024-04-25T14:38:10.446681+0000","last_became_peered":"2024-04-25T14:38:10.446681+0000","last_unstale":"2024-04-25T14:40:40.991670+0000","last_undegraded":"2024-04-25T14:40:40.991670+0000","last_fullsized":"2024-04-25T14:40:40.991670+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:02:58.942053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.224061+0000","last_change":"2024-04-25T14:38:09.551523+0000","last_active":"2024-04-25T14:40:40.224061+0000","last_peered":"2024-04-25T14:40:40.224061+0000","last_clean":"2024-04-25T14:40:40.224061+0000","last_became_active":"2024-04-25T14:38:09.551213+0000","last_became_peered":"2024-04-25T14:38:09.551213+0000","last_unstale":"2024-04-25T14:40:40.224061+0000","last_undegraded":"2024-04-25T14:40:40.224061+0000","last_fullsized":"2024-04-25T14:40:40.224061+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:48.948571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.304011+0000","last_change":"2024-04-25T14:38:10.420032+0000","last_active":"2024-04-25T14:40:40.304011+0000","last_peered":"2024-04-25T14:40:40.304011+0000","last_clean":"2024-04-25T14:40:40.304011+0000","last_became_active":"2024-04-25T14:38:10.418349+0000","last_became_peered":"2024-04-25T14:38:10.418349+0000","last_unstale":"2024-04-25T14:40:40.304011+0000","last_undegraded":"2024-04-25T14:40:40.304011+0000","last_fullsized":"2024-04-25T14:40:40.304011+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:38:55.669414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.181847+0000","last_change":"2024-04-25T14:38:09.567097+0000","last_active":"2024-04-25T14:40:40.181847+0000","last_peered":"2024-04-25T14:40:40.181847+0000","last_clean":"2024-04-25T14:40:40.181847+0000","last_became_active":"2024-04-25T14:38:09.566852+0000","last_became_peered":"2024-04-25T14:38:09.566852+0000","last_unstale":"2024-04-25T14:40:40.181847+0000","last_undegraded":"2024-04-25T14:40:40.181847+0000","last_fullsized":"2024-04-25T14:40:40.181847+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:41:55.773005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.650673+0000","last_change":"2024-04-25T14:38:10.432691+0000","last_active":"2024-04-25T14:40:40.650673+0000","last_peered":"2024-04-25T14:40:40.650673+0000","last_clean":"2024-04-25T14:40:40.650673+0000","last_became_active":"2024-04-25T14:38:10.432321+0000","last_became_peered":"2024-04-25T14:38:10.432321+0000","last_unstale":"2024-04-25T14:40:40.650673+0000","last_undegraded":"2024-04-25T14:40:40.650673+0000","last_fullsized":"2024-04-25T14:40:40.650673+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:47:58.026331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.224545+0000","last_change":"2024-04-25T14:38:09.563692+0000","last_active":"2024-04-25T14:40:40.224545+0000","last_peered":"2024-04-25T14:40:40.224545+0000","last_clean":"2024-04-25T14:40:40.224545+0000","last_became_active":"2024-04-25T14:38:09.563345+0000","last_became_peered":"2024-04-25T14:38:09.563345+0000","last_unstale":"2024-04-25T14:40:40.224545+0000","last_undegraded":"2024-04-25T14:40:40.224545+0000","last_fullsized":"2024-04-25T14:40:40.224545+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:43.221429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.182980+0000","last_change":"2024-04-25T14:38:10.442519+0000","last_active":"2024-04-25T14:40:40.182980+0000","last_peered":"2024-04-25T14:40:40.182980+0000","last_clean":"2024-04-25T14:40:40.182980+0000","last_became_active":"2024-04-25T14:38:10.439045+0000","last_became_peered":"2024-04-25T14:38:10.439045+0000","last_unstale":"2024-04-25T14:40:40.182980+0000","last_undegraded":"2024-04-25T14:40:40.182980+0000","last_fullsized":"2024-04-25T14:40:40.182980+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:42.425106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.304120+0000","last_change":"2024-04-25T14:38:09.550363+0000","last_active":"2024-04-25T14:40:40.304120+0000","last_peered":"2024-04-25T14:40:40.304120+0000","last_clean":"2024-04-25T14:40:40.304120+0000","last_became_active":"2024-04-25T14:38:09.550044+0000","last_became_peered":"2024-04-25T14:38:09.550044+0000","last_unstale":"2024-04-25T14:40:40.304120+0000","last_undegraded":"2024-04-25T14:40:40.304120+0000","last_fullsized":"2024-04-25T14:40:40.304120+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:25.573353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.482361+0000","last_change":"2024-04-25T14:38:10.428736+0000","last_active":"2024-04-25T14:40:40.482361+0000","last_peered":"2024-04-25T14:40:40.482361+0000","last_clean":"2024-04-25T14:40:40.482361+0000","last_became_active":"2024-04-25T14:38:10.428410+0000","last_became_peered":"2024-04-25T14:38:10.428410+0000","last_unstale":"2024-04-25T14:40:40.482361+0000","last_undegraded":"2024-04-25T14:40:40.482361+0000","last_fullsized":"2024-04-25T14:40:40.482361+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:02:51.474220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.224711+0000","last_change":"2024-04-25T14:38:09.552800+0000","last_active":"2024-04-25T14:40:40.224711+0000","last_peered":"2024-04-25T14:40:40.224711+0000","last_clean":"2024-04-25T14:40:40.224711+0000","last_became_active":"2024-04-25T14:38:09.552460+0000","last_became_peered":"2024-04-25T14:38:09.552460+0000","last_unstale":"2024-04-25T14:40:40.224711+0000","last_undegraded":"2024-04-25T14:40:40.224711+0000","last_fullsized":"2024-04-25T14:40:40.224711+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:00.800770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.482907+0000","last_change":"2024-04-25T14:38:10.424359+0000","last_active":"2024-04-25T14:40:40.482907+0000","last_peered":"2024-04-25T14:40:40.482907+0000","last_clean":"2024-04-25T14:40:40.482907+0000","last_became_active":"2024-04-25T14:38:10.424020+0000","last_became_peered":"2024-04-25T14:38:10.424020+0000","last_unstale":"2024-04-25T14:40:40.482907+0000","last_undegraded":"2024-04-25T14:40:40.482907+0000","last_fullsized":"2024-04-25T14:40:40.482907+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:14.700772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"93'65","reported_seq":122,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:42.853177+0000","last_change":"2024-04-25T14:38:09.545637+0000","last_active":"2024-04-25T14:40:42.853177+0000","last_peered":"2024-04-25T14:40:42.853177+0000","last_clean":"2024-04-25T14:40:42.853177+0000","last_became_active":"2024-04-25T14:38:09.545319+0000","last_became_peered":"2024-04-25T14:38:09.545319+0000","last_unstale":"2024-04-25T14:40:42.853177+0000","last_undegraded":"2024-04-25T14:40:42.853177+0000","last_fullsized":"2024-04-25T14:40:42.853177+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:02.284715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.482427+0000","last_change":"2024-04-25T14:38:10.446928+0000","last_active":"2024-04-25T14:40:40.482427+0000","last_peered":"2024-04-25T14:40:40.482427+0000","last_clean":"2024-04-25T14:40:40.482427+0000","last_became_active":"2024-04-25T14:38:10.446518+0000","last_became_peered":"2024-04-25T14:38:10.446518+0000","last_unstale":"2024-04-25T14:40:40.482427+0000","last_undegraded":"2024-04-25T14:40:40.482427+0000","last_fullsized":"2024-04-25T14:40:40.482427+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:40.769059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.677657+0000","last_change":"2024-04-25T14:38:09.557381+0000","last_active":"2024-04-25T14:40:40.677657+0000","last_peered":"2024-04-25T14:40:40.677657+0000","last_clean":"2024-04-25T14:40:40.677657+0000","last_became_active":"2024-04-25T14:38:09.556504+0000","last_became_peered":"2024-04-25T14:38:09.556504+0000","last_unstale":"2024-04-25T14:40:40.677657+0000","last_undegraded":"2024-04-25T14:40:40.677657+0000","last_fullsized":"2024-04-25T14:40:40.677657+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:08.505010+0000","last_clean_scrub_stamp":"2024-04-25T14:38:08.505010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:01:02.761249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.482978+0000","last_change":"2024-04-25T14:38:10.418104+0000","last_active":"2024-04-25T14:40:40.482978+0000","last_peered":"2024-04-25T14:40:40.482978+0000","last_clean":"2024-04-25T14:40:40.482978+0000","last_became_active":"2024-04-25T14:38:10.417774+0000","last_became_peered":"2024-04-25T14:38:10.417774+0000","last_unstale":"2024-04-25T14:40:40.482978+0000","last_undegraded":"2024-04-25T14:40:40.482978+0000","last_fullsized":"2024-04-25T14:40:40.482978+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:32:44.229899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.650780+0000","last_change":"2024-04-25T14:38:10.423546+0000","last_active":"2024-04-25T14:40:40.650780+0000","last_peered":"2024-04-25T14:40:40.650780+0000","last_clean":"2024-04-25T14:40:40.650780+0000","last_became_active":"2024-04-25T14:38:10.423216+0000","last_became_peered":"2024-04-25T14:38:10.423216+0000","last_unstale":"2024-04-25T14:40:40.650780+0000","last_undegraded":"2024-04-25T14:40:40.650780+0000","last_fullsized":"2024-04-25T14:40:40.650780+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:58.640669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.224609+0000","last_change":"2024-04-25T14:38:10.422504+0000","last_active":"2024-04-25T14:40:40.224609+0000","last_peered":"2024-04-25T14:40:40.224609+0000","last_clean":"2024-04-25T14:40:40.224609+0000","last_became_active":"2024-04-25T14:38:10.422095+0000","last_became_peered":"2024-04-25T14:38:10.422095+0000","last_unstale":"2024-04-25T14:40:40.224609+0000","last_undegraded":"2024-04-25T14:40:40.224609+0000","last_fullsized":"2024-04-25T14:40:40.224609+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:25.795962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.224467+0000","last_change":"2024-04-25T14:38:10.447446+0000","last_active":"2024-04-25T14:40:40.224467+0000","last_peered":"2024-04-25T14:40:40.224467+0000","last_clean":"2024-04-25T14:40:40.224467+0000","last_became_active":"2024-04-25T14:38:10.447056+0000","last_became_peered":"2024-04-25T14:38:10.447056+0000","last_unstale":"2024-04-25T14:40:40.224467+0000","last_undegraded":"2024-04-25T14:40:40.224467+0000","last_fullsized":"2024-04-25T14:40:40.224467+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:40.462340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.990840+0000","last_change":"2024-04-25T14:38:10.433444+0000","last_active":"2024-04-25T14:40:40.990840+0000","last_peered":"2024-04-25T14:40:40.990840+0000","last_clean":"2024-04-25T14:40:40.990840+0000","last_became_active":"2024-04-25T14:38:10.433153+0000","last_became_peered":"2024-04-25T14:38:10.433153+0000","last_unstale":"2024-04-25T14:40:40.990840+0000","last_undegraded":"2024-04-25T14:40:40.990840+0000","last_fullsized":"2024-04-25T14:40:40.990840+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:10.722650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.676722+0000","last_change":"2024-04-25T14:38:10.438322+0000","last_active":"2024-04-25T14:40:40.676722+0000","last_peered":"2024-04-25T14:40:40.676722+0000","last_clean":"2024-04-25T14:40:40.676722+0000","last_became_active":"2024-04-25T14:38:10.436103+0000","last_became_peered":"2024-04-25T14:38:10.436103+0000","last_unstale":"2024-04-25T14:40:40.676722+0000","last_undegraded":"2024-04-25T14:40:40.676722+0000","last_fullsized":"2024-04-25T14:40:40.676722+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:16.790202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.650995+0000","last_change":"2024-04-25T14:38:10.419710+0000","last_active":"2024-04-25T14:40:40.650995+0000","last_peered":"2024-04-25T14:40:40.650995+0000","last_clean":"2024-04-25T14:40:40.650995+0000","last_became_active":"2024-04-25T14:38:10.419406+0000","last_became_peered":"2024-04-25T14:38:10.419406+0000","last_unstale":"2024-04-25T14:40:40.650995+0000","last_undegraded":"2024-04-25T14:40:40.650995+0000","last_fullsized":"2024-04-25T14:40:40.650995+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:37.964698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.224748+0000","last_change":"2024-04-25T14:38:10.429510+0000","last_active":"2024-04-25T14:40:40.224748+0000","last_peered":"2024-04-25T14:40:40.224748+0000","last_clean":"2024-04-25T14:40:40.224748+0000","last_became_active":"2024-04-25T14:38:10.429219+0000","last_became_peered":"2024-04-25T14:38:10.429219+0000","last_unstale":"2024-04-25T14:40:40.224748+0000","last_undegraded":"2024-04-25T14:40:40.224748+0000","last_fullsized":"2024-04-25T14:40:40.224748+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:52:46.673126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.181877+0000","last_change":"2024-04-25T14:38:10.441487+0000","last_active":"2024-04-25T14:40:40.181877+0000","last_peered":"2024-04-25T14:40:40.181877+0000","last_clean":"2024-04-25T14:40:40.181877+0000","last_became_active":"2024-04-25T14:38:10.441111+0000","last_became_peered":"2024-04-25T14:38:10.441111+0000","last_unstale":"2024-04-25T14:40:40.181877+0000","last_undegraded":"2024-04-25T14:40:40.181877+0000","last_fullsized":"2024-04-25T14:40:40.181877+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:59.842687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.179847+0000","last_change":"2024-04-25T14:38:10.436511+0000","last_active":"2024-04-25T14:40:40.179847+0000","last_peered":"2024-04-25T14:40:40.179847+0000","last_clean":"2024-04-25T14:40:40.179847+0000","last_became_active":"2024-04-25T14:38:10.436171+0000","last_became_peered":"2024-04-25T14:38:10.436171+0000","last_unstale":"2024-04-25T14:40:40.179847+0000","last_undegraded":"2024-04-25T14:40:40.179847+0000","last_fullsized":"2024-04-25T14:40:40.179847+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:27:19.605907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.181672+0000","last_change":"2024-04-25T14:38:10.433624+0000","last_active":"2024-04-25T14:40:40.181672+0000","last_peered":"2024-04-25T14:40:40.181672+0000","last_clean":"2024-04-25T14:40:40.181672+0000","last_became_active":"2024-04-25T14:38:10.433258+0000","last_became_peered":"2024-04-25T14:38:10.433258+0000","last_unstale":"2024-04-25T14:40:40.181672+0000","last_undegraded":"2024-04-25T14:40:40.181672+0000","last_fullsized":"2024-04-25T14:40:40.181672+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:53.563110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.224234+0000","last_change":"2024-04-25T14:38:10.446924+0000","last_active":"2024-04-25T14:40:40.224234+0000","last_peered":"2024-04-25T14:40:40.224234+0000","last_clean":"2024-04-25T14:40:40.224234+0000","last_became_active":"2024-04-25T14:38:10.446553+0000","last_became_peered":"2024-04-25T14:38:10.446553+0000","last_unstale":"2024-04-25T14:40:40.224234+0000","last_undegraded":"2024-04-25T14:40:40.224234+0000","last_fullsized":"2024-04-25T14:40:40.224234+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:00.203796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.677384+0000","last_change":"2024-04-25T14:38:10.429952+0000","last_active":"2024-04-25T14:40:40.677384+0000","last_peered":"2024-04-25T14:40:40.677384+0000","last_clean":"2024-04-25T14:40:40.677384+0000","last_became_active":"2024-04-25T14:38:10.429626+0000","last_became_peered":"2024-04-25T14:38:10.429626+0000","last_unstale":"2024-04-25T14:40:40.677384+0000","last_undegraded":"2024-04-25T14:40:40.677384+0000","last_fullsized":"2024-04-25T14:40:40.677384+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:07.287113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T14:40:40.650742+0000","last_change":"2024-04-25T14:38:10.418292+0000","last_active":"2024-04-25T14:40:40.650742+0000","last_peered":"2024-04-25T14:40:40.650742+0000","last_clean":"2024-04-25T14:40:40.650742+0000","last_became_active":"2024-04-25T14:38:10.418005+0000","last_became_peered":"2024-04-25T14:38:10.418005+0000","last_unstale":"2024-04-25T14:40:40.650742+0000","last_undegraded":"2024-04-25T14:40:40.650742+0000","last_fullsized":"2024-04-25T14:40:40.650742+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:38:09.521071+0000","last_clean_scrub_stamp":"2024-04-25T14:38:09.521071+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:25.486993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":795824,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1314,"num_read_kb":874,"num_write":420,"num_write_kb":2356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1035,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2617344,"data_stored":2387472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":310,"ondisk_log_size":310,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285614,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1036288,"data_stored":495726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547251,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29156,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93715996,"statfs":{"total":95995035648,"available":95965179904,"internally_reserved":0,"allocated":2068480,"data_stored":1485260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710776,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28424,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716728,"statfs":{"total":95995035648,"available":95965929472,"internally_reserved":0,"allocated":1441792,"data_stored":889156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972415,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36740,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708412,"statfs":{"total":95995035648,"available":95957413888,"internally_reserved":0,"allocated":1437696,"data_stored":889054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266755,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28624,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716528,"statfs":{"total":95995035648,"available":95965724672,"internally_reserved":0,"allocated":1642496,"data_stored":1089555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430279,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":495329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574947,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1105920,"data_stored":563732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738473,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1056768,"data_stored":496600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411424,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36436,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708716,"statfs":{"total":95995035648,"available":95957725184,"internally_reserved":0,"allocated":1122304,"data_stored":564460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149785,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28872,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716280,"statfs":{"total":95995035648,"available":95965470720,"internally_reserved":0,"allocated":1777664,"data_stored":1217088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888150,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1048576,"data_stored":498544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691919,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37396,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707756,"statfs":{"total":95995035648,"available":95956742144,"internally_reserved":0,"allocated":1986560,"data_stored":1416238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":69338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":741376,"data_stored":721732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":360448,"data_stored":330631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":399653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:40:55.315 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:40:55.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.579+0000 7f698fe00640 1 -- 172.21.15.53:0/3644299674 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69901041c0 msgr2=0x7f69901045a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:55.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.579+0000 7f698fe00640 1 --2- 172.21.15.53:0/3644299674 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69901041c0 0x7f69901045a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6980009860 tx=0x7f698002a340 comp rx=0 tx=0).stop 2024-04-25T14:40:55.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.579+0000 7f698fe00640 1 -- 172.21.15.53:0/3644299674 shutdown_connections 2024-04-25T14:40:55.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.579+0000 7f698fe00640 1 --2- 172.21.15.53:0/3644299674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6990109290 0x7f6990110e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:55.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.579+0000 7f698fe00640 1 --2- 172.21.15.53:0/3644299674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6990104b70 0x7f6990108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:55.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.579+0000 7f698fe00640 1 --2- 172.21.15.53:0/3644299674 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69901041c0 0x7f69901045a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:55.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.579+0000 7f698fe00640 1 -- 172.21.15.53:0/3644299674 >> 172.21.15.53:0/3644299674 conn(0x7f69900ff9e0 msgr2=0x7f6990101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:55.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.579+0000 7f698fe00640 1 -- 172.21.15.53:0/3644299674 shutdown_connections 2024-04-25T14:40:55.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.579+0000 7f698fe00640 1 -- 172.21.15.53:0/3644299674 wait complete. 2024-04-25T14:40:55.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.580+0000 7f698fe00640 1 Processor -- start 2024-04-25T14:40:55.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.581+0000 7f698fe00640 1 -- start start 2024-04-25T14:40:55.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.581+0000 7f698fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69901041c0 0x7f69901a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:55.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.581+0000 7f698fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6990109290 0x7f69901a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:55.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.581+0000 7f698fe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f699019a5e0 0x7f699019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:55.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.581+0000 7f698fe00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6990114200 con 0x7f69901041c0 2024-04-25T14:40:55.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.581+0000 7f698fe00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6990114080 con 0x7f699019a5e0 2024-04-25T14:40:55.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.581+0000 7f698fe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6990114380 con 0x7f6990109290 2024-04-25T14:40:55.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.581+0000 7f698ea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69901041c0 0x7f69901a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:55.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.581+0000 7f698e000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6990109290 0x7f69901a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:55.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.581+0000 7f698ea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69901041c0 0x7f69901a0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42134/0 (socket says 172.21.15.53:42134) 2024-04-25T14:40:55.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.581+0000 7f698ea00640 1 -- 172.21.15.53:0/2177429714 learned_addr learned my addr 172.21.15.53:0/2177429714 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:55.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.581+0000 7f698f400640 1 --2- 172.21.15.53:0/2177429714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f699019a5e0 0x7f699019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:55.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.582+0000 7f698ea00640 1 -- 172.21.15.53:0/2177429714 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6990109290 msgr2=0x7f69901a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:55.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.582+0000 7f698ea00640 1 --2- 172.21.15.53:0/2177429714 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6990109290 0x7f69901a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:55.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.582+0000 7f698ea00640 1 -- 172.21.15.53:0/2177429714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f699019a5e0 msgr2=0x7f699019aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:55.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.582+0000 7f698ea00640 1 --2- 172.21.15.53:0/2177429714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f699019a5e0 0x7f699019aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:55.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.582+0000 7f698ea00640 1 -- 172.21.15.53:0/2177429714 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f699019b100 con 0x7f69901041c0 2024-04-25T14:40:55.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.582+0000 7f698f400640 1 --2- 172.21.15.53:0/2177429714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f699019a5e0 0x7f699019aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:40:55.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.582+0000 7f698ea00640 1 --2- 172.21.15.53:0/2177429714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69901041c0 0x7f69901a0430 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6980035040 tx=0x7f6980032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:55.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.584+0000 7f6987400640 1 -- 172.21.15.53:0/2177429714 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6980037040 con 0x7f69901041c0 2024-04-25T14:40:55.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.584+0000 7f6987400640 1 -- 172.21.15.53:0/2177429714 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6980041070 con 0x7f69901041c0 2024-04-25T14:40:55.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.584+0000 7f698fe00640 1 -- 172.21.15.53:0/2177429714 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f699019b390 con 0x7f69901041c0 2024-04-25T14:40:55.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.584+0000 7f6987400640 1 -- 172.21.15.53:0/2177429714 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69800035f0 con 0x7f69901041c0 2024-04-25T14:40:55.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.584+0000 7f698fe00640 1 -- 172.21.15.53:0/2177429714 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69901a7580 con 0x7f69901041c0 2024-04-25T14:40:55.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.585+0000 7f698fe00640 1 -- 172.21.15.53:0/2177429714 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69900690e0 con 0x7f69901041c0 2024-04-25T14:40:55.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.587+0000 7f6987400640 1 -- 172.21.15.53:0/2177429714 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7f6980004050 con 0x7f69901041c0 2024-04-25T14:40:55.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.589+0000 7f6987400640 1 --2- 172.21.15.53:0/2177429714 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f69740a3e30 0x7f69740a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:55.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.589+0000 7f6987400640 1 -- 172.21.15.53:0/2177429714 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8543+0+0 (secure 0 0 0) 0x7f69800f0360 con 0x7f69901041c0 2024-04-25T14:40:55.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.590+0000 7f698e000640 1 --2- 172.21.15.53:0/2177429714 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f69740a3e30 0x7f69740a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:55.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.590+0000 7f698e000640 1 --2- 172.21.15.53:0/2177429714 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f69740a3e30 0x7f69740a62f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6990103170 tx=0x7f6978009260 comp rx=0 tx=0).ready entity=mgr.15093 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:55.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.594+0000 7f6987400640 1 -- 172.21.15.53:0/2177429714 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f69800bc930 con 0x7f69901041c0 2024-04-25T14:40:55.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.803+0000 7f698fe00640 1 -- 172.21.15.53:0/2177429714 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f699010c4c0 con 0x7f69901041c0 2024-04-25T14:40:55.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.805+0000 7f6987400640 1 -- 172.21.15.53:0/2177429714 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+23108 (secure 0 0 0) 0x7f69800bc230 con 0x7f69901041c0 2024-04-25T14:40:55.806 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:40:55.806 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":93,"fsid":"fd30e6ce-030f-11ef-bc93-c7b262605968","created":"2024-04-25T14:28:54.330326+0000","modified":"2024-04-25T14:40:40.169020+0000","last_up_change":"2024-04-25T14:37:10.921148+0000","last_in_change":"2024-04-25T14:36:49.566459+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:33:02.338115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:38:07.517974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:38:08.994122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aecfbaf0-39da-4749-a49b-c7205dabd4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6803","nonce":464176015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6805","nonce":464176015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6809","nonce":464176015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6807","nonce":464176015}]},"public_addr":"172.21.15.53:6803/464176015","cluster_addr":"172.21.15.53:6805/464176015","heartbeat_back_addr":"172.21.15.53:6809/464176015","heartbeat_front_addr":"172.21.15.53:6807/464176015","state":["exists","up"]},{"osd":1,"uuid":"2c4a21ae-c43e-4839-8a56-c3fc63aeb7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6801","nonce":1304621982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6803","nonce":1304621982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6807","nonce":1304621982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6805","nonce":1304621982}]},"public_addr":"172.21.15.150:6801/1304621982","cluster_addr":"172.21.15.150:6803/1304621982","heartbeat_back_addr":"172.21.15.150:6807/1304621982","heartbeat_front_addr":"172.21.15.150:6805/1304621982","state":["exists","up"]},{"osd":2,"uuid":"e8effd43-adc0-472a-9eed-a9273b06dc0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6801","nonce":574509311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6803","nonce":574509311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6807","nonce":574509311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6805","nonce":574509311}]},"public_addr":"172.21.15.183:6801/574509311","cluster_addr":"172.21.15.183:6803/574509311","heartbeat_back_addr":"172.21.15.183:6807/574509311","heartbeat_front_addr":"172.21.15.183:6805/574509311","state":["exists","up"]},{"osd":3,"uuid":"e7cfced4-9b40-40de-a303-75ea8d5df4a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6811","nonce":570672219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6813","nonce":570672219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6817","nonce":570672219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6815","nonce":570672219}]},"public_addr":"172.21.15.53:6811/570672219","cluster_addr":"172.21.15.53:6813/570672219","heartbeat_back_addr":"172.21.15.53:6817/570672219","heartbeat_front_addr":"172.21.15.53:6815/570672219","state":["exists","up"]},{"osd":4,"uuid":"44a9e070-2282-43f1-9f05-b1f8ff81e540","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6809","nonce":1526850103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6811","nonce":1526850103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6815","nonce":1526850103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6813","nonce":1526850103}]},"public_addr":"172.21.15.150:6809/1526850103","cluster_addr":"172.21.15.150:6811/1526850103","heartbeat_back_addr":"172.21.15.150:6815/1526850103","heartbeat_front_addr":"172.21.15.150:6813/1526850103","state":["exists","up"]},{"osd":5,"uuid":"2c5619d2-c0f9-47d6-bf7e-2f4afd1b09f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6809","nonce":773726576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6811","nonce":773726576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6815","nonce":773726576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6813","nonce":773726576}]},"public_addr":"172.21.15.183:6809/773726576","cluster_addr":"172.21.15.183:6811/773726576","heartbeat_back_addr":"172.21.15.183:6815/773726576","heartbeat_front_addr":"172.21.15.183:6813/773726576","state":["exists","up"]},{"osd":6,"uuid":"1cf74d94-c118-4738-b6db-040af458a8eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6819","nonce":355884319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6821","nonce":355884319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6825","nonce":355884319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6823","nonce":355884319}]},"public_addr":"172.21.15.53:6819/355884319","cluster_addr":"172.21.15.53:6821/355884319","heartbeat_back_addr":"172.21.15.53:6825/355884319","heartbeat_front_addr":"172.21.15.53:6823/355884319","state":["exists","up"]},{"osd":7,"uuid":"0ce97c04-cd41-4080-bbbc-be9cf5fadc02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6817","nonce":229022562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6819","nonce":229022562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6823","nonce":229022562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6821","nonce":229022562}]},"public_addr":"172.21.15.150:6817/229022562","cluster_addr":"172.21.15.150:6819/229022562","heartbeat_back_addr":"172.21.15.150:6823/229022562","heartbeat_front_addr":"172.21.15.150:6821/229022562","state":["exists","up"]},{"osd":8,"uuid":"5d57787d-2a7c-4514-bd01-357ced413f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6817","nonce":301003844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6819","nonce":301003844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6823","nonce":301003844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6821","nonce":301003844}]},"public_addr":"172.21.15.183:6817/301003844","cluster_addr":"172.21.15.183:6819/301003844","heartbeat_back_addr":"172.21.15.183:6823/301003844","heartbeat_front_addr":"172.21.15.183:6821/301003844","state":["exists","up"]},{"osd":9,"uuid":"e11dd8eb-c55a-4d70-8c58-457540b7a50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6827","nonce":3449260978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6829","nonce":3449260978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6833","nonce":3449260978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6831","nonce":3449260978}]},"public_addr":"172.21.15.53:6827/3449260978","cluster_addr":"172.21.15.53:6829/3449260978","heartbeat_back_addr":"172.21.15.53:6833/3449260978","heartbeat_front_addr":"172.21.15.53:6831/3449260978","state":["exists","up"]},{"osd":10,"uuid":"5de4803d-ad37-4a90-add5-0048f44fc4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6825","nonce":3435992308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6827","nonce":3435992308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6831","nonce":3435992308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6829","nonce":3435992308}]},"public_addr":"172.21.15.150:6825/3435992308","cluster_addr":"172.21.15.150:6827/3435992308","heartbeat_back_addr":"172.21.15.150:6831/3435992308","heartbeat_front_addr":"172.21.15.150:6829/3435992308","state":["exists","up"]},{"osd":11,"uuid":"0e819a68-ed56-477e-b9c0-81e8cc0bce11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6825","nonce":1957842686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6827","nonce":1957842686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6831","nonce":1957842686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6829","nonce":1957842686}]},"public_addr":"172.21.15.183:6825/1957842686","cluster_addr":"172.21.15.183:6827/1957842686","heartbeat_back_addr":"172.21.15.183:6831/1957842686","heartbeat_front_addr":"172.21.15.183:6829/1957842686","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:09.840629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:34.769537+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:58.112969+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:33:29.656790+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:33:55.181287+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:34:19.772645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:34:53.833921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:35:19.076733+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:35:43.163390+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:36:16.568340+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:36:42.492768+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:37:07.944639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"3.38","mappings":[{"from":5,"to":7},{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6800/4154933108":"2024-04-26T14:40:40.168940+0000","172.21.15.53:0/3916258673":"2024-04-26T14:40:40.168940+0000","172.21.15.53:0/309091034":"2024-04-26T14:40:40.168940+0000","172.21.15.53:0/4267178590":"2024-04-26T14:40:40.168940+0000","172.21.15.53:0/920465993":"2024-04-26T14:40:40.168940+0000","172.21.15.53:6800/3887954889":"2024-04-26T14:30:14.162383+0000","172.21.15.53:6801/4154933108":"2024-04-26T14:40:40.168940+0000","172.21.15.53:0/1014501022":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/2576091441":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/1233527569":"2024-04-26T14:29:38.106242+0000","172.21.15.53:6801/3887954889":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/931889810":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/177196141":"2024-04-26T14:30:14.162383+0000","172.21.15.53:6800/1722497390":"2024-04-26T14:29:38.106242+0000","172.21.15.53:6801/1722497390":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/1927549198":"2024-04-26T14:30:14.162383+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:40:55.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.808+0000 7f698fe00640 1 -- 172.21.15.53:0/2177429714 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f69740a3e30 msgr2=0x7f69740a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:55.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.808+0000 7f698fe00640 1 --2- 172.21.15.53:0/2177429714 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f69740a3e30 0x7f69740a62f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6990103170 tx=0x7f6978009260 comp rx=0 tx=0).stop 2024-04-25T14:40:55.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.808+0000 7f698fe00640 1 -- 172.21.15.53:0/2177429714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69901041c0 msgr2=0x7f69901a0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:55.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.808+0000 7f698fe00640 1 --2- 172.21.15.53:0/2177429714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69901041c0 0x7f69901a0430 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6980035040 tx=0x7f6980032d50 comp rx=0 tx=0).stop 2024-04-25T14:40:55.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.808+0000 7f698fe00640 1 -- 172.21.15.53:0/2177429714 shutdown_connections 2024-04-25T14:40:55.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.808+0000 7f698fe00640 1 --2- 172.21.15.53:0/2177429714 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f69740a3e30 0x7f69740a62f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:55.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.809+0000 7f698fe00640 1 --2- 172.21.15.53:0/2177429714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f699019a5e0 0x7f699019aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:55.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.809+0000 7f698fe00640 1 --2- 172.21.15.53:0/2177429714 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6990109290 0x7f69901a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:55.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.809+0000 7f698fe00640 1 --2- 172.21.15.53:0/2177429714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69901041c0 0x7f69901a0430 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:55.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.809+0000 7f698fe00640 1 -- 172.21.15.53:0/2177429714 >> 172.21.15.53:0/2177429714 conn(0x7f69900ff9e0 msgr2=0x7f6990100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:55.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.809+0000 7f698fe00640 1 -- 172.21.15.53:0/2177429714 shutdown_connections 2024-04-25T14:40:55.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:55.809+0000 7f698fe00640 1 -- 172.21.15.53:0/2177429714 wait complete. 2024-04-25T14:40:55.828 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:33:02.338115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:38:07.517974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:38:08.994122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:40:55.829 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:40:55.905 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:55 smithi053 ceph-mon[27773]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 76 KiB/s wr, 46 op/s 2024-04-25T14:40:55.905 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:55 smithi053 ceph-mon[27773]: from='client.25313 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:40:55.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:55 smithi183 ceph-mon[32584]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 76 KiB/s wr, 46 op/s 2024-04-25T14:40:55.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:55 smithi183 ceph-mon[32584]: from='client.25313 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:40:55.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:55 smithi150 ceph-mon[32504]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 76 KiB/s wr, 46 op/s 2024-04-25T14:40:55.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:55 smithi150 ceph-mon[32504]: from='client.25313 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:40:56.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.050+0000 7fa711a00640 1 -- 172.21.15.53:0/3656662198 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa70c108e90 msgr2=0x7fa70c1152b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:56.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.050+0000 7fa711a00640 1 --2- 172.21.15.53:0/3656662198 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa70c108e90 0x7fa70c1152b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa704009ca0 tx=0x7fa70402a950 comp rx=0 tx=0).stop 2024-04-25T14:40:56.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.051+0000 7fa711a00640 1 -- 172.21.15.53:0/3656662198 shutdown_connections 2024-04-25T14:40:56.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.051+0000 7fa711a00640 1 --2- 172.21.15.53:0/3656662198 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa70c108e90 0x7fa70c1152b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.051+0000 7fa711a00640 1 --2- 172.21.15.53:0/3656662198 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa70c1084f0 0x7fa70c108950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.051+0000 7fa711a00640 1 --2- 172.21.15.53:0/3656662198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa70c10d3c0 0x7fa70c10d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.051+0000 7fa711a00640 1 -- 172.21.15.53:0/3656662198 >> 172.21.15.53:0/3656662198 conn(0x7fa70c077d70 msgr2=0x7fa70c106d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:56.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.051+0000 7fa711a00640 1 -- 172.21.15.53:0/3656662198 shutdown_connections 2024-04-25T14:40:56.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.051+0000 7fa711a00640 1 -- 172.21.15.53:0/3656662198 wait complete. 2024-04-25T14:40:56.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.052+0000 7fa711a00640 1 Processor -- start 2024-04-25T14:40:56.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.052+0000 7fa711a00640 1 -- start start 2024-04-25T14:40:56.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.052+0000 7fa711a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa70c1084f0 0x7fa70c1a4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:56.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.052+0000 7fa711a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa70c108e90 0x7fa70c1a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:56.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.052+0000 7fa711a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa70c10d3c0 0x7fa70c19ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:56.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.052+0000 7fa711a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa70c1186e0 con 0x7fa70c108e90 2024-04-25T14:40:56.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.052+0000 7fa711a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa70c118560 con 0x7fa70c1084f0 2024-04-25T14:40:56.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.052+0000 7fa711a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa70c118860 con 0x7fa70c10d3c0 2024-04-25T14:40:56.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.053+0000 7fa703e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa70c108e90 0x7fa70c1a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:56.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.053+0000 7fa703e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa70c108e90 0x7fa70c1a4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42166/0 (socket says 172.21.15.53:42166) 2024-04-25T14:40:56.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.053+0000 7fa703e00640 1 -- 172.21.15.53:0/24422096 learned_addr learned my addr 172.21.15.53:0/24422096 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:56.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.053+0000 7fa711000640 1 --2- 172.21.15.53:0/24422096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa70c10d3c0 0x7fa70c19ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:56.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.053+0000 7fa70be00640 1 --2- 172.21.15.53:0/24422096 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa70c1084f0 0x7fa70c1a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:56.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.053+0000 7fa703e00640 1 -- 172.21.15.53:0/24422096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa70c10d3c0 msgr2=0x7fa70c19ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:56.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.053+0000 7fa703e00640 1 --2- 172.21.15.53:0/24422096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa70c10d3c0 0x7fa70c19ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.053+0000 7fa703e00640 1 -- 172.21.15.53:0/24422096 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa70c1084f0 msgr2=0x7fa70c1a4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:56.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.053+0000 7fa703e00640 1 --2- 172.21.15.53:0/24422096 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa70c1084f0 0x7fa70c1a4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.053+0000 7fa703e00640 1 -- 172.21.15.53:0/24422096 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa70c19f180 con 0x7fa70c108e90 2024-04-25T14:40:56.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.053+0000 7fa703e00640 1 --2- 172.21.15.53:0/24422096 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa70c108e90 0x7fa70c1a4d40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa6f400e9f0 tx=0x7fa6f400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:56.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.054+0000 7fa709600640 1 -- 172.21.15.53:0/24422096 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f400cd80 con 0x7fa70c108e90 2024-04-25T14:40:56.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.054+0000 7fa709600640 1 -- 172.21.15.53:0/24422096 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6f40108e0 con 0x7fa70c108e90 2024-04-25T14:40:56.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.054+0000 7fa709600640 1 -- 172.21.15.53:0/24422096 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4018980 con 0x7fa70c108e90 2024-04-25T14:40:56.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.054+0000 7fa711a00640 1 -- 172.21.15.53:0/24422096 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa70c19f470 con 0x7fa70c108e90 2024-04-25T14:40:56.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.054+0000 7fa711a00640 1 -- 172.21.15.53:0/24422096 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa70c1ab6c0 con 0x7fa70c108e90 2024-04-25T14:40:56.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.055+0000 7fa711a00640 1 -- 172.21.15.53:0/24422096 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa70c0690e0 con 0x7fa70c108e90 2024-04-25T14:40:56.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.057+0000 7fa709600640 1 -- 172.21.15.53:0/24422096 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7fa6f4010430 con 0x7fa70c108e90 2024-04-25T14:40:56.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.058+0000 7fa709600640 1 --2- 172.21.15.53:0/24422096 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fa6f00a3db0 0x7fa6f00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:56.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.059+0000 7fa709600640 1 -- 172.21.15.53:0/24422096 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8543+0+0 (secure 0 0 0) 0x7fa6f40d0cd0 con 0x7fa70c108e90 2024-04-25T14:40:56.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.059+0000 7fa70be00640 1 --2- 172.21.15.53:0/24422096 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fa6f00a3db0 0x7fa6f00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:56.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.059+0000 7fa70be00640 1 --2- 172.21.15.53:0/24422096 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fa6f00a3db0 0x7fa6f00a6270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc0095f0 tx=0x7fa6fc005f90 comp rx=0 tx=0).ready entity=mgr.15093 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:56.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.063+0000 7fa709600640 1 -- 172.21.15.53:0/24422096 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa6f409cb40 con 0x7fa70c108e90 2024-04-25T14:40:56.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.270+0000 7fa711a00640 1 -- 172.21.15.53:0/24422096 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa70c110960 con 0x7fa70c108e90 2024-04-25T14:40:56.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.271+0000 7fa709600640 1 -- 172.21.15.53:0/24422096 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7fa6f40a0e40 con 0x7fa70c108e90 2024-04-25T14:40:56.271 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2024-04-25T14:40:56.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.273+0000 7fa711a00640 1 -- 172.21.15.53:0/24422096 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fa6f00a3db0 msgr2=0x7fa6f00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:56.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.273+0000 7fa711a00640 1 --2- 172.21.15.53:0/24422096 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fa6f00a3db0 0x7fa6f00a6270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc0095f0 tx=0x7fa6fc005f90 comp rx=0 tx=0).stop 2024-04-25T14:40:56.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.273+0000 7fa711a00640 1 -- 172.21.15.53:0/24422096 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa70c108e90 msgr2=0x7fa70c1a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:56.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.273+0000 7fa711a00640 1 --2- 172.21.15.53:0/24422096 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa70c108e90 0x7fa70c1a4d40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa6f400e9f0 tx=0x7fa6f400eeb0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.274+0000 7fa711a00640 1 -- 172.21.15.53:0/24422096 shutdown_connections 2024-04-25T14:40:56.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.274+0000 7fa711a00640 1 --2- 172.21.15.53:0/24422096 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fa6f00a3db0 0x7fa6f00a6270 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.274+0000 7fa711a00640 1 --2- 172.21.15.53:0/24422096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa70c10d3c0 0x7fa70c19ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.274+0000 7fa711a00640 1 --2- 172.21.15.53:0/24422096 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa70c108e90 0x7fa70c1a4d40 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.274+0000 7fa711a00640 1 --2- 172.21.15.53:0/24422096 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa70c1084f0 0x7fa70c1a4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.274+0000 7fa711a00640 1 -- 172.21.15.53:0/24422096 >> 172.21.15.53:0/24422096 conn(0x7fa70c077d70 msgr2=0x7fa70c0783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:56.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.274+0000 7fa711a00640 1 -- 172.21.15.53:0/24422096 shutdown_connections 2024-04-25T14:40:56.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.274+0000 7fa711a00640 1 -- 172.21.15.53:0/24422096 wait complete. 2024-04-25T14:40:56.295 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:40:56.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.519+0000 7ff008a00640 1 -- 172.21.15.53:0/2168777665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0041084f0 msgr2=0x7ff004108950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:56.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.519+0000 7ff008a00640 1 --2- 172.21.15.53:0/2168777665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0041084f0 0x7ff004108950 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fefec009900 tx=0x7fefec02a480 comp rx=0 tx=0).stop 2024-04-25T14:40:56.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.520+0000 7ff008a00640 1 -- 172.21.15.53:0/2168777665 shutdown_connections 2024-04-25T14:40:56.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.520+0000 7ff008a00640 1 --2- 172.21.15.53:0/2168777665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff004108e90 0x7ff0041152b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.520+0000 7ff008a00640 1 --2- 172.21.15.53:0/2168777665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0041084f0 0x7ff004108950 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.520+0000 7ff008a00640 1 --2- 172.21.15.53:0/2168777665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff00410d3c0 0x7ff00410d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.520+0000 7ff008a00640 1 -- 172.21.15.53:0/2168777665 >> 172.21.15.53:0/2168777665 conn(0x7ff004077d70 msgr2=0x7ff004106d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:56.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.520+0000 7ff008a00640 1 -- 172.21.15.53:0/2168777665 shutdown_connections 2024-04-25T14:40:56.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.520+0000 7ff008a00640 1 -- 172.21.15.53:0/2168777665 wait complete. 2024-04-25T14:40:56.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.521+0000 7ff008a00640 1 Processor -- start 2024-04-25T14:40:56.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.521+0000 7ff008a00640 1 -- start start 2024-04-25T14:40:56.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.521+0000 7ff008a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0041084f0 0x7ff0041a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:56.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.522+0000 7ff008a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff004108e90 0x7ff0041a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:56.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.522+0000 7ff008a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff00410d3c0 0x7ff00419ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:56.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.522+0000 7ff008a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff004118690 con 0x7ff004108e90 2024-04-25T14:40:56.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.522+0000 7ff008a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff004118510 con 0x7ff00410d3c0 2024-04-25T14:40:56.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.522+0000 7ff008a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff004118810 con 0x7ff0041084f0 2024-04-25T14:40:56.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.522+0000 7feffbe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff004108e90 0x7ff0041a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:56.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.522+0000 7feffbe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff004108e90 0x7ff0041a4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42198/0 (socket says 172.21.15.53:42198) 2024-04-25T14:40:56.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.522+0000 7feffbe00640 1 -- 172.21.15.53:0/3382634738 learned_addr learned my addr 172.21.15.53:0/3382634738 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:56.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.522+0000 7ff002800640 1 --2- 172.21.15.53:0/3382634738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0041084f0 0x7ff0041a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:56.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.522+0000 7ff003200640 1 --2- 172.21.15.53:0/3382634738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff00410d3c0 0x7ff00419ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:56.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.522+0000 7ff002800640 1 -- 172.21.15.53:0/3382634738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff00410d3c0 msgr2=0x7ff00419ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:56.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.522+0000 7ff002800640 1 --2- 172.21.15.53:0/3382634738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff00410d3c0 0x7ff00419ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.522+0000 7ff002800640 1 -- 172.21.15.53:0/3382634738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff004108e90 msgr2=0x7ff0041a4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:56.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.522+0000 7ff002800640 1 --2- 172.21.15.53:0/3382634738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff004108e90 0x7ff0041a4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.523+0000 7ff002800640 1 -- 172.21.15.53:0/3382634738 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff00419f1b0 con 0x7ff0041084f0 2024-04-25T14:40:56.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.523+0000 7ff002800640 1 --2- 172.21.15.53:0/3382634738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0041084f0 0x7ff0041a4880 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7feff400e9f0 tx=0x7feff400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:56.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.523+0000 7feffb400640 1 -- 172.21.15.53:0/3382634738 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff400ccc0 con 0x7ff0041084f0 2024-04-25T14:40:56.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.523+0000 7feffb400640 1 -- 172.21.15.53:0/3382634738 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feff40108e0 con 0x7ff0041084f0 2024-04-25T14:40:56.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.523+0000 7ff008a00640 1 -- 172.21.15.53:0/3382634738 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff00419f4a0 con 0x7ff0041084f0 2024-04-25T14:40:56.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.523+0000 7feffb400640 1 -- 172.21.15.53:0/3382634738 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff4018880 con 0x7ff0041084f0 2024-04-25T14:40:56.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.523+0000 7ff008a00640 1 -- 172.21.15.53:0/3382634738 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0041ab740 con 0x7ff0041084f0 2024-04-25T14:40:56.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.525+0000 7ff008a00640 1 -- 172.21.15.53:0/3382634738 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0040690e0 con 0x7ff0041084f0 2024-04-25T14:40:56.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.527+0000 7feffb400640 1 -- 172.21.15.53:0/3382634738 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7feff4010430 con 0x7ff0041084f0 2024-04-25T14:40:56.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.529+0000 7feffb400640 1 --2- 172.21.15.53:0/3382634738 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fefe80a3db0 0x7fefe80a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:56.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.529+0000 7feffb400640 1 -- 172.21.15.53:0/3382634738 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8543+0+0 (secure 0 0 0) 0x7feff40d1100 con 0x7ff0041084f0 2024-04-25T14:40:56.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.529+0000 7feffbe00640 1 --2- 172.21.15.53:0/3382634738 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fefe80a3db0 0x7fefe80a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:56.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.530+0000 7feffbe00640 1 --2- 172.21.15.53:0/3382634738 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fefe80a3db0 0x7fefe80a6270 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fefec031440 tx=0x7fefec0058d0 comp rx=0 tx=0).ready entity=mgr.15093 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:56.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.530+0000 7feffb400640 1 -- 172.21.15.53:0/3382634738 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feff409cf70 con 0x7ff0041084f0 2024-04-25T14:40:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.681+0000 7f8bfea00640 1 -- 172.21.15.53:0/2617026745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bf8102990 msgr2=0x7f8bf81069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.681+0000 7f8bfea00640 1 --2- 172.21.15.53:0/2617026745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bf8102990 0x7f8bf81069d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8be4009900 tx=0x7f8be402a3f0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.681+0000 7f8bfea00640 1 -- 172.21.15.53:0/2617026745 shutdown_connections 2024-04-25T14:40:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.681+0000 7f8bfea00640 1 --2- 172.21.15.53:0/2617026745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8bf81070b0 0x7f8bf810ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.681+0000 7f8bfea00640 1 --2- 172.21.15.53:0/2617026745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bf8102990 0x7f8bf81069d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.681+0000 7f8bfea00640 1 --2- 172.21.15.53:0/2617026745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bf8101fe0 0x7f8bf81023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.681+0000 7f8bfea00640 1 -- 172.21.15.53:0/2617026745 >> 172.21.15.53:0/2617026745 conn(0x7f8bf80fd8a0 msgr2=0x7f8bf80ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.681+0000 7f8bfea00640 1 -- 172.21.15.53:0/2617026745 shutdown_connections 2024-04-25T14:40:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.681+0000 7f8bfea00640 1 -- 172.21.15.53:0/2617026745 wait complete. 2024-04-25T14:40:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.682+0000 7f8bfea00640 1 Processor -- start 2024-04-25T14:40:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.682+0000 7f8bfea00640 1 -- start start 2024-04-25T14:40:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.683+0000 7f8bfea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bf8101fe0 0x7f8bf81a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.683+0000 7f8bfea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bf81070b0 0x7f8bf81a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.683+0000 7f8bfea00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8bf819a6a0 0x7f8bf819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.683+0000 7f8bfea00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf8112020 con 0x7f8bf8101fe0 2024-04-25T14:40:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.683+0000 7f8bfea00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf8111ea0 con 0x7f8bf819a6a0 2024-04-25T14:40:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.683+0000 7f8bfea00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf81121a0 con 0x7f8bf81070b0 2024-04-25T14:40:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.683+0000 7f8bfd600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bf8101fe0 0x7f8bf81a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.683+0000 7f8bfcc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bf81070b0 0x7f8bf81a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.683+0000 7f8bfcc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bf81070b0 0x7f8bf81a0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:49222/0 (socket says 172.21.15.53:49222) 2024-04-25T14:40:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.683+0000 7f8bfd600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bf8101fe0 0x7f8bf81a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42214/0 (socket says 172.21.15.53:42214) 2024-04-25T14:40:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.683+0000 7f8bfe000640 1 --2- 172.21.15.53:0/3300866015 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8bf819a6a0 0x7f8bf819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.683+0000 7f8bfcc00640 1 -- 172.21.15.53:0/3300866015 learned_addr learned my addr 172.21.15.53:0/3300866015 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.684+0000 7f8bfe000640 1 -- 172.21.15.53:0/3300866015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bf81070b0 msgr2=0x7f8bf81a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.684+0000 7f8bfe000640 1 --2- 172.21.15.53:0/3300866015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bf81070b0 0x7f8bf81a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.684+0000 7f8bfe000640 1 -- 172.21.15.53:0/3300866015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bf8101fe0 msgr2=0x7f8bf81a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.684+0000 7f8bfe000640 1 --2- 172.21.15.53:0/3300866015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bf8101fe0 0x7f8bf81a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.684+0000 7f8bfe000640 1 -- 172.21.15.53:0/3300866015 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf819b2a0 con 0x7f8bf819a6a0 2024-04-25T14:40:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.684+0000 7f8bfd600640 1 --2- 172.21.15.53:0/3300866015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bf8101fe0 0x7f8bf81a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:40:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.684+0000 7f8bfe000640 1 --2- 172.21.15.53:0/3300866015 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8bf819a6a0 0x7f8bf819ab20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8bf000ec50 tx=0x7f8bf000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.684+0000 7f8bf6000640 1 -- 172.21.15.53:0/3300866015 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf0003b70 con 0x7f8bf819a6a0 2024-04-25T14:40:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.684+0000 7f8bfea00640 1 -- 172.21.15.53:0/3300866015 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bf81a7380 con 0x7f8bf819a6a0 2024-04-25T14:40:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.684+0000 7f8bf6000640 1 -- 172.21.15.53:0/3300866015 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bf0003d10 con 0x7f8bf819a6a0 2024-04-25T14:40:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.684+0000 7f8bf6000640 1 -- 172.21.15.53:0/3300866015 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf0010830 con 0x7f8bf819a6a0 2024-04-25T14:40:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.685+0000 7f8bfea00640 1 -- 172.21.15.53:0/3300866015 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bf81a7730 con 0x7f8bf819a6a0 2024-04-25T14:40:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.688+0000 7f8c02fa4740 1 Processor -- start 2024-04-25T14:40:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.688+0000 7f8bf6000640 1 -- 172.21.15.53:0/3300866015 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7f8bf0003ec0 con 0x7f8bf819a6a0 2024-04-25T14:40:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.688+0000 7f8c02fa4740 1 -- start start 2024-04-25T14:40:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.689+0000 7f8c02fa4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a3f947d9b0 0x55a3f947dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.689+0000 7f8c02fa4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3f947e2d0 0x55a3f9488650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.689+0000 7f8c02fa4740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3f9488b90 0x55a3f948af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.689+0000 7f8c02fa4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a3f93e3900 con 0x55a3f947e2d0 2024-04-25T14:40:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.689+0000 7f8c02fa4740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a3f9389ef0 con 0x55a3f9488b90 2024-04-25T14:40:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.689+0000 7f8c02fa4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a3f93df8c0 con 0x55a3f947d9b0 2024-04-25T14:40:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.689+0000 7f8bfd600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3f947e2d0 0x55a3f9488650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.689+0000 7f8bfcc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a3f947d9b0 0x55a3f947dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.689+0000 7f8bfcc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a3f947d9b0 0x55a3f947dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:49228/0 (socket says 172.21.15.53:49228) 2024-04-25T14:40:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.689+0000 7f8bfcc00640 1 -- 172.21.15.53:0/1548122966 learned_addr learned my addr 172.21.15.53:0/1548122966 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.689+0000 7f8bfd600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3f947e2d0 0x55a3f9488650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42226/0 (socket says 172.21.15.53:42226) 2024-04-25T14:40:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.690+0000 7f8bfcc00640 1 --2- 172.21.15.53:0/1548122966 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3f9488b90 0x55a3f948af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.690+0000 7f8bfd600640 1 -- 172.21.15.53:0/1548122966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a3f947d9b0 msgr2=0x55a3f947dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:40:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.690+0000 7f8bf6000640 1 --2- 172.21.15.53:0/3300866015 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f8be00a3da0 0x7f8be00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.690+0000 7f8bfd600640 1 --2- 172.21.15.53:0/1548122966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a3f947d9b0 0x55a3f947dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.690+0000 7f8bfd600640 1 -- 172.21.15.53:0/1548122966 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3f9488b90 msgr2=0x55a3f948af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:40:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.690+0000 7f8bfd600640 1 --2- 172.21.15.53:0/1548122966 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3f9488b90 0x55a3f948af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.690+0000 7f8bfd600640 1 -- 172.21.15.53:0/1548122966 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3f90f6210 con 0x55a3f947e2d0 2024-04-25T14:40:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.690+0000 7f8bf6000640 1 -- 172.21.15.53:0/3300866015 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8543+0+0 (secure 0 0 0) 0x7f8bf00d0eb0 con 0x7f8bf819a6a0 2024-04-25T14:40:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.690+0000 7f8bfe000640 1 --2- 172.21.15.53:0/3300866015 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f8be00a3da0 0x7f8be00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.690+0000 7f8bfd600640 1 --2- 172.21.15.53:0/1548122966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3f947e2d0 0x55a3f9488650 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8bec009fd0 tx=0x7f8bec0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eda4c535828f27dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.691+0000 7f8bea600640 1 -- 172.21.15.53:0/1548122966 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec004030 con 0x55a3f947e2d0 2024-04-25T14:40:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.691+0000 7f8bea600640 1 -- 172.21.15.53:0/1548122966 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bec005020 con 0x55a3f947e2d0 2024-04-25T14:40:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.691+0000 7f8bea600640 1 -- 172.21.15.53:0/1548122966 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec004c50 con 0x55a3f947e2d0 2024-04-25T14:40:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.692+0000 7f8c02fa4740 1 -- 172.21.15.53:0/1548122966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3f947e2d0 msgr2=0x55a3f9488650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.692+0000 7f8bfe000640 1 --2- 172.21.15.53:0/3300866015 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f8be00a3da0 0x7f8be00a6260 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8bf00ce530 tx=0x7f8bf0005f70 comp rx=0 tx=0).ready entity=mgr.15093 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.692+0000 7f8c02fa4740 1 --2- 172.21.15.53:0/1548122966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3f947e2d0 0x55a3f9488650 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8bec009fd0 tx=0x7f8bec0118d0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.692+0000 7f8c02fa4740 1 -- 172.21.15.53:0/1548122966 shutdown_connections 2024-04-25T14:40:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.692+0000 7f8c02fa4740 1 --2- 172.21.15.53:0/1548122966 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3f9488b90 0x55a3f948af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.692+0000 7f8c02fa4740 1 --2- 172.21.15.53:0/1548122966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3f947e2d0 0x55a3f9488650 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.692+0000 7f8c02fa4740 1 --2- 172.21.15.53:0/1548122966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a3f947d9b0 0x55a3f947dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.692+0000 7f8c02fa4740 1 -- 172.21.15.53:0/1548122966 >> 172.21.15.53:0/1548122966 conn(0x55a3f9479ec0 msgr2=0x55a3f947a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.692+0000 7f8c02fa4740 1 -- 172.21.15.53:0/1548122966 shutdown_connections 2024-04-25T14:40:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.692+0000 7f8c02fa4740 1 -- 172.21.15.53:0/1548122966 wait complete. 2024-04-25T14:40:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.692+0000 7f8c02fa4740 1 -- start start 2024-04-25T14:40:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.693+0000 7f8c02fa4740 1 Processor -- start 2024-04-25T14:40:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.693+0000 7f8c02fa4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a3f947d9b0 0x55a3f9485a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.693+0000 7f8c02fa4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3f947e2d0 0x55a3f9485f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.693+0000 7f8c02fa4740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3f9488b90 0x55a3f94864d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.693+0000 7f8c02fa4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a3f948b7c0 con 0x55a3f947e2d0 2024-04-25T14:40:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.693+0000 7f8c02fa4740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a3f939d310 con 0x55a3f9488b90 2024-04-25T14:40:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.693+0000 7f8c02fa4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a3f948b940 con 0x55a3f947d9b0 2024-04-25T14:40:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.693+0000 7f8c02fa4740 10 client.0 _ll_register_callbacks cb 0x55a3f936bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:40:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.693+0000 7f8bfd600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3f947e2d0 0x55a3f9485f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.694+0000 7f8bfd600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3f947e2d0 0x55a3f9485f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42242/0 (socket says 172.21.15.53:42242) 2024-04-25T14:40:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.694+0000 7f8bfd600640 1 -- 172.21.15.53:0/2122419386 learned_addr learned my addr 172.21.15.53:0/2122419386 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.694+0000 7f8bfcc00640 1 --2- 172.21.15.53:0/2122419386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a3f947d9b0 0x55a3f9485a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.694+0000 7f8bfcc00640 1 --2- 172.21.15.53:0/2122419386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3f9488b90 0x55a3f94864d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.694+0000 7f8bfd600640 1 -- 172.21.15.53:0/2122419386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a3f947d9b0 msgr2=0x55a3f9485a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:40:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.694+0000 7f8bfd600640 1 --2- 172.21.15.53:0/2122419386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a3f947d9b0 0x55a3f9485a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.694+0000 7f8bfd600640 1 -- 172.21.15.53:0/2122419386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3f9488b90 msgr2=0x55a3f94864d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:40:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.694+0000 7f8bfd600640 1 --2- 172.21.15.53:0/2122419386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3f9488b90 0x55a3f94864d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.694+0000 7f8bfd600640 1 -- 172.21.15.53:0/2122419386 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3f91c2620 con 0x55a3f947e2d0 2024-04-25T14:40:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.694+0000 7f8bfd600640 1 --2- 172.21.15.53:0/2122419386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3f947e2d0 0x55a3f9485f90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8bec009fa0 tx=0x7f8bec004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b28ff130c60a8f84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.694+0000 7f8bdea00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-04-25T14:40:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.695+0000 7f8bdea00640 1 -- 172.21.15.53:0/2122419386 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec028020 con 0x55a3f947e2d0 2024-04-25T14:40:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.695+0000 7f8bdea00640 1 -- 172.21.15.53:0/2122419386 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8543+0+0 (secure 0 0 0) 0x7f8bec016040 con 0x55a3f947e2d0 2024-04-25T14:40:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.695+0000 7f8c02fa4740 1 -- 172.21.15.53:0/2122419386 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bec005020 con 0x55a3f947e2d0 2024-04-25T14:40:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.695+0000 7f8bdea00640 1 -- 172.21.15.53:0/2122419386 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f8bec033050 con 0x55a3f947e2d0 2024-04-25T14:40:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.695+0000 7f8c02fa4740 10 client.15228 fetch_fsmap learned FSMap version 22 2024-04-25T14:40:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.695+0000 7f8c02fa4740 1 -- 172.21.15.53:0/2122419386 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55a3f9497990 con 0x55a3f947e2d0 2024-04-25T14:40:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.695+0000 7f8bdea00640 1 -- 172.21.15.53:0/2122419386 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bec005020 con 0x55a3f947e2d0 2024-04-25T14:40:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.696+0000 7f8bdea00640 1 -- 172.21.15.53:0/2122419386 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f8bec0038c0 con 0x55a3f947e2d0 2024-04-25T14:40:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.696+0000 7f8c02fa4740 10 client.15228 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:40:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.696+0000 7f8c02fa4740 10 client.15228 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:40:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.696+0000 7f8c02fa4740 20 client.15228 populate_metadata read hostname 'smithi053' 2024-04-25T14:40:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.696+0000 7f8c02fa4740 1 --2- 172.21.15.53:0/2122419386 >> [v2:172.21.15.150:6838/149573616,v1:172.21.15.150:6839/149573616] conn(0x55a3f9498df0 0x55a3f949b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.696+0000 7f8c02fa4740 4 client.15228 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:40:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.696+0000 7f8c02fa4740 1 -- 172.21.15.53:0/2122419386 --> [v2:172.21.15.150:6838/149573616,v1:172.21.15.150:6839/149573616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3f93f78b0 con 0x55a3f9498df0 2024-04-25T14:40:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.696+0000 7f8bfcc00640 1 --2- 172.21.15.53:0/2122419386 >> [v2:172.21.15.150:6838/149573616,v1:172.21.15.150:6839/149573616] conn(0x55a3f9498df0 0x55a3f949b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.697+0000 7f8bfcc00640 1 --2- 172.21.15.53:0/2122419386 >> [v2:172.21.15.150:6838/149573616,v1:172.21.15.150:6839/149573616] conn(0x55a3f9498df0 0x55a3f949b1e0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ce132c536142d47 server_cookie=3d8c8dfcb74a4491 in_seq=0 out_seq=0 2024-04-25T14:40:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.697+0000 7f8bdea00640 10 client.15228 ms_handle_connect on v2:172.21.15.150:6838/149573616 2024-04-25T14:40:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.697+0000 7f8bdea00640 1 -- 172.21.15.53:0/2122419386 <== mds.0 v2:172.21.15.150:6838/149573616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a3f93f78b0 con 0x55a3f9498df0 2024-04-25T14:40:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.697+0000 7f8bdea00640 10 client.15228 handle_command_reply: tid=0 2024-04-25T14:40:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.698+0000 7f8c02fa4740 1 client.15228 shutdown 2024-04-25T14:40:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.698+0000 7f8c02fa4740 1 -- 172.21.15.53:0/2122419386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3f947e2d0 msgr2=0x55a3f9485f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.698+0000 7f8c02fa4740 1 --2- 172.21.15.53:0/2122419386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3f947e2d0 0x55a3f9485f90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8bec009fa0 tx=0x7f8bec004740 comp rx=0 tx=0).stop 2024-04-25T14:40:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.698+0000 7f8c02fa4740 1 -- 172.21.15.53:0/2122419386 shutdown_connections 2024-04-25T14:40:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.698+0000 7f8c02fa4740 1 --2- 172.21.15.53:0/2122419386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3f9488b90 0x55a3f94864d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.698+0000 7f8c02fa4740 1 --2- 172.21.15.53:0/2122419386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3f947e2d0 0x55a3f9485f90 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.698+0000 7f8c02fa4740 1 --2- 172.21.15.53:0/2122419386 >> [v2:172.21.15.150:6838/149573616,v1:172.21.15.150:6839/149573616] conn(0x55a3f9498df0 0x55a3f949b1e0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.698+0000 7f8c02fa4740 1 --2- 172.21.15.53:0/2122419386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a3f947d9b0 0x55a3f9485a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.698+0000 7f8c02fa4740 1 -- 172.21.15.53:0/2122419386 >> 172.21.15.53:0/2122419386 conn(0x55a3f9479ec0 msgr2=0x55a3f947fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.698+0000 7f8bdea00640 0 client.15228 ms_handle_reset on v2:172.21.15.150:6838/149573616 2024-04-25T14:40:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.698+0000 7f8c02fa4740 1 -- 172.21.15.53:0/2122419386 shutdown_connections 2024-04-25T14:40:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.698+0000 7f8c02fa4740 1 -- 172.21.15.53:0/2122419386 wait complete. 2024-04-25T14:40:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.698+0000 7f8c02fa4740 20 client.15228 trim_cache size 0 max 16384 2024-04-25T14:40:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.730+0000 7f8c02fa4740 1 Processor -- start 2024-04-25T14:40:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.730+0000 7f8c02fa4740 1 -- start start 2024-04-25T14:40:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.731+0000 7f8c02fa4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3f947d9b0 0x55a3f9494680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.731+0000 7f8c02fa4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a3f947e2d0 0x55a3f9480370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.731+0000 7f8c02fa4740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3f9488b90 0x55a3f94a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:56.731 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.731+0000 7f8c02fa4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf81b2550 con 0x55a3f947d9b0 2024-04-25T14:40:56.731 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.731+0000 7f8c02fa4740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf81afe20 con 0x55a3f9488b90 2024-04-25T14:40:56.731 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.731+0000 7f8c02fa4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf81adc70 con 0x55a3f947e2d0 2024-04-25T14:40:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.731+0000 7f8bfcc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3f947d9b0 0x55a3f9494680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.731+0000 7f8bfcc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3f947d9b0 0x55a3f9494680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42248/0 (socket says 172.21.15.53:42248) 2024-04-25T14:40:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.731+0000 7f8bfcc00640 1 -- 172.21.15.53:0/4118323593 learned_addr learned my addr 172.21.15.53:0/4118323593 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.731+0000 7f8bfd600640 1 --2- 172.21.15.53:0/4118323593 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a3f947e2d0 0x55a3f9480370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.731+0000 7f8bfcc00640 1 --2- 172.21.15.53:0/4118323593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3f9488b90 0x55a3f94a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.731+0000 7f8bfcc00640 1 -- 172.21.15.53:0/4118323593 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a3f947e2d0 msgr2=0x55a3f9480370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:40:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.731+0000 7f8bfcc00640 1 --2- 172.21.15.53:0/4118323593 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a3f947e2d0 0x55a3f9480370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.731+0000 7f8bfcc00640 1 -- 172.21.15.53:0/4118323593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3f9488b90 msgr2=0x55a3f94a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:40:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.731+0000 7f8bfcc00640 1 --2- 172.21.15.53:0/4118323593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3f9488b90 0x55a3f94a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.731+0000 7f8bfcc00640 1 -- 172.21.15.53:0/4118323593 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3f9475820 con 0x55a3f947d9b0 2024-04-25T14:40:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.732+0000 7f8bfcc00640 1 --2- 172.21.15.53:0/4118323593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3f947d9b0 0x55a3f9494680 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8be402a9b0 tx=0x7f8be4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a3f107f9be9477 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.732+0000 7f8be9c00640 1 -- 172.21.15.53:0/4118323593 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be4035040 con 0x55a3f947d9b0 2024-04-25T14:40:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.732+0000 7f8be9c00640 1 -- 172.21.15.53:0/4118323593 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8be4003da0 con 0x55a3f947d9b0 2024-04-25T14:40:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.732+0000 7f8be9c00640 1 -- 172.21.15.53:0/4118323593 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be40049d0 con 0x55a3f947d9b0 2024-04-25T14:40:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.733+0000 7f8c02fa4740 1 -- 172.21.15.53:0/4118323593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3f947d9b0 msgr2=0x55a3f9494680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.733+0000 7f8c02fa4740 1 --2- 172.21.15.53:0/4118323593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3f947d9b0 0x55a3f9494680 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8be402a9b0 tx=0x7f8be4009260 comp rx=0 tx=0).stop 2024-04-25T14:40:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.733+0000 7f8c02fa4740 1 -- 172.21.15.53:0/4118323593 shutdown_connections 2024-04-25T14:40:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.733+0000 7f8c02fa4740 1 --2- 172.21.15.53:0/4118323593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3f9488b90 0x55a3f94a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.733+0000 7f8c02fa4740 1 --2- 172.21.15.53:0/4118323593 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a3f947e2d0 0x55a3f9480370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.733+0000 7f8c02fa4740 1 --2- 172.21.15.53:0/4118323593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3f947d9b0 0x55a3f9494680 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.733+0000 7f8c02fa4740 1 -- 172.21.15.53:0/4118323593 >> 172.21.15.53:0/4118323593 conn(0x55a3f9479ec0 msgr2=0x55a3f94862a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.733+0000 7f8c02fa4740 1 -- 172.21.15.53:0/4118323593 shutdown_connections 2024-04-25T14:40:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.733+0000 7f8c02fa4740 1 -- 172.21.15.53:0/4118323593 wait complete. 2024-04-25T14:40:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.734+0000 7f8c02fa4740 1 -- start start 2024-04-25T14:40:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.734+0000 7f8c02fa4740 1 Processor -- start 2024-04-25T14:40:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.734+0000 7f8c02fa4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3f947d9b0 0x55a3f94f7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.734+0000 7f8c02fa4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a3f947e2d0 0x55a3f94f7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.734+0000 7f8c02fa4740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3f9488b90 0x55a3f948df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.734+0000 7f8c02fa4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf81ae2e0 con 0x55a3f947d9b0 2024-04-25T14:40:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.734+0000 7f8c02fa4740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf81af750 con 0x55a3f9488b90 2024-04-25T14:40:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.734+0000 7f8c02fa4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf81ae9b0 con 0x55a3f947e2d0 2024-04-25T14:40:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.734+0000 7f8c02fa4740 10 client.0 _ll_register_callbacks cb 0x7f8bec009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:40:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.735+0000 7f8bfcc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3f947d9b0 0x55a3f94f7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.735+0000 7f8bfcc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3f947d9b0 0x55a3f94f7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42258/0 (socket says 172.21.15.53:42258) 2024-04-25T14:40:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.735+0000 7f8bfcc00640 1 -- 172.21.15.53:0/2750711416 learned_addr learned my addr 172.21.15.53:0/2750711416 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.735+0000 7f8bfd600640 1 --2- 172.21.15.53:0/2750711416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a3f947e2d0 0x55a3f94f7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.735+0000 7f8bfcc00640 1 --2- 172.21.15.53:0/2750711416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3f9488b90 0x55a3f948df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.735+0000 7f8bfd600640 1 -- 172.21.15.53:0/2750711416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3f9488b90 msgr2=0x55a3f948df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:40:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.735+0000 7f8bfd600640 1 --2- 172.21.15.53:0/2750711416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3f9488b90 0x55a3f948df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.735+0000 7f8bfd600640 1 -- 172.21.15.53:0/2750711416 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3f947d9b0 msgr2=0x55a3f94f7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:40:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.735+0000 7f8bfd600640 1 --2- 172.21.15.53:0/2750711416 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3f947d9b0 0x55a3f94f7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.735+0000 7f8bfd600640 1 -- 172.21.15.53:0/2750711416 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3f948e660 con 0x55a3f947e2d0 2024-04-25T14:40:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.735+0000 7f8bfcc00640 1 --2- 172.21.15.53:0/2750711416 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3f947d9b0 0x55a3f94f7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:40:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.736+0000 7f8bfd600640 1 --2- 172.21.15.53:0/2750711416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a3f947e2d0 0x55a3f94f7bc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8bec003ce0 tx=0x7f8bec003f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac4fad2c94a8924d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.736+0000 7f8bdea00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T14:40:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.736+0000 7f8bdea00640 1 -- 172.21.15.53:0/2750711416 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec028040 con 0x55a3f947e2d0 2024-04-25T14:40:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.736+0000 7f8c02fa4740 1 -- 172.21.15.53:0/2750711416 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8be4003da0 con 0x55a3f947e2d0 2024-04-25T14:40:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.736+0000 7f8bdea00640 1 -- 172.21.15.53:0/2750711416 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8543+0+0 (secure 0 0 0) 0x7f8bec004460 con 0x55a3f947e2d0 2024-04-25T14:40:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.736+0000 7f8bdea00640 1 -- 172.21.15.53:0/2750711416 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f8bec046050 con 0x55a3f947e2d0 2024-04-25T14:40:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.736+0000 7f8c02fa4740 10 client.25375 fetch_fsmap learned FSMap version 22 2024-04-25T14:40:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.736+0000 7f8c02fa4740 1 -- 172.21.15.53:0/2750711416 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55a3f94fee40 con 0x55a3f947e2d0 2024-04-25T14:40:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.737+0000 7f8bdea00640 1 -- 172.21.15.53:0/2750711416 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8be4003da0 con 0x55a3f947e2d0 2024-04-25T14:40:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.737+0000 7f8bdea00640 1 -- 172.21.15.53:0/2750711416 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f8bec00fc50 con 0x55a3f947e2d0 2024-04-25T14:40:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.737+0000 7f8c02fa4740 10 client.25375 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:40:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.737+0000 7f8c02fa4740 10 client.25375 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:40:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.737+0000 7f8c02fa4740 20 client.25375 populate_metadata read hostname 'smithi053' 2024-04-25T14:40:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.737+0000 7f8c02fa4740 1 --2- 172.21.15.53:0/2750711416 >> [v2:172.21.15.150:6838/149573616,v1:172.21.15.150:6839/149573616] conn(0x55a3f94ff520 0x55a3f9501940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.737+0000 7f8c02fa4740 4 client.25375 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:40:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.737+0000 7f8c02fa4740 1 -- 172.21.15.53:0/2750711416 --> [v2:172.21.15.150:6838/149573616,v1:172.21.15.150:6839/149573616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a3f93f1320 con 0x55a3f94ff520 2024-04-25T14:40:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.737+0000 7f8bfcc00640 1 --2- 172.21.15.53:0/2750711416 >> [v2:172.21.15.150:6838/149573616,v1:172.21.15.150:6839/149573616] conn(0x55a3f94ff520 0x55a3f9501940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.738+0000 7f8bfcc00640 1 --2- 172.21.15.53:0/2750711416 >> [v2:172.21.15.150:6838/149573616,v1:172.21.15.150:6839/149573616] conn(0x55a3f94ff520 0x55a3f9501940 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6b173837d6b6d48 server_cookie=8a9ec0d8ce94e1d0 in_seq=0 out_seq=0 2024-04-25T14:40:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.738+0000 7f8bdea00640 10 client.25375 ms_handle_connect on v2:172.21.15.150:6838/149573616 2024-04-25T14:40:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.749+0000 7f8bdea00640 1 -- 172.21.15.53:0/2750711416 <== mds.0 v2:172.21.15.150:6838/149573616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a3f93f1320 con 0x55a3f94ff520 2024-04-25T14:40:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.749+0000 7f8bdea00640 10 client.25375 handle_command_reply: tid=0 2024-04-25T14:40:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.749+0000 7f8c02fa4740 1 client.25375 shutdown 2024-04-25T14:40:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.749+0000 7f8c02fa4740 1 -- 172.21.15.53:0/2750711416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a3f947e2d0 msgr2=0x55a3f94f7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.749+0000 7f8c02fa4740 1 --2- 172.21.15.53:0/2750711416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a3f947e2d0 0x55a3f94f7bc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8bec003ce0 tx=0x7f8bec003f50 comp rx=0 tx=0).stop 2024-04-25T14:40:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.749+0000 7f8c02fa4740 1 -- 172.21.15.53:0/2750711416 shutdown_connections 2024-04-25T14:40:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.749+0000 7f8c02fa4740 1 --2- 172.21.15.53:0/2750711416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3f9488b90 0x55a3f948df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.749+0000 7f8c02fa4740 1 --2- 172.21.15.53:0/2750711416 >> [v2:172.21.15.150:6838/149573616,v1:172.21.15.150:6839/149573616] conn(0x55a3f94ff520 0x55a3f9501940 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.749+0000 7f8c02fa4740 1 --2- 172.21.15.53:0/2750711416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a3f947e2d0 0x55a3f94f7bc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.749+0000 7f8c02fa4740 1 --2- 172.21.15.53:0/2750711416 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3f947d9b0 0x55a3f94f7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.749+0000 7f8bdea00640 0 client.25375 ms_handle_reset on v2:172.21.15.150:6838/149573616 2024-04-25T14:40:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.749+0000 7f8c02fa4740 1 -- 172.21.15.53:0/2750711416 >> 172.21.15.53:0/2750711416 conn(0x55a3f9479ec0 msgr2=0x55a3f94859f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.749+0000 7f8c02fa4740 1 -- 172.21.15.53:0/2750711416 shutdown_connections 2024-04-25T14:40:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.749+0000 7f8c02fa4740 1 -- 172.21.15.53:0/2750711416 wait complete. 2024-04-25T14:40:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.750+0000 7f8c02fa4740 20 client.25375 trim_cache size 0 max 16384 2024-04-25T14:40:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:{ 2024-04-25T14:40:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "message": "", 2024-04-25T14:40:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout: "return_code": 0 2024-04-25T14:40:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stdout:} 2024-04-25T14:40:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.750+0000 7f8bfea00640 1 -- 172.21.15.53:0/3300866015 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f8be00a3da0 msgr2=0x7f8be00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.750+0000 7f8bfea00640 1 --2- 172.21.15.53:0/3300866015 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f8be00a3da0 0x7f8be00a6260 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8bf00ce530 tx=0x7f8bf0005f70 comp rx=0 tx=0).stop 2024-04-25T14:40:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.750+0000 7f8bfea00640 1 -- 172.21.15.53:0/3300866015 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8bf819a6a0 msgr2=0x7f8bf819ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.750+0000 7f8bfea00640 1 --2- 172.21.15.53:0/3300866015 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8bf819a6a0 0x7f8bf819ab20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8bf000ec50 tx=0x7f8bf000c6a0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.750+0000 7f8bfea00640 1 -- 172.21.15.53:0/3300866015 shutdown_connections 2024-04-25T14:40:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.750+0000 7f8bfea00640 1 --2- 172.21.15.53:0/3300866015 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f8be00a3da0 0x7f8be00a6260 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.750+0000 7f8bfea00640 1 --2- 172.21.15.53:0/3300866015 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8bf819a6a0 0x7f8bf819ab20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.750+0000 7f8bfea00640 1 --2- 172.21.15.53:0/3300866015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bf81070b0 0x7f8bf81a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.750+0000 7f8bfea00640 1 --2- 172.21.15.53:0/3300866015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bf8101fe0 0x7f8bf81a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.750+0000 7f8bfea00640 1 -- 172.21.15.53:0/3300866015 >> 172.21.15.53:0/3300866015 conn(0x7f8bf80fd8a0 msgr2=0x7f8bf80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.751+0000 7f8bfea00640 1 -- 172.21.15.53:0/3300866015 shutdown_connections 2024-04-25T14:40:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi053.front.sepia.ceph.com.smithi053.stderr:2024-04-25T14:40:56.751+0000 7f8bfea00640 1 -- 172.21.15.53:0/3300866015 wait complete. 2024-04-25T14:40:56.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.763+0000 7ff008a00640 1 -- 172.21.15.53:0/3382634738 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff004110900 con 0x7ff0041084f0 2024-04-25T14:40:56.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.764+0000 7feffb400640 1 -- 172.21.15.53:0/3382634738 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7feff40a1270 con 0x7ff0041084f0 2024-04-25T14:40:56.764 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2024-04-25T14:40:56.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.767+0000 7ff008a00640 1 -- 172.21.15.53:0/3382634738 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fefe80a3db0 msgr2=0x7fefe80a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:56.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.767+0000 7ff008a00640 1 --2- 172.21.15.53:0/3382634738 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fefe80a3db0 0x7fefe80a6270 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fefec031440 tx=0x7fefec0058d0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.767+0000 7ff008a00640 1 -- 172.21.15.53:0/3382634738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0041084f0 msgr2=0x7ff0041a4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:56.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.767+0000 7ff008a00640 1 --2- 172.21.15.53:0/3382634738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0041084f0 0x7ff0041a4880 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7feff400e9f0 tx=0x7feff400eeb0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.767+0000 7ff008a00640 1 -- 172.21.15.53:0/3382634738 shutdown_connections 2024-04-25T14:40:56.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.767+0000 7ff008a00640 1 --2- 172.21.15.53:0/3382634738 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fefe80a3db0 0x7fefe80a6270 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.767+0000 7ff008a00640 1 --2- 172.21.15.53:0/3382634738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff00410d3c0 0x7ff00419ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.767+0000 7ff008a00640 1 --2- 172.21.15.53:0/3382634738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff004108e90 0x7ff0041a4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.767+0000 7ff008a00640 1 --2- 172.21.15.53:0/3382634738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0041084f0 0x7ff0041a4880 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.767+0000 7ff008a00640 1 -- 172.21.15.53:0/3382634738 >> 172.21.15.53:0/3382634738 conn(0x7ff004077d70 msgr2=0x7ff004078430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:56.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.767+0000 7ff008a00640 1 -- 172.21.15.53:0/3382634738 shutdown_connections 2024-04-25T14:40:56.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.767+0000 7ff008a00640 1 -- 172.21.15.53:0/3382634738 wait complete. 2024-04-25T14:40:56.786 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:40:56.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:56 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/2177429714' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:40:56.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:56 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/24422096' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:40:56.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.996+0000 7f783a600640 1 -- 172.21.15.53:0/307530264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7834109340 msgr2=0x7f7834110f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:56.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.996+0000 7f783a600640 1 --2- 172.21.15.53:0/307530264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7834109340 0x7f7834110f10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f782c009ca0 tx=0x7f782c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.997+0000 7f783a600640 1 -- 172.21.15.53:0/307530264 shutdown_connections 2024-04-25T14:40:56.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.997+0000 7f783a600640 1 --2- 172.21.15.53:0/307530264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7834109340 0x7f7834110f10 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.997+0000 7f783a600640 1 --2- 172.21.15.53:0/307530264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7834104c20 0x7f7834108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.997+0000 7f783a600640 1 --2- 172.21.15.53:0/307530264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7834104270 0x7f7834104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:56.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.997+0000 7f783a600640 1 -- 172.21.15.53:0/307530264 >> 172.21.15.53:0/307530264 conn(0x7f78340ffa90 msgr2=0x7f7834101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:56.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.997+0000 7f783a600640 1 -- 172.21.15.53:0/307530264 shutdown_connections 2024-04-25T14:40:56.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.997+0000 7f783a600640 1 -- 172.21.15.53:0/307530264 wait complete. 2024-04-25T14:40:56.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.998+0000 7f783a600640 1 Processor -- start 2024-04-25T14:40:56.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.998+0000 7f783a600640 1 -- start start 2024-04-25T14:40:56.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.998+0000 7f783a600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7834104270 0x7f78341a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:56.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.998+0000 7f783a600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7834109340 0x7f78341a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:56.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.999+0000 7f783a600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f783419a5e0 0x7f783419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:56.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.999+0000 7f783a600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f78341142e0 con 0x7f7834109340 2024-04-25T14:40:56.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.999+0000 7f783a600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7834114160 con 0x7f7834104270 2024-04-25T14:40:56.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.999+0000 7f783a600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7834114460 con 0x7f783419a5e0 2024-04-25T14:40:57.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.999+0000 7f7833e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7834109340 0x7f78341a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:57.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.999+0000 7f7833e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7834109340 0x7f78341a0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42284/0 (socket says 172.21.15.53:42284) 2024-04-25T14:40:57.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.999+0000 7f7833e00640 1 -- 172.21.15.53:0/3513216933 learned_addr learned my addr 172.21.15.53:0/3513216933 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:57.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.999+0000 7f7839200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7834104270 0x7f78341a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:57.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.999+0000 7f7839c00640 1 --2- 172.21.15.53:0/3513216933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f783419a5e0 0x7f783419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:57.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.999+0000 7f7833e00640 1 -- 172.21.15.53:0/3513216933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f783419a5e0 msgr2=0x7f783419aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:57.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.999+0000 7f7833e00640 1 --2- 172.21.15.53:0/3513216933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f783419a5e0 0x7f783419aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:57.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.999+0000 7f7833e00640 1 -- 172.21.15.53:0/3513216933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7834104270 msgr2=0x7f78341a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:57.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.999+0000 7f7833e00640 1 --2- 172.21.15.53:0/3513216933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7834104270 0x7f78341a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:57.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:56.999+0000 7f7833e00640 1 -- 172.21.15.53:0/3513216933 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f783419b100 con 0x7f7834109340 2024-04-25T14:40:57.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.000+0000 7f7833e00640 1 --2- 172.21.15.53:0/3513216933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7834109340 0x7f78341a0970 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f782400e9a0 tx=0x7f782400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:57.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.000+0000 7f7831600640 1 -- 172.21.15.53:0/3513216933 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782400cd80 con 0x7f7834109340 2024-04-25T14:40:57.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.000+0000 7f7831600640 1 -- 172.21.15.53:0/3513216933 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78240035f0 con 0x7f7834109340 2024-04-25T14:40:57.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.000+0000 7f783a600640 1 -- 172.21.15.53:0/3513216933 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f783419b3f0 con 0x7f7834109340 2024-04-25T14:40:57.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.000+0000 7f7831600640 1 -- 172.21.15.53:0/3513216933 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7824010690 con 0x7f7834109340 2024-04-25T14:40:57.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.000+0000 7f783a600640 1 -- 172.21.15.53:0/3513216933 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78341a7690 con 0x7f7834109340 2024-04-25T14:40:57.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.001+0000 7f783a600640 1 -- 172.21.15.53:0/3513216933 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78340690e0 con 0x7f7834109340 2024-04-25T14:40:57.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.003+0000 7f7831600640 1 -- 172.21.15.53:0/3513216933 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7f7824004000 con 0x7f7834109340 2024-04-25T14:40:57.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.005+0000 7f7831600640 1 --2- 172.21.15.53:0/3513216933 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f781c0a3e30 0x7f781c0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:57.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.005+0000 7f7831600640 1 -- 172.21.15.53:0/3513216933 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8543+0+0 (secure 0 0 0) 0x7f78240d0b80 con 0x7f7834109340 2024-04-25T14:40:57.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.005+0000 7f7839200640 1 --2- 172.21.15.53:0/3513216933 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f781c0a3e30 0x7f781c0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:57.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.006+0000 7f7839200640 1 --2- 172.21.15.53:0/3513216933 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f781c0a3e30 0x7f781c0a62f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7828009bd0 tx=0x7f7828005f90 comp rx=0 tx=0).ready entity=mgr.15093 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:57.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.009+0000 7f7831600640 1 -- 172.21.15.53:0/3513216933 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f78240042f0 con 0x7f7834109340 2024-04-25T14:40:57.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:56 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/2177429714' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:40:57.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:56 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/24422096' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:40:57.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:56 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/2177429714' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:40:57.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:56 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/24422096' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:40:57.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.220+0000 7f783a600640 1 -- 172.21.15.53:0/3513216933 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f783410c5d0 con 0x7f7834109340 2024-04-25T14:40:57.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.221+0000 7f7831600640 1 -- 172.21.15.53:0/3513216933 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f782409d040 con 0x7f7834109340 2024-04-25T14:40:57.220 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2024-04-25T14:40:57.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.224+0000 7f783a600640 1 -- 172.21.15.53:0/3513216933 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f781c0a3e30 msgr2=0x7f781c0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:57.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.224+0000 7f783a600640 1 --2- 172.21.15.53:0/3513216933 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f781c0a3e30 0x7f781c0a62f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7828009bd0 tx=0x7f7828005f90 comp rx=0 tx=0).stop 2024-04-25T14:40:57.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.224+0000 7f783a600640 1 -- 172.21.15.53:0/3513216933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7834109340 msgr2=0x7f78341a0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:57.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.224+0000 7f783a600640 1 --2- 172.21.15.53:0/3513216933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7834109340 0x7f78341a0970 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f782400e9a0 tx=0x7f782400ee60 comp rx=0 tx=0).stop 2024-04-25T14:40:57.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.224+0000 7f783a600640 1 -- 172.21.15.53:0/3513216933 shutdown_connections 2024-04-25T14:40:57.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.224+0000 7f783a600640 1 --2- 172.21.15.53:0/3513216933 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f781c0a3e30 0x7f781c0a62f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:57.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.224+0000 7f783a600640 1 --2- 172.21.15.53:0/3513216933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f783419a5e0 0x7f783419aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:57.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.224+0000 7f783a600640 1 --2- 172.21.15.53:0/3513216933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7834109340 0x7f78341a0970 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:57.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.224+0000 7f783a600640 1 --2- 172.21.15.53:0/3513216933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7834104270 0x7f78341a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:57.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.224+0000 7f783a600640 1 -- 172.21.15.53:0/3513216933 >> 172.21.15.53:0/3513216933 conn(0x7f78340ffa90 msgr2=0x7f7834101d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:57.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.224+0000 7f783a600640 1 -- 172.21.15.53:0/3513216933 shutdown_connections 2024-04-25T14:40:57.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.224+0000 7f783a600640 1 -- 172.21.15.53:0/3513216933 wait complete. 2024-04-25T14:40:57.244 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:40:57.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.456+0000 7f3608c00640 1 -- 172.21.15.53:0/3887951312 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3604108d90 msgr2=0x7f3604115200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:57.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.456+0000 7f3608c00640 1 --2- 172.21.15.53:0/3887951312 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3604108d90 0x7f3604115200 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f35fc009be0 tx=0x7f35fc02a750 comp rx=0 tx=0).stop 2024-04-25T14:40:57.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.456+0000 7f3608c00640 1 -- 172.21.15.53:0/3887951312 shutdown_connections 2024-04-25T14:40:57.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.456+0000 7f3608c00640 1 --2- 172.21.15.53:0/3887951312 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3604108d90 0x7f3604115200 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:57.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.456+0000 7f3608c00640 1 --2- 172.21.15.53:0/3887951312 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36041083f0 0x7f3604108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:57.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.456+0000 7f3608c00640 1 --2- 172.21.15.53:0/3887951312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f360410e460 0x7f360410e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:57.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.456+0000 7f3608c00640 1 -- 172.21.15.53:0/3887951312 >> 172.21.15.53:0/3887951312 conn(0x7f3604077ad0 msgr2=0x7f3604079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:57.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.457+0000 7f3608c00640 1 -- 172.21.15.53:0/3887951312 shutdown_connections 2024-04-25T14:40:57.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.457+0000 7f3608c00640 1 -- 172.21.15.53:0/3887951312 wait complete. 2024-04-25T14:40:57.459 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.458+0000 7f3608c00640 1 Processor -- start 2024-04-25T14:40:57.459 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.458+0000 7f3608c00640 1 -- start start 2024-04-25T14:40:57.459 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.458+0000 7f3608c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36041083f0 0x7f36041a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:57.459 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.458+0000 7f3608c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3604108d90 0x7f36041a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:57.459 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.458+0000 7f3608c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f360410e460 0x7f360419e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:57.459 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.458+0000 7f3608c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f36041185d0 con 0x7f36041083f0 2024-04-25T14:40:57.459 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.458+0000 7f3608c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3604118450 con 0x7f360410e460 2024-04-25T14:40:57.459 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.458+0000 7f3608c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3604118750 con 0x7f3604108d90 2024-04-25T14:40:57.459 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.458+0000 7f35faa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3604108d90 0x7f36041a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:57.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.458+0000 7f35faa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3604108d90 0x7f36041a4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:49272/0 (socket says 172.21.15.53:49272) 2024-04-25T14:40:57.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.458+0000 7f35faa00640 1 -- 172.21.15.53:0/2729777089 learned_addr learned my addr 172.21.15.53:0/2729777089 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:57.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.458+0000 7f3603400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36041083f0 0x7f36041a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:57.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.459+0000 7f3603e00640 1 --2- 172.21.15.53:0/2729777089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f360410e460 0x7f360419e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:57.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.459+0000 7f35faa00640 1 -- 172.21.15.53:0/2729777089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f360410e460 msgr2=0x7f360419e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:57.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.459+0000 7f35faa00640 1 --2- 172.21.15.53:0/2729777089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f360410e460 0x7f360419e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:57.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.459+0000 7f35faa00640 1 -- 172.21.15.53:0/2729777089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36041083f0 msgr2=0x7f36041a46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:57.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.459+0000 7f35faa00640 1 --2- 172.21.15.53:0/2729777089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36041083f0 0x7f36041a46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:57.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.459+0000 7f35faa00640 1 -- 172.21.15.53:0/2729777089 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f360419eff0 con 0x7f3604108d90 2024-04-25T14:40:57.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.459+0000 7f35faa00640 1 --2- 172.21.15.53:0/2729777089 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3604108d90 0x7f36041a4c00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f35ec00e9f0 tx=0x7f35ec00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:57.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.459+0000 7f3600c00640 1 -- 172.21.15.53:0/2729777089 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ec00ccc0 con 0x7f3604108d90 2024-04-25T14:40:57.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.460+0000 7f3600c00640 1 -- 172.21.15.53:0/2729777089 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35ec0108e0 con 0x7f3604108d90 2024-04-25T14:40:57.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.460+0000 7f3608c00640 1 -- 172.21.15.53:0/2729777089 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f360419f2e0 con 0x7f3604108d90 2024-04-25T14:40:57.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.460+0000 7f3600c00640 1 -- 172.21.15.53:0/2729777089 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ec018880 con 0x7f3604108d90 2024-04-25T14:40:57.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.460+0000 7f3608c00640 1 -- 172.21.15.53:0/2729777089 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36041ab580 con 0x7f3604108d90 2024-04-25T14:40:57.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.461+0000 7f3608c00640 1 -- 172.21.15.53:0/2729777089 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36040690e0 con 0x7f3604108d90 2024-04-25T14:40:57.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.463+0000 7f3600c00640 1 -- 172.21.15.53:0/2729777089 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7f35ec010430 con 0x7f3604108d90 2024-04-25T14:40:57.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.464+0000 7f3600c00640 1 --2- 172.21.15.53:0/2729777089 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f35e80a3db0 0x7f35e80a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:57.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.465+0000 7f3600c00640 1 -- 172.21.15.53:0/2729777089 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8543+0+0 (secure 0 0 0) 0x7f35ec0d1360 con 0x7f3604108d90 2024-04-25T14:40:57.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.465+0000 7f3603400640 1 --2- 172.21.15.53:0/2729777089 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f35e80a3db0 0x7f35e80a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:57.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.465+0000 7f3603400640 1 --2- 172.21.15.53:0/2729777089 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f35e80a3db0 0x7f35e80a6270 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f360419f9b0 tx=0x7f35f4009260 comp rx=0 tx=0).ready entity=mgr.15093 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:57.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.466+0000 7f3600c00640 1 -- 172.21.15.53:0/2729777089 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f35ec09d820 con 0x7f3604108d90 2024-04-25T14:40:57.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.721+0000 7f3608c00640 1 -- 172.21.15.53:0/2729777089 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f36041108b0 con 0x7f3604108d90 2024-04-25T14:40:57.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.723+0000 7f3600c00640 1 -- 172.21.15.53:0/2729777089 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11862 (secure 0 0 0) 0x7f35ec09d120 con 0x7f3604108d90 2024-04-25T14:40:57.723 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:40:57.723 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14460,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6841/41110727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6840","nonce":41110727},{"type":"v1","addr":"172.21.15.53:6841","nonce":41110727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2966992472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2966992472},{"type":"v1","addr":"172.21.15.150:6833","nonce":2966992472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/94143643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":94143643},{"type":"v1","addr":"172.21.15.183:6833","nonce":94143643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24470,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/506682943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":506682943},{"type":"v1","addr":"172.21.15.53:6835","nonce":506682943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1369488844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1369488844},{"type":"v1","addr":"172.21.15.150:6835","nonce":1369488844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3786232682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3786232682},{"type":"v1","addr":"172.21.15.53:6837","nonce":3786232682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":22,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T14:38:10.037451+0000","modified":"2024-04-25T14:40:40.222021+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24584,"mds_1":24565,"mds_2":24560},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24532":{"gid":24532,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1189328602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1189328602},{"type":"v1","addr":"172.21.15.183:6835","nonce":1189328602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/2344343627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2344343627},{"type":"v1","addr":"172.21.15.150:6837","nonce":2344343627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24544":{"gid":24544,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1163336384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1163336384},{"type":"v1","addr":"172.21.15.183:6837","nonce":1163336384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6839/570000540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":570000540},{"type":"v1","addr":"172.21.15.53:6839","nonce":570000540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.183:6839/1554258915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1554258915},{"type":"v1","addr":"172.21.15.183:6839","nonce":1554258915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/149573616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":149573616},{"type":"v1","addr":"172.21.15.150:6839","nonce":149573616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24560,24565,24584]},"id":1}]} 2024-04-25T14:40:57.723 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 22 2024-04-25T14:40:57.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.726+0000 7f3608c00640 1 -- 172.21.15.53:0/2729777089 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f35e80a3db0 msgr2=0x7f35e80a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:57.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.726+0000 7f3608c00640 1 --2- 172.21.15.53:0/2729777089 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f35e80a3db0 0x7f35e80a6270 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f360419f9b0 tx=0x7f35f4009260 comp rx=0 tx=0).stop 2024-04-25T14:40:57.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.726+0000 7f3608c00640 1 -- 172.21.15.53:0/2729777089 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3604108d90 msgr2=0x7f36041a4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:57.727 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.726+0000 7f3608c00640 1 --2- 172.21.15.53:0/2729777089 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3604108d90 0x7f36041a4c00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f35ec00e9f0 tx=0x7f35ec00edf0 comp rx=0 tx=0).stop 2024-04-25T14:40:57.727 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.726+0000 7f3608c00640 1 -- 172.21.15.53:0/2729777089 shutdown_connections 2024-04-25T14:40:57.727 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.726+0000 7f3608c00640 1 --2- 172.21.15.53:0/2729777089 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f35e80a3db0 0x7f35e80a6270 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:57.727 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.726+0000 7f3608c00640 1 --2- 172.21.15.53:0/2729777089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f360410e460 0x7f360419e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:57.727 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.726+0000 7f3608c00640 1 --2- 172.21.15.53:0/2729777089 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3604108d90 0x7f36041a4c00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:57.727 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.726+0000 7f3608c00640 1 --2- 172.21.15.53:0/2729777089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36041083f0 0x7f36041a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:57.727 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.726+0000 7f3608c00640 1 -- 172.21.15.53:0/2729777089 >> 172.21.15.53:0/2729777089 conn(0x7f3604077ad0 msgr2=0x7f3604079330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:57.727 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.726+0000 7f3608c00640 1 -- 172.21.15.53:0/2729777089 shutdown_connections 2024-04-25T14:40:57.727 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.726+0000 7f3608c00640 1 -- 172.21.15.53:0/2729777089 wait complete. 2024-04-25T14:40:57.746 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:40:57.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.951+0000 7f98ed200640 1 -- 172.21.15.53:0/2372305407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98e8104b60 msgr2=0x7f98e8108ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:57.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.951+0000 7f98ed200640 1 --2- 172.21.15.53:0/2372305407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98e8104b60 0x7f98e8108ba0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f98d4009900 tx=0x7f98d402a410 comp rx=0 tx=0).stop 2024-04-25T14:40:57.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.951+0000 7f98ed200640 1 -- 172.21.15.53:0/2372305407 shutdown_connections 2024-04-25T14:40:57.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.952+0000 7f98ed200640 1 --2- 172.21.15.53:0/2372305407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98e8109280 0x7f98e8110e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:57.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.952+0000 7f98ed200640 1 --2- 172.21.15.53:0/2372305407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98e8104b60 0x7f98e8108ba0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:57.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.952+0000 7f98ed200640 1 --2- 172.21.15.53:0/2372305407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98e81041b0 0x7f98e8104590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:57.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.952+0000 7f98ed200640 1 -- 172.21.15.53:0/2372305407 >> 172.21.15.53:0/2372305407 conn(0x7f98e80ff9f0 msgr2=0x7f98e8101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:57.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.952+0000 7f98ed200640 1 -- 172.21.15.53:0/2372305407 shutdown_connections 2024-04-25T14:40:57.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.952+0000 7f98ed200640 1 -- 172.21.15.53:0/2372305407 wait complete. 2024-04-25T14:40:57.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.953+0000 7f98ed200640 1 Processor -- start 2024-04-25T14:40:57.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.953+0000 7f98ed200640 1 -- start start 2024-04-25T14:40:57.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.953+0000 7f98ed200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98e81041b0 0x7f98e81a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:57.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.953+0000 7f98ed200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98e8109280 0x7f98e81a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:57.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.953+0000 7f98ed200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98e819a610 0x7f98e819aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:57.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.953+0000 7f98ed200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f98e8114210 con 0x7f98e819a610 2024-04-25T14:40:57.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.953+0000 7f98ed200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f98e8114090 con 0x7f98e81041b0 2024-04-25T14:40:57.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.953+0000 7f98ed200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f98e8114390 con 0x7f98e8109280 2024-04-25T14:40:57.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.953+0000 7f98e7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98e819a610 0x7f98e819aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:57.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.954+0000 7f98e7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98e819a610 0x7f98e819aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42312/0 (socket says 172.21.15.53:42312) 2024-04-25T14:40:57.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.954+0000 7f98e7e00640 1 -- 172.21.15.53:0/2436188059 learned_addr learned my addr 172.21.15.53:0/2436188059 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:57.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.954+0000 7f98e6a00640 1 --2- 172.21.15.53:0/2436188059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98e8109280 0x7f98e81a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:57.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.954+0000 7f98e7400640 1 --2- 172.21.15.53:0/2436188059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98e81041b0 0x7f98e81a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:57.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.954+0000 7f98e7e00640 1 -- 172.21.15.53:0/2436188059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98e8109280 msgr2=0x7f98e81a0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:57.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.954+0000 7f98e7e00640 1 --2- 172.21.15.53:0/2436188059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98e8109280 0x7f98e81a0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:57.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.954+0000 7f98e7e00640 1 -- 172.21.15.53:0/2436188059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98e81041b0 msgr2=0x7f98e81a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:57.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.954+0000 7f98e7e00640 1 --2- 172.21.15.53:0/2436188059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98e81041b0 0x7f98e81a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:57.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.954+0000 7f98e7e00640 1 -- 172.21.15.53:0/2436188059 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e819b130 con 0x7f98e819a610 2024-04-25T14:40:57.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.954+0000 7f98e7e00640 1 --2- 172.21.15.53:0/2436188059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98e819a610 0x7f98e819aac0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f98e000eb10 tx=0x7f98e000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:57.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.954+0000 7f98dfe00640 1 -- 172.21.15.53:0/2436188059 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e0004b20 con 0x7f98e819a610 2024-04-25T14:40:57.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.955+0000 7f98dfe00640 1 -- 172.21.15.53:0/2436188059 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98e0004cc0 con 0x7f98e819a610 2024-04-25T14:40:57.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.955+0000 7f98dfe00640 1 -- 172.21.15.53:0/2436188059 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e0010670 con 0x7f98e819a610 2024-04-25T14:40:57.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.955+0000 7f98ed200640 1 -- 172.21.15.53:0/2436188059 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98e819b420 con 0x7f98e819a610 2024-04-25T14:40:57.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.955+0000 7f98ed200640 1 -- 172.21.15.53:0/2436188059 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98e81a75d0 con 0x7f98e819a610 2024-04-25T14:40:57.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.956+0000 7f98ed200640 1 -- 172.21.15.53:0/2436188059 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98e80690e0 con 0x7f98e819a610 2024-04-25T14:40:57.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.958+0000 7f98dfe00640 1 -- 172.21.15.53:0/2436188059 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7f98e0005020 con 0x7f98e819a610 2024-04-25T14:40:57.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.960+0000 7f98dfe00640 1 --2- 172.21.15.53:0/2436188059 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f98cc0a3e30 0x7f98cc0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:57.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.961+0000 7f98dfe00640 1 -- 172.21.15.53:0/2436188059 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8543+0+0 (secure 0 0 0) 0x7f98e00d0eb0 con 0x7f98e819a610 2024-04-25T14:40:57.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.961+0000 7f98e7400640 1 --2- 172.21.15.53:0/2436188059 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f98cc0a3e30 0x7f98cc0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:57.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.962+0000 7f98e7400640 1 --2- 172.21.15.53:0/2436188059 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f98cc0a3e30 0x7f98cc0a62f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f98d8009bd0 tx=0x7f98d8005f90 comp rx=0 tx=0).ready entity=mgr.15093 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:57.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:57.965+0000 7f98dfe00640 1 -- 172.21.15.53:0/2436188059 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f98e009d370 con 0x7f98e819a610 2024-04-25T14:40:58.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:57 smithi053 ceph-mon[27773]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 76 KiB/s wr, 46 op/s 2024-04-25T14:40:58.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:57 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/3382634738' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:40:58.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:57 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/3513216933' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:40:58.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.173+0000 7f98ed200640 1 -- 172.21.15.53:0/2436188059 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f98e8104b60 con 0x7f98e819a610 2024-04-25T14:40:58.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.178+0000 7f98dfe00640 1 -- 172.21.15.53:0/2436188059 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+23108 (secure 0 0 0) 0x7f98e0087020 con 0x7f98e819a610 2024-04-25T14:40:58.178 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:40:58.178 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":93,"fsid":"fd30e6ce-030f-11ef-bc93-c7b262605968","created":"2024-04-25T14:28:54.330326+0000","modified":"2024-04-25T14:40:40.169020+0000","last_up_change":"2024-04-25T14:37:10.921148+0000","last_in_change":"2024-04-25T14:36:49.566459+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:33:02.338115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:38:07.517974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:38:08.994122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aecfbaf0-39da-4749-a49b-c7205dabd4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6803","nonce":464176015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6805","nonce":464176015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6809","nonce":464176015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6807","nonce":464176015}]},"public_addr":"172.21.15.53:6803/464176015","cluster_addr":"172.21.15.53:6805/464176015","heartbeat_back_addr":"172.21.15.53:6809/464176015","heartbeat_front_addr":"172.21.15.53:6807/464176015","state":["exists","up"]},{"osd":1,"uuid":"2c4a21ae-c43e-4839-8a56-c3fc63aeb7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6801","nonce":1304621982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6803","nonce":1304621982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6807","nonce":1304621982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6805","nonce":1304621982}]},"public_addr":"172.21.15.150:6801/1304621982","cluster_addr":"172.21.15.150:6803/1304621982","heartbeat_back_addr":"172.21.15.150:6807/1304621982","heartbeat_front_addr":"172.21.15.150:6805/1304621982","state":["exists","up"]},{"osd":2,"uuid":"e8effd43-adc0-472a-9eed-a9273b06dc0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6801","nonce":574509311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6803","nonce":574509311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6807","nonce":574509311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6805","nonce":574509311}]},"public_addr":"172.21.15.183:6801/574509311","cluster_addr":"172.21.15.183:6803/574509311","heartbeat_back_addr":"172.21.15.183:6807/574509311","heartbeat_front_addr":"172.21.15.183:6805/574509311","state":["exists","up"]},{"osd":3,"uuid":"e7cfced4-9b40-40de-a303-75ea8d5df4a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6811","nonce":570672219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6813","nonce":570672219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6817","nonce":570672219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6815","nonce":570672219}]},"public_addr":"172.21.15.53:6811/570672219","cluster_addr":"172.21.15.53:6813/570672219","heartbeat_back_addr":"172.21.15.53:6817/570672219","heartbeat_front_addr":"172.21.15.53:6815/570672219","state":["exists","up"]},{"osd":4,"uuid":"44a9e070-2282-43f1-9f05-b1f8ff81e540","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6809","nonce":1526850103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6811","nonce":1526850103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6815","nonce":1526850103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6813","nonce":1526850103}]},"public_addr":"172.21.15.150:6809/1526850103","cluster_addr":"172.21.15.150:6811/1526850103","heartbeat_back_addr":"172.21.15.150:6815/1526850103","heartbeat_front_addr":"172.21.15.150:6813/1526850103","state":["exists","up"]},{"osd":5,"uuid":"2c5619d2-c0f9-47d6-bf7e-2f4afd1b09f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6809","nonce":773726576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6811","nonce":773726576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6815","nonce":773726576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6813","nonce":773726576}]},"public_addr":"172.21.15.183:6809/773726576","cluster_addr":"172.21.15.183:6811/773726576","heartbeat_back_addr":"172.21.15.183:6815/773726576","heartbeat_front_addr":"172.21.15.183:6813/773726576","state":["exists","up"]},{"osd":6,"uuid":"1cf74d94-c118-4738-b6db-040af458a8eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6819","nonce":355884319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6821","nonce":355884319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6825","nonce":355884319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6823","nonce":355884319}]},"public_addr":"172.21.15.53:6819/355884319","cluster_addr":"172.21.15.53:6821/355884319","heartbeat_back_addr":"172.21.15.53:6825/355884319","heartbeat_front_addr":"172.21.15.53:6823/355884319","state":["exists","up"]},{"osd":7,"uuid":"0ce97c04-cd41-4080-bbbc-be9cf5fadc02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6817","nonce":229022562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6819","nonce":229022562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6823","nonce":229022562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6821","nonce":229022562}]},"public_addr":"172.21.15.150:6817/229022562","cluster_addr":"172.21.15.150:6819/229022562","heartbeat_back_addr":"172.21.15.150:6823/229022562","heartbeat_front_addr":"172.21.15.150:6821/229022562","state":["exists","up"]},{"osd":8,"uuid":"5d57787d-2a7c-4514-bd01-357ced413f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6817","nonce":301003844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6819","nonce":301003844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6823","nonce":301003844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6821","nonce":301003844}]},"public_addr":"172.21.15.183:6817/301003844","cluster_addr":"172.21.15.183:6819/301003844","heartbeat_back_addr":"172.21.15.183:6823/301003844","heartbeat_front_addr":"172.21.15.183:6821/301003844","state":["exists","up"]},{"osd":9,"uuid":"e11dd8eb-c55a-4d70-8c58-457540b7a50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6827","nonce":3449260978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6829","nonce":3449260978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6833","nonce":3449260978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6831","nonce":3449260978}]},"public_addr":"172.21.15.53:6827/3449260978","cluster_addr":"172.21.15.53:6829/3449260978","heartbeat_back_addr":"172.21.15.53:6833/3449260978","heartbeat_front_addr":"172.21.15.53:6831/3449260978","state":["exists","up"]},{"osd":10,"uuid":"5de4803d-ad37-4a90-add5-0048f44fc4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6825","nonce":3435992308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6827","nonce":3435992308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6831","nonce":3435992308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6829","nonce":3435992308}]},"public_addr":"172.21.15.150:6825/3435992308","cluster_addr":"172.21.15.150:6827/3435992308","heartbeat_back_addr":"172.21.15.150:6831/3435992308","heartbeat_front_addr":"172.21.15.150:6829/3435992308","state":["exists","up"]},{"osd":11,"uuid":"0e819a68-ed56-477e-b9c0-81e8cc0bce11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6825","nonce":1957842686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6827","nonce":1957842686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6831","nonce":1957842686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6829","nonce":1957842686}]},"public_addr":"172.21.15.183:6825/1957842686","cluster_addr":"172.21.15.183:6827/1957842686","heartbeat_back_addr":"172.21.15.183:6831/1957842686","heartbeat_front_addr":"172.21.15.183:6829/1957842686","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:09.840629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:34.769537+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:58.112969+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:33:29.656790+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:33:55.181287+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:34:19.772645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:34:53.833921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:35:19.076733+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:35:43.163390+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:36:16.568340+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:36:42.492768+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:37:07.944639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"3.38","mappings":[{"from":5,"to":7},{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6800/4154933108":"2024-04-26T14:40:40.168940+0000","172.21.15.53:0/3916258673":"2024-04-26T14:40:40.168940+0000","172.21.15.53:0/309091034":"2024-04-26T14:40:40.168940+0000","172.21.15.53:0/4267178590":"2024-04-26T14:40:40.168940+0000","172.21.15.53:0/920465993":"2024-04-26T14:40:40.168940+0000","172.21.15.53:6800/3887954889":"2024-04-26T14:30:14.162383+0000","172.21.15.53:6801/4154933108":"2024-04-26T14:40:40.168940+0000","172.21.15.53:0/1014501022":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/2576091441":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/1233527569":"2024-04-26T14:29:38.106242+0000","172.21.15.53:6801/3887954889":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/931889810":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/177196141":"2024-04-26T14:30:14.162383+0000","172.21.15.53:6800/1722497390":"2024-04-26T14:29:38.106242+0000","172.21.15.53:6801/1722497390":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/1927549198":"2024-04-26T14:30:14.162383+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:40:58.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.180+0000 7f98ed200640 1 -- 172.21.15.53:0/2436188059 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f98cc0a3e30 msgr2=0x7f98cc0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:58.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.180+0000 7f98ed200640 1 --2- 172.21.15.53:0/2436188059 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f98cc0a3e30 0x7f98cc0a62f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f98d8009bd0 tx=0x7f98d8005f90 comp rx=0 tx=0).stop 2024-04-25T14:40:58.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.180+0000 7f98ed200640 1 -- 172.21.15.53:0/2436188059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98e819a610 msgr2=0x7f98e819aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:58.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.180+0000 7f98ed200640 1 --2- 172.21.15.53:0/2436188059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98e819a610 0x7f98e819aac0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f98e000eb10 tx=0x7f98e000c2d0 comp rx=0 tx=0).stop 2024-04-25T14:40:58.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.181+0000 7f98ed200640 1 -- 172.21.15.53:0/2436188059 shutdown_connections 2024-04-25T14:40:58.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.181+0000 7f98ed200640 1 --2- 172.21.15.53:0/2436188059 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f98cc0a3e30 0x7f98cc0a62f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:58.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.181+0000 7f98ed200640 1 --2- 172.21.15.53:0/2436188059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98e819a610 0x7f98e819aac0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:58.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.181+0000 7f98ed200640 1 --2- 172.21.15.53:0/2436188059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98e8109280 0x7f98e81a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:58.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.181+0000 7f98ed200640 1 --2- 172.21.15.53:0/2436188059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98e81041b0 0x7f98e81a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:58.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.181+0000 7f98ed200640 1 -- 172.21.15.53:0/2436188059 >> 172.21.15.53:0/2436188059 conn(0x7f98e80ff9f0 msgr2=0x7f98e8100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:58.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.181+0000 7f98ed200640 1 -- 172.21.15.53:0/2436188059 shutdown_connections 2024-04-25T14:40:58.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.181+0000 7f98ed200640 1 -- 172.21.15.53:0/2436188059 wait complete. 2024-04-25T14:40:58.201 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:33:02.338115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:38:07.517974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:38:08.994122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:40:58.201 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:40:58.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:57 smithi183 ceph-mon[32584]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 76 KiB/s wr, 46 op/s 2024-04-25T14:40:58.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:57 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/3382634738' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:40:58.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:57 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/3513216933' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:40:58.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:57 smithi150 ceph-mon[32504]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 76 KiB/s wr, 46 op/s 2024-04-25T14:40:58.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:57 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/3382634738' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:40:58.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:57 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/3513216933' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:40:58.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.425+0000 7f114e800640 1 -- 172.21.15.53:0/3460443472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11481094d0 msgr2=0x7f11481110a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:58.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.425+0000 7f114e800640 1 --2- 172.21.15.53:0/3460443472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11481094d0 0x7f11481110a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1140009900 tx=0x7f114002a410 comp rx=0 tx=0).stop 2024-04-25T14:40:58.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.425+0000 7f114e800640 1 -- 172.21.15.53:0/3460443472 shutdown_connections 2024-04-25T14:40:58.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.425+0000 7f114e800640 1 --2- 172.21.15.53:0/3460443472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11481094d0 0x7f11481110a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:58.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.425+0000 7f114e800640 1 --2- 172.21.15.53:0/3460443472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1148104db0 0x7f1148108df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:58.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.425+0000 7f114e800640 1 --2- 172.21.15.53:0/3460443472 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1148104400 0x7f11481047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:58.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.425+0000 7f114e800640 1 -- 172.21.15.53:0/3460443472 >> 172.21.15.53:0/3460443472 conn(0x7f11480ffce0 msgr2=0x7f1148102100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:58.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.426+0000 7f114e800640 1 -- 172.21.15.53:0/3460443472 shutdown_connections 2024-04-25T14:40:58.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.426+0000 7f114e800640 1 -- 172.21.15.53:0/3460443472 wait complete. 2024-04-25T14:40:58.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.427+0000 7f114e800640 1 Processor -- start 2024-04-25T14:40:58.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.427+0000 7f114e800640 1 -- start start 2024-04-25T14:40:58.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.427+0000 7f114e800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1148104400 0x7f11481a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:58.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.427+0000 7f114e800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11481094d0 0x7f11481a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:58.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.427+0000 7f114e800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f114819a7f0 0x7f114819aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:58.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.427+0000 7f114e800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1148114470 con 0x7f114819a7f0 2024-04-25T14:40:58.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.427+0000 7f114e800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f11481142f0 con 0x7f1148104400 2024-04-25T14:40:58.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.427+0000 7f114e800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f11481145f0 con 0x7f11481094d0 2024-04-25T14:40:58.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.427+0000 7f1147400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1148104400 0x7f11481a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:58.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.427+0000 7f1146a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11481094d0 0x7f11481a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:58.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.427+0000 7f1147400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1148104400 0x7f11481a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.53:47856/0 (socket says 172.21.15.53:47856) 2024-04-25T14:40:58.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.427+0000 7f1146a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11481094d0 0x7f11481a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:49302/0 (socket says 172.21.15.53:49302) 2024-04-25T14:40:58.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.427+0000 7f1147400640 1 -- 172.21.15.53:0/2736193433 learned_addr learned my addr 172.21.15.53:0/2736193433 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:58.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.428+0000 7f1147e00640 1 --2- 172.21.15.53:0/2736193433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f114819a7f0 0x7f114819aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:58.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.428+0000 7f1147400640 1 -- 172.21.15.53:0/2736193433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11481094d0 msgr2=0x7f11481a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:58.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.428+0000 7f1147400640 1 --2- 172.21.15.53:0/2736193433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11481094d0 0x7f11481a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:58.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.428+0000 7f1147400640 1 -- 172.21.15.53:0/2736193433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f114819a7f0 msgr2=0x7f114819aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:58.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.428+0000 7f1147400640 1 --2- 172.21.15.53:0/2736193433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f114819a7f0 0x7f114819aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:58.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.428+0000 7f1147400640 1 -- 172.21.15.53:0/2736193433 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f114819b380 con 0x7f1148104400 2024-04-25T14:40:58.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.428+0000 7f1147400640 1 --2- 172.21.15.53:0/2736193433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1148104400 0x7f11481a0680 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f113800ebc0 tx=0x7f113800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:58.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.429+0000 7f113fe00640 1 -- 172.21.15.53:0/2736193433 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1138010630 con 0x7f1148104400 2024-04-25T14:40:58.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.429+0000 7f113fe00640 1 -- 172.21.15.53:0/2736193433 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1138010c80 con 0x7f1148104400 2024-04-25T14:40:58.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.429+0000 7f114e800640 1 -- 172.21.15.53:0/2736193433 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f114819b670 con 0x7f1148104400 2024-04-25T14:40:58.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.429+0000 7f113fe00640 1 -- 172.21.15.53:0/2736193433 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1138018b60 con 0x7f1148104400 2024-04-25T14:40:58.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.429+0000 7f114e800640 1 -- 172.21.15.53:0/2736193433 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11481a78a0 con 0x7f1148104400 2024-04-25T14:40:58.430 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.430+0000 7f114e800640 1 -- 172.21.15.53:0/2736193433 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11481058d0 con 0x7f1148104400 2024-04-25T14:40:58.432 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.433+0000 7f113fe00640 1 -- 172.21.15.53:0/2736193433 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7f1138018d00 con 0x7f1148104400 2024-04-25T14:40:58.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.434+0000 7f113fe00640 1 --2- 172.21.15.53:0/2736193433 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f112c0a3e30 0x7f112c0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:58.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.435+0000 7f113fe00640 1 -- 172.21.15.53:0/2736193433 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8543+0+0 (secure 0 0 0) 0x7f11380d13d0 con 0x7f1148104400 2024-04-25T14:40:58.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.435+0000 7f1146a00640 1 --2- 172.21.15.53:0/2736193433 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f112c0a3e30 0x7f112c0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:58.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.436+0000 7f1146a00640 1 --2- 172.21.15.53:0/2736193433 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f112c0a3e30 0x7f112c0a62f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f11340095f0 tx=0x7f1134005f90 comp rx=0 tx=0).ready entity=mgr.15093 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:58.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.437+0000 7f113fe00640 1 -- 172.21.15.53:0/2736193433 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f114819b670 con 0x7f1148104400 2024-04-25T14:40:58.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.651+0000 7f114e800640 1 -- 172.21.15.53:0/2736193433 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1148104db0 con 0x7f1148104400 2024-04-25T14:40:58.651 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.652+0000 7f113fe00640 1 -- 172.21.15.53:0/2736193433 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f113809d240 con 0x7f1148104400 2024-04-25T14:40:58.652 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2024-04-25T14:40:58.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.654+0000 7f114e800640 1 -- 172.21.15.53:0/2736193433 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f112c0a3e30 msgr2=0x7f112c0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:58.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.654+0000 7f114e800640 1 --2- 172.21.15.53:0/2736193433 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f112c0a3e30 0x7f112c0a62f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f11340095f0 tx=0x7f1134005f90 comp rx=0 tx=0).stop 2024-04-25T14:40:58.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.654+0000 7f114e800640 1 -- 172.21.15.53:0/2736193433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1148104400 msgr2=0x7f11481a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:58.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.654+0000 7f114e800640 1 --2- 172.21.15.53:0/2736193433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1148104400 0x7f11481a0680 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f113800ebc0 tx=0x7f113800c6a0 comp rx=0 tx=0).stop 2024-04-25T14:40:58.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.654+0000 7f114e800640 1 -- 172.21.15.53:0/2736193433 shutdown_connections 2024-04-25T14:40:58.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.654+0000 7f114e800640 1 --2- 172.21.15.53:0/2736193433 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f112c0a3e30 0x7f112c0a62f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:58.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.654+0000 7f114e800640 1 --2- 172.21.15.53:0/2736193433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f114819a7f0 0x7f114819aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:58.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.654+0000 7f114e800640 1 --2- 172.21.15.53:0/2736193433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11481094d0 0x7f11481a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:58.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.654+0000 7f114e800640 1 --2- 172.21.15.53:0/2736193433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1148104400 0x7f11481a0680 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:58.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.654+0000 7f114e800640 1 -- 172.21.15.53:0/2736193433 >> 172.21.15.53:0/2736193433 conn(0x7f11480ffce0 msgr2=0x7f1148102010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:58.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.655+0000 7f114e800640 1 -- 172.21.15.53:0/2736193433 shutdown_connections 2024-04-25T14:40:58.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.655+0000 7f114e800640 1 -- 172.21.15.53:0/2736193433 wait complete. 2024-04-25T14:40:58.674 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:40:58.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.891+0000 7f02d0c00640 1 -- 172.21.15.53:0/3419639017 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02cc102fb0 msgr2=0x7f02cc103390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:58.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.891+0000 7f02d0c00640 1 --2- 172.21.15.53:0/3419639017 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02cc102fb0 0x7f02cc103390 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f02bc009be0 tx=0x7f02bc02a750 comp rx=0 tx=0).stop 2024-04-25T14:40:58.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.891+0000 7f02d0c00640 1 -- 172.21.15.53:0/3419639017 shutdown_connections 2024-04-25T14:40:58.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.891+0000 7f02d0c00640 1 --2- 172.21.15.53:0/3419639017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02cc10c3d0 0x7f02cc10e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:58.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.891+0000 7f02d0c00640 1 --2- 172.21.15.53:0/3419639017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02cc1038d0 0x7f02cc10bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:58.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.891+0000 7f02d0c00640 1 --2- 172.21.15.53:0/3419639017 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02cc102fb0 0x7f02cc103390 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:58.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.891+0000 7f02d0c00640 1 -- 172.21.15.53:0/3419639017 >> 172.21.15.53:0/3419639017 conn(0x7f02cc0fc7d0 msgr2=0x7f02cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:58.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.891+0000 7f02d0c00640 1 -- 172.21.15.53:0/3419639017 shutdown_connections 2024-04-25T14:40:58.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.891+0000 7f02d0c00640 1 -- 172.21.15.53:0/3419639017 wait complete. 2024-04-25T14:40:58.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.893+0000 7f02d0c00640 1 Processor -- start 2024-04-25T14:40:58.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.893+0000 7f02d0c00640 1 -- start start 2024-04-25T14:40:58.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.893+0000 7f02d0c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02cc102fb0 0x7f02cc1a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:58.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.893+0000 7f02d0c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02cc1038d0 0x7f02cc1a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:58.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.893+0000 7f02d0c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02cc10c3d0 0x7f02cc19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:58.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.893+0000 7f02d0c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f02cc112060 con 0x7f02cc10c3d0 2024-04-25T14:40:58.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.893+0000 7f02d0c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f02cc111ee0 con 0x7f02cc102fb0 2024-04-25T14:40:58.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.893+0000 7f02d0c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f02cc1121e0 con 0x7f02cc1038d0 2024-04-25T14:40:58.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.894+0000 7f02cbe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02cc10c3d0 0x7f02cc19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:58.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.894+0000 7f02cbe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02cc10c3d0 0x7f02cc19a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42352/0 (socket says 172.21.15.53:42352) 2024-04-25T14:40:58.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.894+0000 7f02cbe00640 1 -- 172.21.15.53:0/4169885948 learned_addr learned my addr 172.21.15.53:0/4169885948 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:58.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.894+0000 7f02cb400640 1 --2- 172.21.15.53:0/4169885948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02cc102fb0 0x7f02cc1a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:58.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.894+0000 7f02caa00640 1 --2- 172.21.15.53:0/4169885948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02cc1038d0 0x7f02cc1a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:58.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.894+0000 7f02cbe00640 1 -- 172.21.15.53:0/4169885948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02cc1038d0 msgr2=0x7f02cc1a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:58.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.894+0000 7f02cbe00640 1 --2- 172.21.15.53:0/4169885948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02cc1038d0 0x7f02cc1a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:58.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.894+0000 7f02cbe00640 1 -- 172.21.15.53:0/4169885948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02cc102fb0 msgr2=0x7f02cc1a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:58.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.894+0000 7f02cbe00640 1 --2- 172.21.15.53:0/4169885948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02cc102fb0 0x7f02cc1a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:58.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.894+0000 7f02cbe00640 1 -- 172.21.15.53:0/4169885948 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02cc19af20 con 0x7f02cc10c3d0 2024-04-25T14:40:58.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.894+0000 7f02cbe00640 1 --2- 172.21.15.53:0/4169885948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02cc10c3d0 0x7f02cc19a660 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f02c400e940 tx=0x7f02c400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:58.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.895+0000 7f02c3e00640 1 -- 172.21.15.53:0/4169885948 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c400cd80 con 0x7f02cc10c3d0 2024-04-25T14:40:58.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.895+0000 7f02c3e00640 1 -- 172.21.15.53:0/4169885948 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02c40035f0 con 0x7f02cc10c3d0 2024-04-25T14:40:58.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.895+0000 7f02c3e00640 1 -- 172.21.15.53:0/4169885948 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c4010670 con 0x7f02cc10c3d0 2024-04-25T14:40:58.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.895+0000 7f02d0c00640 1 -- 172.21.15.53:0/4169885948 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02cc19b210 con 0x7f02cc10c3d0 2024-04-25T14:40:58.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.895+0000 7f02d0c00640 1 -- 172.21.15.53:0/4169885948 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02cc1a7520 con 0x7f02cc10c3d0 2024-04-25T14:40:58.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.896+0000 7f02d0c00640 1 -- 172.21.15.53:0/4169885948 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02cc0699c0 con 0x7f02cc10c3d0 2024-04-25T14:40:58.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.898+0000 7f02c3e00640 1 -- 172.21.15.53:0/4169885948 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7f02c4003ac0 con 0x7f02cc10c3d0 2024-04-25T14:40:58.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.900+0000 7f02c3e00640 1 --2- 172.21.15.53:0/4169885948 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f02b00a3db0 0x7f02b00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:58.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.900+0000 7f02c3e00640 1 -- 172.21.15.53:0/4169885948 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8543+0+0 (secure 0 0 0) 0x7f02c40d0ce0 con 0x7f02cc10c3d0 2024-04-25T14:40:58.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.900+0000 7f02cb400640 1 --2- 172.21.15.53:0/4169885948 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f02b00a3db0 0x7f02b00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:58.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.901+0000 7f02cb400640 1 --2- 172.21.15.53:0/4169885948 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f02b00a3db0 0x7f02b00a6270 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f02bc000f80 tx=0x7f02bc005fd0 comp rx=0 tx=0).ready entity=mgr.15093 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:58.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:58.904+0000 7f02c3e00640 1 -- 172.21.15.53:0/4169885948 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f02c409d1a0 con 0x7f02cc10c3d0 2024-04-25T14:40:59.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:58 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/2729777089' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:40:59.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:58 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/2436188059' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:40:59.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:58 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/2736193433' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:40:59.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.110+0000 7f02d0c00640 1 -- 172.21.15.53:0/4169885948 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f02cc1a76f0 con 0x7f02cc10c3d0 2024-04-25T14:40:59.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.110+0000 7f02c3e00640 1 -- 172.21.15.53:0/4169885948 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f02c409caa0 con 0x7f02cc10c3d0 2024-04-25T14:40:59.110 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2024-04-25T14:40:59.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.113+0000 7f02d0c00640 1 -- 172.21.15.53:0/4169885948 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f02b00a3db0 msgr2=0x7f02b00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:59.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.113+0000 7f02d0c00640 1 --2- 172.21.15.53:0/4169885948 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f02b00a3db0 0x7f02b00a6270 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f02bc000f80 tx=0x7f02bc005fd0 comp rx=0 tx=0).stop 2024-04-25T14:40:59.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.113+0000 7f02d0c00640 1 -- 172.21.15.53:0/4169885948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02cc10c3d0 msgr2=0x7f02cc19a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:59.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.113+0000 7f02d0c00640 1 --2- 172.21.15.53:0/4169885948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02cc10c3d0 0x7f02cc19a660 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f02c400e940 tx=0x7f02c400ee00 comp rx=0 tx=0).stop 2024-04-25T14:40:59.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.113+0000 7f02d0c00640 1 -- 172.21.15.53:0/4169885948 shutdown_connections 2024-04-25T14:40:59.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.113+0000 7f02d0c00640 1 --2- 172.21.15.53:0/4169885948 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f02b00a3db0 0x7f02b00a6270 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:59.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.113+0000 7f02d0c00640 1 --2- 172.21.15.53:0/4169885948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02cc10c3d0 0x7f02cc19a660 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:59.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.113+0000 7f02d0c00640 1 --2- 172.21.15.53:0/4169885948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02cc1038d0 0x7f02cc1a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:59.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.113+0000 7f02d0c00640 1 --2- 172.21.15.53:0/4169885948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02cc102fb0 0x7f02cc1a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:59.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.113+0000 7f02d0c00640 1 -- 172.21.15.53:0/4169885948 >> 172.21.15.53:0/4169885948 conn(0x7f02cc0fc7d0 msgr2=0x7f02cc101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:59.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.113+0000 7f02d0c00640 1 -- 172.21.15.53:0/4169885948 shutdown_connections 2024-04-25T14:40:59.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.113+0000 7f02d0c00640 1 -- 172.21.15.53:0/4169885948 wait complete. 2024-04-25T14:40:59.131 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:40:59.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:58 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/2729777089' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:40:59.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:58 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/2436188059' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:40:59.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:58 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/2736193433' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:40:59.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:58 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/2729777089' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:40:59.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:58 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/2436188059' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:40:59.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:58 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/2736193433' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:40:59.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.342+0000 7faa7fe00640 1 -- 172.21.15.53:0/2489508925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa80108e90 msgr2=0x7faa801152b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:59.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.342+0000 7faa7fe00640 1 --2- 172.21.15.53:0/2489508925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa80108e90 0x7faa801152b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7faa78009ca0 tx=0x7faa7802a950 comp rx=0 tx=0).stop 2024-04-25T14:40:59.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.342+0000 7faa7fe00640 1 -- 172.21.15.53:0/2489508925 shutdown_connections 2024-04-25T14:40:59.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.342+0000 7faa7fe00640 1 --2- 172.21.15.53:0/2489508925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa80108e90 0x7faa801152b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:59.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.342+0000 7faa7fe00640 1 --2- 172.21.15.53:0/2489508925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa801084f0 0x7faa80108950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:59.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.342+0000 7faa7fe00640 1 --2- 172.21.15.53:0/2489508925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa8010d3c0 0x7faa8010d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:59.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.342+0000 7faa7fe00640 1 -- 172.21.15.53:0/2489508925 >> 172.21.15.53:0/2489508925 conn(0x7faa80077d70 msgr2=0x7faa80106d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:59.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.342+0000 7faa7fe00640 1 -- 172.21.15.53:0/2489508925 shutdown_connections 2024-04-25T14:40:59.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.342+0000 7faa7fe00640 1 -- 172.21.15.53:0/2489508925 wait complete. 2024-04-25T14:40:59.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.343+0000 7faa7fe00640 1 Processor -- start 2024-04-25T14:40:59.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.343+0000 7faa7fe00640 1 -- start start 2024-04-25T14:40:59.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.344+0000 7faa7fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa801084f0 0x7faa801a4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:59.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.344+0000 7faa7fe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa80108e90 0x7faa801a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:59.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.344+0000 7faa7fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa8010d3c0 0x7faa8019ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:59.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.344+0000 7faa7fe00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7faa801186e0 con 0x7faa801084f0 2024-04-25T14:40:59.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.344+0000 7faa7fe00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7faa80118560 con 0x7faa80108e90 2024-04-25T14:40:59.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.344+0000 7faa7fe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faa80118860 con 0x7faa8010d3c0 2024-04-25T14:40:59.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.344+0000 7faa7ea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa801084f0 0x7faa801a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:59.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.344+0000 7faa7ea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa801084f0 0x7faa801a4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42376/0 (socket says 172.21.15.53:42376) 2024-04-25T14:40:59.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.344+0000 7faa7ea00640 1 -- 172.21.15.53:0/1773784107 learned_addr learned my addr 172.21.15.53:0/1773784107 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:59.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.344+0000 7faa7e000640 1 --2- 172.21.15.53:0/1773784107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa80108e90 0x7faa801a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:59.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.344+0000 7faa7f400640 1 --2- 172.21.15.53:0/1773784107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa8010d3c0 0x7faa8019ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:59.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.345+0000 7faa7ea00640 1 -- 172.21.15.53:0/1773784107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa8010d3c0 msgr2=0x7faa8019ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:59.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.345+0000 7faa7ea00640 1 --2- 172.21.15.53:0/1773784107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa8010d3c0 0x7faa8019ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:59.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.345+0000 7faa7ea00640 1 -- 172.21.15.53:0/1773784107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa80108e90 msgr2=0x7faa801a4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:59.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.345+0000 7faa7ea00640 1 --2- 172.21.15.53:0/1773784107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa80108e90 0x7faa801a4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:59.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.345+0000 7faa7ea00640 1 -- 172.21.15.53:0/1773784107 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa8019f180 con 0x7faa801084f0 2024-04-25T14:40:59.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.345+0000 7faa7ea00640 1 --2- 172.21.15.53:0/1773784107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa801084f0 0x7faa801a4800 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7faa7000d950 tx=0x7faa7000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:59.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.345+0000 7faa77400640 1 -- 172.21.15.53:0/1773784107 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa70017070 con 0x7faa801084f0 2024-04-25T14:40:59.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.345+0000 7faa77400640 1 -- 172.21.15.53:0/1773784107 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa70009c30 con 0x7faa801084f0 2024-04-25T14:40:59.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.345+0000 7faa7fe00640 1 -- 172.21.15.53:0/1773784107 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa8019f470 con 0x7faa801084f0 2024-04-25T14:40:59.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.345+0000 7faa77400640 1 -- 172.21.15.53:0/1773784107 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa7000f620 con 0x7faa801084f0 2024-04-25T14:40:59.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.346+0000 7faa7fe00640 1 -- 172.21.15.53:0/1773784107 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa801ab6c0 con 0x7faa801084f0 2024-04-25T14:40:59.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.347+0000 7faa7fe00640 1 -- 172.21.15.53:0/1773784107 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa800690e0 con 0x7faa801084f0 2024-04-25T14:40:59.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.349+0000 7faa77400640 1 -- 172.21.15.53:0/1773784107 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7faa70010460 con 0x7faa801084f0 2024-04-25T14:40:59.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.351+0000 7faa77400640 1 --2- 172.21.15.53:0/1773784107 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7faa640a3db0 0x7faa640a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:59.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.352+0000 7faa77400640 1 -- 172.21.15.53:0/1773784107 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8543+0+0 (secure 0 0 0) 0x7faa700d0510 con 0x7faa801084f0 2024-04-25T14:40:59.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.352+0000 7faa7e000640 1 --2- 172.21.15.53:0/1773784107 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7faa640a3db0 0x7faa640a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:59.352 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.352+0000 7faa7e000640 1 --2- 172.21.15.53:0/1773784107 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7faa640a3db0 0x7faa640a6270 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faa6c0095f0 tx=0x7faa6c005f90 comp rx=0 tx=0).ready entity=mgr.15093 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:59.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.356+0000 7faa77400640 1 -- 172.21.15.53:0/1773784107 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faa70009de0 con 0x7faa801084f0 2024-04-25T14:40:59.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.563+0000 7faa7fe00640 1 -- 172.21.15.53:0/1773784107 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7faa80110900 con 0x7faa801084f0 2024-04-25T14:40:59.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.564+0000 7faa77400640 1 -- 172.21.15.53:0/1773784107 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7faa7009c380 con 0x7faa801084f0 2024-04-25T14:40:59.564 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2024-04-25T14:40:59.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.568+0000 7faa7fe00640 1 -- 172.21.15.53:0/1773784107 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7faa640a3db0 msgr2=0x7faa640a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:59.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.568+0000 7faa7fe00640 1 --2- 172.21.15.53:0/1773784107 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7faa640a3db0 0x7faa640a6270 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faa6c0095f0 tx=0x7faa6c005f90 comp rx=0 tx=0).stop 2024-04-25T14:40:59.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.568+0000 7faa7fe00640 1 -- 172.21.15.53:0/1773784107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa801084f0 msgr2=0x7faa801a4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:59.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.568+0000 7faa7fe00640 1 --2- 172.21.15.53:0/1773784107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa801084f0 0x7faa801a4800 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7faa7000d950 tx=0x7faa7000de10 comp rx=0 tx=0).stop 2024-04-25T14:40:59.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.568+0000 7faa7fe00640 1 -- 172.21.15.53:0/1773784107 shutdown_connections 2024-04-25T14:40:59.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.568+0000 7faa7fe00640 1 --2- 172.21.15.53:0/1773784107 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7faa640a3db0 0x7faa640a6270 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:59.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.568+0000 7faa7fe00640 1 --2- 172.21.15.53:0/1773784107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa8010d3c0 0x7faa8019ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:59.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.568+0000 7faa7fe00640 1 --2- 172.21.15.53:0/1773784107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa80108e90 0x7faa801a4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:59.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.568+0000 7faa7fe00640 1 --2- 172.21.15.53:0/1773784107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa801084f0 0x7faa801a4800 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:59.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.568+0000 7faa7fe00640 1 -- 172.21.15.53:0/1773784107 >> 172.21.15.53:0/1773784107 conn(0x7faa80077d70 msgr2=0x7faa800783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:59.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.568+0000 7faa7fe00640 1 -- 172.21.15.53:0/1773784107 shutdown_connections 2024-04-25T14:40:59.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.568+0000 7faa7fe00640 1 -- 172.21.15.53:0/1773784107 wait complete. 2024-04-25T14:40:59.589 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:40:59.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.810+0000 7f66bee00640 1 -- 172.21.15.53:0/2297878236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66b8104390 msgr2=0x7f66b8104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:59.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.810+0000 7f66bee00640 1 --2- 172.21.15.53:0/2297878236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66b8104390 0x7f66b8104770 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f66a8009ca0 tx=0x7f66a802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:40:59.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.810+0000 7f66bee00640 1 -- 172.21.15.53:0/2297878236 shutdown_connections 2024-04-25T14:40:59.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.810+0000 7f66bee00640 1 --2- 172.21.15.53:0/2297878236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66b8109460 0x7f66b8111030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:59.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.810+0000 7f66bee00640 1 --2- 172.21.15.53:0/2297878236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66b8104d40 0x7f66b8108d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:59.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.810+0000 7f66bee00640 1 --2- 172.21.15.53:0/2297878236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66b8104390 0x7f66b8104770 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:59.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.810+0000 7f66bee00640 1 -- 172.21.15.53:0/2297878236 >> 172.21.15.53:0/2297878236 conn(0x7f66b80ffc50 msgr2=0x7f66b8102070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:40:59.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.810+0000 7f66bee00640 1 -- 172.21.15.53:0/2297878236 shutdown_connections 2024-04-25T14:40:59.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.810+0000 7f66bee00640 1 -- 172.21.15.53:0/2297878236 wait complete. 2024-04-25T14:40:59.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.811+0000 7f66bee00640 1 Processor -- start 2024-04-25T14:40:59.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.812+0000 7f66bee00640 1 -- start start 2024-04-25T14:40:59.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.812+0000 7f66bee00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66b8104390 0x7f66b81a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:59.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.812+0000 7f66bee00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66b8109460 0x7f66b81a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:59.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.812+0000 7f66bee00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66b819a790 0x7f66b819ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:59.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.812+0000 7f66bee00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f66b81143f0 con 0x7f66b8109460 2024-04-25T14:40:59.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.812+0000 7f66bee00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f66b8114270 con 0x7f66b819a790 2024-04-25T14:40:59.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.812+0000 7f66bee00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f66b8114570 con 0x7f66b8104390 2024-04-25T14:40:59.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.812+0000 7f66b7400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66b8109460 0x7f66b81a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:59.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.812+0000 7f66b7400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66b8109460 0x7f66b81a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42400/0 (socket says 172.21.15.53:42400) 2024-04-25T14:40:59.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.812+0000 7f66b7400640 1 -- 172.21.15.53:0/3363122735 learned_addr learned my addr 172.21.15.53:0/3363122735 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:40:59.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.812+0000 7f66bce00640 1 --2- 172.21.15.53:0/3363122735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66b819a790 0x7f66b819ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:59.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.813+0000 7f66b7e00640 1 --2- 172.21.15.53:0/3363122735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66b8104390 0x7f66b81a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:59.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.813+0000 7f66b7400640 1 -- 172.21.15.53:0/3363122735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66b8104390 msgr2=0x7f66b81a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:59.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.813+0000 7f66b7400640 1 --2- 172.21.15.53:0/3363122735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66b8104390 0x7f66b81a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:59.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.813+0000 7f66b7400640 1 -- 172.21.15.53:0/3363122735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66b819a790 msgr2=0x7f66b819ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:40:59.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.813+0000 7f66b7400640 1 --2- 172.21.15.53:0/3363122735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66b819a790 0x7f66b819ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:40:59.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.813+0000 7f66b7400640 1 -- 172.21.15.53:0/3363122735 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b819b320 con 0x7f66b8109460 2024-04-25T14:40:59.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.813+0000 7f66bce00640 1 --2- 172.21.15.53:0/3363122735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66b819a790 0x7f66b819ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:40:59.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.813+0000 7f66b7400640 1 --2- 172.21.15.53:0/3363122735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66b8109460 0x7f66b81a0b60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f66a000e9a0 tx=0x7f66a000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:59.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.814+0000 7f66b4c00640 1 -- 172.21.15.53:0/3363122735 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a000cd80 con 0x7f66b8109460 2024-04-25T14:40:59.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.814+0000 7f66b4c00640 1 -- 172.21.15.53:0/3363122735 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66a00035f0 con 0x7f66b8109460 2024-04-25T14:40:59.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.814+0000 7f66b4c00640 1 -- 172.21.15.53:0/3363122735 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a0010690 con 0x7f66b8109460 2024-04-25T14:40:59.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.814+0000 7f66bee00640 1 -- 172.21.15.53:0/3363122735 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66b819b610 con 0x7f66b8109460 2024-04-25T14:40:59.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.814+0000 7f66bee00640 1 -- 172.21.15.53:0/3363122735 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66b81a77f0 con 0x7f66b8109460 2024-04-25T14:40:59.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.815+0000 7f66bee00640 1 -- 172.21.15.53:0/3363122735 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66b8105860 con 0x7f66b8109460 2024-04-25T14:40:59.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.817+0000 7f66b4c00640 1 -- 172.21.15.53:0/3363122735 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7f66a0004000 con 0x7f66b8109460 2024-04-25T14:40:59.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.819+0000 7f66b4c00640 1 --2- 172.21.15.53:0/3363122735 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f669c0a3db0 0x7f669c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:40:59.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.819+0000 7f66b4c00640 1 -- 172.21.15.53:0/3363122735 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8543+0+0 (secure 0 0 0) 0x7f66a00d0c90 con 0x7f66b8109460 2024-04-25T14:40:59.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.819+0000 7f66b7e00640 1 --2- 172.21.15.53:0/3363122735 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f669c0a3db0 0x7f669c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:40:59.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.820+0000 7f66b7e00640 1 --2- 172.21.15.53:0/3363122735 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f669c0a3db0 0x7f669c0a6270 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f66a8006010 tx=0x7f66a8009820 comp rx=0 tx=0).ready entity=mgr.15093 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:40:59.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:40:59.823+0000 7f66b4c00640 1 -- 172.21.15.53:0/3363122735 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f66b819b610 con 0x7f66b8109460 2024-04-25T14:41:00.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:59 smithi053 ceph-mon[27773]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 76 KiB/s wr, 49 op/s 2024-04-25T14:41:00.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:59 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/4169885948' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:41:00.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:40:59 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1773784107' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:41:00.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.072+0000 7f66bee00640 1 -- 172.21.15.53:0/3363122735 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66b810c690 con 0x7f66b8109460 2024-04-25T14:41:00.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.074+0000 7f66b4c00640 1 -- 172.21.15.53:0/3363122735 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11862 (secure 0 0 0) 0x7f66a00042f0 con 0x7f66b8109460 2024-04-25T14:41:00.074 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:41:00.074 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14460,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6841/41110727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6840","nonce":41110727},{"type":"v1","addr":"172.21.15.53:6841","nonce":41110727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2966992472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2966992472},{"type":"v1","addr":"172.21.15.150:6833","nonce":2966992472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/94143643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":94143643},{"type":"v1","addr":"172.21.15.183:6833","nonce":94143643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24470,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/506682943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":506682943},{"type":"v1","addr":"172.21.15.53:6835","nonce":506682943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1369488844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1369488844},{"type":"v1","addr":"172.21.15.150:6835","nonce":1369488844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3786232682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3786232682},{"type":"v1","addr":"172.21.15.53:6837","nonce":3786232682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":22,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T14:38:10.037451+0000","modified":"2024-04-25T14:40:40.222021+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24584,"mds_1":24565,"mds_2":24560},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24532":{"gid":24532,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1189328602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1189328602},{"type":"v1","addr":"172.21.15.183:6835","nonce":1189328602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/2344343627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2344343627},{"type":"v1","addr":"172.21.15.150:6837","nonce":2344343627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24544":{"gid":24544,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1163336384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1163336384},{"type":"v1","addr":"172.21.15.183:6837","nonce":1163336384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6839/570000540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":570000540},{"type":"v1","addr":"172.21.15.53:6839","nonce":570000540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.183:6839/1554258915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1554258915},{"type":"v1","addr":"172.21.15.183:6839","nonce":1554258915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/149573616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":149573616},{"type":"v1","addr":"172.21.15.150:6839","nonce":149573616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24560,24565,24584]},"id":1}]} 2024-04-25T14:41:00.075 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 22 2024-04-25T14:41:00.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.077+0000 7f66bee00640 1 -- 172.21.15.53:0/3363122735 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f669c0a3db0 msgr2=0x7f669c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:41:00.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.077+0000 7f66bee00640 1 --2- 172.21.15.53:0/3363122735 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f669c0a3db0 0x7f669c0a6270 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f66a8006010 tx=0x7f66a8009820 comp rx=0 tx=0).stop 2024-04-25T14:41:00.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.077+0000 7f66bee00640 1 -- 172.21.15.53:0/3363122735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66b8109460 msgr2=0x7f66b81a0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:41:00.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.077+0000 7f66bee00640 1 --2- 172.21.15.53:0/3363122735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66b8109460 0x7f66b81a0b60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f66a000e9a0 tx=0x7f66a000ee60 comp rx=0 tx=0).stop 2024-04-25T14:41:00.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.077+0000 7f66bee00640 1 -- 172.21.15.53:0/3363122735 shutdown_connections 2024-04-25T14:41:00.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.077+0000 7f66bee00640 1 --2- 172.21.15.53:0/3363122735 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f669c0a3db0 0x7f669c0a6270 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:41:00.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.077+0000 7f66bee00640 1 --2- 172.21.15.53:0/3363122735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66b819a790 0x7f66b819ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:41:00.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.077+0000 7f66bee00640 1 --2- 172.21.15.53:0/3363122735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66b8109460 0x7f66b81a0b60 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:41:00.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.077+0000 7f66bee00640 1 --2- 172.21.15.53:0/3363122735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66b8104390 0x7f66b81a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:41:00.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.077+0000 7f66bee00640 1 -- 172.21.15.53:0/3363122735 >> 172.21.15.53:0/3363122735 conn(0x7f66b80ffc50 msgr2=0x7f66b8100fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:41:00.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.078+0000 7f66bee00640 1 -- 172.21.15.53:0/3363122735 shutdown_connections 2024-04-25T14:41:00.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.078+0000 7f66bee00640 1 -- 172.21.15.53:0/3363122735 wait complete. 2024-04-25T14:41:00.097 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:41:00.105 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:59 smithi183 ceph-mon[32584]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 76 KiB/s wr, 49 op/s 2024-04-25T14:41:00.105 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:59 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/4169885948' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:41:00.105 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:40:59 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/1773784107' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:41:00.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:59 smithi150 ceph-mon[32504]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 76 KiB/s wr, 49 op/s 2024-04-25T14:41:00.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:59 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/4169885948' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:41:00.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:40:59 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/1773784107' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:41:00.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.316+0000 7f542f200640 1 -- 172.21.15.53:0/1025150262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54301041b0 msgr2=0x7f5430104590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:41:00.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.316+0000 7f542f200640 1 --2- 172.21.15.53:0/1025150262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54301041b0 0x7f5430104590 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5420009860 tx=0x7f542002a340 comp rx=0 tx=0).stop 2024-04-25T14:41:00.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.316+0000 7f542f200640 1 -- 172.21.15.53:0/1025150262 shutdown_connections 2024-04-25T14:41:00.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.316+0000 7f542f200640 1 --2- 172.21.15.53:0/1025150262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5430109280 0x7f5430110e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:41:00.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.316+0000 7f542f200640 1 --2- 172.21.15.53:0/1025150262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5430104b60 0x7f5430108ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:41:00.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.316+0000 7f542f200640 1 --2- 172.21.15.53:0/1025150262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54301041b0 0x7f5430104590 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:41:00.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.316+0000 7f542f200640 1 -- 172.21.15.53:0/1025150262 >> 172.21.15.53:0/1025150262 conn(0x7f54300ff9f0 msgr2=0x7f5430101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:41:00.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.316+0000 7f542f200640 1 -- 172.21.15.53:0/1025150262 shutdown_connections 2024-04-25T14:41:00.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.316+0000 7f542f200640 1 -- 172.21.15.53:0/1025150262 wait complete. 2024-04-25T14:41:00.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.317+0000 7f542f200640 1 Processor -- start 2024-04-25T14:41:00.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.317+0000 7f542f200640 1 -- start start 2024-04-25T14:41:00.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.318+0000 7f542f200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54301041b0 0x7f54301a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:41:00.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.318+0000 7f542f200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5430109280 0x7f54301a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:41:00.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.318+0000 7f542f200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f543019a600 0x7f543019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:41:00.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.318+0000 7f542f200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f54301141c0 con 0x7f5430109280 2024-04-25T14:41:00.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.318+0000 7f542f200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5430114040 con 0x7f543019a600 2024-04-25T14:41:00.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.318+0000 7f542f200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5430114340 con 0x7f54301041b0 2024-04-25T14:41:00.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.318+0000 7f542de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54301041b0 0x7f54301a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:41:00.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.318+0000 7f542de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54301041b0 0x7f54301a0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:49374/0 (socket says 172.21.15.53:49374) 2024-04-25T14:41:00.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.318+0000 7f542de00640 1 -- 172.21.15.53:0/57107861 learned_addr learned my addr 172.21.15.53:0/57107861 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:41:00.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.318+0000 7f542d400640 1 --2- 172.21.15.53:0/57107861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5430109280 0x7f54301a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:41:00.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.318+0000 7f542e800640 1 --2- 172.21.15.53:0/57107861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f543019a600 0x7f543019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:41:00.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.319+0000 7f542de00640 1 -- 172.21.15.53:0/57107861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f543019a600 msgr2=0x7f543019aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:41:00.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.319+0000 7f542de00640 1 --2- 172.21.15.53:0/57107861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f543019a600 0x7f543019aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:41:00.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.319+0000 7f542de00640 1 -- 172.21.15.53:0/57107861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5430109280 msgr2=0x7f54301a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:41:00.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.319+0000 7f542de00640 1 --2- 172.21.15.53:0/57107861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5430109280 0x7f54301a0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:41:00.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.319+0000 7f542de00640 1 -- 172.21.15.53:0/57107861 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f543019b120 con 0x7f54301041b0 2024-04-25T14:41:00.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.319+0000 7f542e800640 1 --2- 172.21.15.53:0/57107861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f543019a600 0x7f543019aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:41:00.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.319+0000 7f542de00640 1 --2- 172.21.15.53:0/57107861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54301041b0 0x7f54301a0450 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f54200303f0 tx=0x7f5420032da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:41:00.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.319+0000 7f5426a00640 1 -- 172.21.15.53:0/57107861 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5420040020 con 0x7f54301041b0 2024-04-25T14:41:00.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.319+0000 7f5426a00640 1 -- 172.21.15.53:0/57107861 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5420030dd0 con 0x7f54301041b0 2024-04-25T14:41:00.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.319+0000 7f542f200640 1 -- 172.21.15.53:0/57107861 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f543019b3b0 con 0x7f54301041b0 2024-04-25T14:41:00.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.319+0000 7f5426a00640 1 -- 172.21.15.53:0/57107861 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5420037040 con 0x7f54301041b0 2024-04-25T14:41:00.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.319+0000 7f542f200640 1 -- 172.21.15.53:0/57107861 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54301a75a0 con 0x7f54301041b0 2024-04-25T14:41:00.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.320+0000 7f542f200640 1 -- 172.21.15.53:0/57107861 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54300690e0 con 0x7f54301041b0 2024-04-25T14:41:00.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.322+0000 7f5426a00640 1 -- 172.21.15.53:0/57107861 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7f5420043020 con 0x7f54301041b0 2024-04-25T14:41:00.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.324+0000 7f5426a00640 1 --2- 172.21.15.53:0/57107861 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f54140a3db0 0x7f54140a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:41:00.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.324+0000 7f5426a00640 1 -- 172.21.15.53:0/57107861 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8543+0+0 (secure 0 0 0) 0x7f54200f05e0 con 0x7f54301041b0 2024-04-25T14:41:00.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.324+0000 7f542d400640 1 --2- 172.21.15.53:0/57107861 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f54140a3db0 0x7f54140a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:41:00.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.325+0000 7f542d400640 1 --2- 172.21.15.53:0/57107861 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f54140a3db0 0x7f54140a6270 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f54300ffe10 tx=0x7f541c009260 comp rx=0 tx=0).ready entity=mgr.15093 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:41:00.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.328+0000 7f5426a00640 1 -- 172.21.15.53:0/57107861 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f542004a020 con 0x7f54301041b0 2024-04-25T14:41:00.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.558+0000 7f542f200640 1 -- 172.21.15.53:0/57107861 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f543010c510 con 0x7f54301041b0 2024-04-25T14:41:00.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.560+0000 7f5426a00640 1 -- 172.21.15.53:0/57107861 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+23108 (secure 0 0 0) 0x7f54200328c0 con 0x7f54301041b0 2024-04-25T14:41:00.561 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:41:00.561 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":93,"fsid":"fd30e6ce-030f-11ef-bc93-c7b262605968","created":"2024-04-25T14:28:54.330326+0000","modified":"2024-04-25T14:40:40.169020+0000","last_up_change":"2024-04-25T14:37:10.921148+0000","last_in_change":"2024-04-25T14:36:49.566459+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:33:02.338115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:38:07.517974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:38:08.994122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aecfbaf0-39da-4749-a49b-c7205dabd4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6803","nonce":464176015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6805","nonce":464176015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6809","nonce":464176015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6807","nonce":464176015}]},"public_addr":"172.21.15.53:6803/464176015","cluster_addr":"172.21.15.53:6805/464176015","heartbeat_back_addr":"172.21.15.53:6809/464176015","heartbeat_front_addr":"172.21.15.53:6807/464176015","state":["exists","up"]},{"osd":1,"uuid":"2c4a21ae-c43e-4839-8a56-c3fc63aeb7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6801","nonce":1304621982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6803","nonce":1304621982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6807","nonce":1304621982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6805","nonce":1304621982}]},"public_addr":"172.21.15.150:6801/1304621982","cluster_addr":"172.21.15.150:6803/1304621982","heartbeat_back_addr":"172.21.15.150:6807/1304621982","heartbeat_front_addr":"172.21.15.150:6805/1304621982","state":["exists","up"]},{"osd":2,"uuid":"e8effd43-adc0-472a-9eed-a9273b06dc0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6801","nonce":574509311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6803","nonce":574509311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6807","nonce":574509311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6805","nonce":574509311}]},"public_addr":"172.21.15.183:6801/574509311","cluster_addr":"172.21.15.183:6803/574509311","heartbeat_back_addr":"172.21.15.183:6807/574509311","heartbeat_front_addr":"172.21.15.183:6805/574509311","state":["exists","up"]},{"osd":3,"uuid":"e7cfced4-9b40-40de-a303-75ea8d5df4a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6811","nonce":570672219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6813","nonce":570672219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6817","nonce":570672219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6815","nonce":570672219}]},"public_addr":"172.21.15.53:6811/570672219","cluster_addr":"172.21.15.53:6813/570672219","heartbeat_back_addr":"172.21.15.53:6817/570672219","heartbeat_front_addr":"172.21.15.53:6815/570672219","state":["exists","up"]},{"osd":4,"uuid":"44a9e070-2282-43f1-9f05-b1f8ff81e540","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6809","nonce":1526850103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6811","nonce":1526850103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6815","nonce":1526850103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6813","nonce":1526850103}]},"public_addr":"172.21.15.150:6809/1526850103","cluster_addr":"172.21.15.150:6811/1526850103","heartbeat_back_addr":"172.21.15.150:6815/1526850103","heartbeat_front_addr":"172.21.15.150:6813/1526850103","state":["exists","up"]},{"osd":5,"uuid":"2c5619d2-c0f9-47d6-bf7e-2f4afd1b09f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6809","nonce":773726576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6811","nonce":773726576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6815","nonce":773726576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6813","nonce":773726576}]},"public_addr":"172.21.15.183:6809/773726576","cluster_addr":"172.21.15.183:6811/773726576","heartbeat_back_addr":"172.21.15.183:6815/773726576","heartbeat_front_addr":"172.21.15.183:6813/773726576","state":["exists","up"]},{"osd":6,"uuid":"1cf74d94-c118-4738-b6db-040af458a8eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6819","nonce":355884319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6821","nonce":355884319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6825","nonce":355884319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6823","nonce":355884319}]},"public_addr":"172.21.15.53:6819/355884319","cluster_addr":"172.21.15.53:6821/355884319","heartbeat_back_addr":"172.21.15.53:6825/355884319","heartbeat_front_addr":"172.21.15.53:6823/355884319","state":["exists","up"]},{"osd":7,"uuid":"0ce97c04-cd41-4080-bbbc-be9cf5fadc02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6817","nonce":229022562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6819","nonce":229022562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6823","nonce":229022562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6821","nonce":229022562}]},"public_addr":"172.21.15.150:6817/229022562","cluster_addr":"172.21.15.150:6819/229022562","heartbeat_back_addr":"172.21.15.150:6823/229022562","heartbeat_front_addr":"172.21.15.150:6821/229022562","state":["exists","up"]},{"osd":8,"uuid":"5d57787d-2a7c-4514-bd01-357ced413f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6817","nonce":301003844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6819","nonce":301003844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6823","nonce":301003844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6821","nonce":301003844}]},"public_addr":"172.21.15.183:6817/301003844","cluster_addr":"172.21.15.183:6819/301003844","heartbeat_back_addr":"172.21.15.183:6823/301003844","heartbeat_front_addr":"172.21.15.183:6821/301003844","state":["exists","up"]},{"osd":9,"uuid":"e11dd8eb-c55a-4d70-8c58-457540b7a50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6827","nonce":3449260978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6829","nonce":3449260978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6833","nonce":3449260978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6831","nonce":3449260978}]},"public_addr":"172.21.15.53:6827/3449260978","cluster_addr":"172.21.15.53:6829/3449260978","heartbeat_back_addr":"172.21.15.53:6833/3449260978","heartbeat_front_addr":"172.21.15.53:6831/3449260978","state":["exists","up"]},{"osd":10,"uuid":"5de4803d-ad37-4a90-add5-0048f44fc4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6825","nonce":3435992308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6827","nonce":3435992308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6831","nonce":3435992308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6829","nonce":3435992308}]},"public_addr":"172.21.15.150:6825/3435992308","cluster_addr":"172.21.15.150:6827/3435992308","heartbeat_back_addr":"172.21.15.150:6831/3435992308","heartbeat_front_addr":"172.21.15.150:6829/3435992308","state":["exists","up"]},{"osd":11,"uuid":"0e819a68-ed56-477e-b9c0-81e8cc0bce11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6825","nonce":1957842686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6827","nonce":1957842686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6831","nonce":1957842686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6829","nonce":1957842686}]},"public_addr":"172.21.15.183:6825/1957842686","cluster_addr":"172.21.15.183:6827/1957842686","heartbeat_back_addr":"172.21.15.183:6831/1957842686","heartbeat_front_addr":"172.21.15.183:6829/1957842686","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:09.840629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:34.769537+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:58.112969+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:33:29.656790+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:33:55.181287+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:34:19.772645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:34:53.833921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:35:19.076733+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:35:43.163390+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:36:16.568340+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:36:42.492768+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:37:07.944639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"3.38","mappings":[{"from":5,"to":7},{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6800/4154933108":"2024-04-26T14:40:40.168940+0000","172.21.15.53:0/3916258673":"2024-04-26T14:40:40.168940+0000","172.21.15.53:0/309091034":"2024-04-26T14:40:40.168940+0000","172.21.15.53:0/4267178590":"2024-04-26T14:40:40.168940+0000","172.21.15.53:0/920465993":"2024-04-26T14:40:40.168940+0000","172.21.15.53:6800/3887954889":"2024-04-26T14:30:14.162383+0000","172.21.15.53:6801/4154933108":"2024-04-26T14:40:40.168940+0000","172.21.15.53:0/1014501022":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/2576091441":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/1233527569":"2024-04-26T14:29:38.106242+0000","172.21.15.53:6801/3887954889":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/931889810":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/177196141":"2024-04-26T14:30:14.162383+0000","172.21.15.53:6800/1722497390":"2024-04-26T14:29:38.106242+0000","172.21.15.53:6801/1722497390":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/1927549198":"2024-04-26T14:30:14.162383+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:41:00.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.563+0000 7f542f200640 1 -- 172.21.15.53:0/57107861 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f54140a3db0 msgr2=0x7f54140a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:41:00.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.563+0000 7f542f200640 1 --2- 172.21.15.53:0/57107861 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f54140a3db0 0x7f54140a6270 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f54300ffe10 tx=0x7f541c009260 comp rx=0 tx=0).stop 2024-04-25T14:41:00.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.563+0000 7f542f200640 1 -- 172.21.15.53:0/57107861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54301041b0 msgr2=0x7f54301a0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:41:00.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.563+0000 7f542f200640 1 --2- 172.21.15.53:0/57107861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54301041b0 0x7f54301a0450 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f54200303f0 tx=0x7f5420032da0 comp rx=0 tx=0).stop 2024-04-25T14:41:00.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.564+0000 7f542f200640 1 -- 172.21.15.53:0/57107861 shutdown_connections 2024-04-25T14:41:00.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.564+0000 7f542f200640 1 --2- 172.21.15.53:0/57107861 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f54140a3db0 0x7f54140a6270 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:41:00.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.564+0000 7f542f200640 1 --2- 172.21.15.53:0/57107861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f543019a600 0x7f543019aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:41:00.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.564+0000 7f542f200640 1 --2- 172.21.15.53:0/57107861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5430109280 0x7f54301a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:41:00.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.564+0000 7f542f200640 1 --2- 172.21.15.53:0/57107861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54301041b0 0x7f54301a0450 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:41:00.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.564+0000 7f542f200640 1 -- 172.21.15.53:0/57107861 >> 172.21.15.53:0/57107861 conn(0x7f54300ff9f0 msgr2=0x7f5430100e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:41:00.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.564+0000 7f542f200640 1 -- 172.21.15.53:0/57107861 shutdown_connections 2024-04-25T14:41:00.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:41:00.564+0000 7f542f200640 1 -- 172.21.15.53:0/57107861 wait complete. 2024-04-25T14:41:00.583 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get run_dir 2024-04-25T14:41:00.653 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:41:00 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:41:00.393+0000 7f4828e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:41:00.653 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:41:00 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:41:00.393+0000 7f4828e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:41:00.653 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:41:00 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:41:00.393+0000 7f4828e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:41:00.653 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:41:00 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:41:00.393+0000 7f4828e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:41:00.653 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:41:00 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:41:00.393+0000 7f4828e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:41:00.776 INFO:teuthology.orchestra.run.smithi053.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2024-04-25T14:41:00.785 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-25T14:41:00.785 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa/tasks/quiescer.py", line 403, in task quiescer = Quiescer(ctx=ctx, fscid=fs['id'], mds_cluster=mds_cluster, cluster_name=cluster_name, **config) File "/home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa/tasks/quiescer.py", line 72, in __init__ self.run_dir = mds_cluster.get_config("run_dir") File "/home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa/tasks/cephfs/filesystem.py", line 353, in get_config return super(MDSClusterBase, self).get_config(key, service_type) File "/home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa/tasks/cephfs/filesystem.py", line 249, in get_config return self.json_asok(['config', 'get', key], service_type, service_id)[key] File "/home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa/tasks/cephfs/filesystem.py", line 266, in json_asok proc = self.mon_manager.admin_socket(service_type, service_id, command, timeout=timeout) File "/home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa/tasks/ceph_manager.py", line 1922, in admin_socket return remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi053 with status 22: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get run_dir' 2024-04-25T14:41:01.025 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=7b3a822e70ef45d2877e0db6610e3fba Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa/tasks/quiescer.py", line 403, in task quiescer = Quiescer(ctx=ctx, fscid=fs['id'], mds_cluster=mds_cluster, cluster_name=cluster_name, **config) File "/home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa/tasks/quiescer.py", line 72, in __init__ self.run_dir = mds_cluster.get_config("run_dir") File "/home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa/tasks/cephfs/filesystem.py", line 353, in get_config return super(MDSClusterBase, self).get_config(key, service_type) File "/home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa/tasks/cephfs/filesystem.py", line 249, in get_config return self.json_asok(['config', 'get', key], service_type, service_id)[key] File "/home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa/tasks/cephfs/filesystem.py", line 266, in json_asok proc = self.mon_manager.admin_socket(service_type, service_id, command, timeout=timeout) File "/home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa/tasks/ceph_manager.py", line 1922, in admin_socket return remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi053 with status 22: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get run_dir' 2024-04-25T14:41:01.029 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-04-25T14:41:01.039 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-25T14:41:01.047 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi053.front.sepia.ceph.com 2024-04-25T14:41:01.051 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:41:00 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/3363122735' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:41:01.051 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:41:00 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/57107861' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:41:01.074 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-25T14:41:01.084 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-25T14:41:01.085 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T14:41:01.085 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi053.front.sepia.ceph.com 2024-04-25T14:41:01.085 DEBUG:teuthology.orchestra.run.smithi053:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-25T14:41:01.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:41:00 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/3363122735' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:41:01.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:41:00 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/57107861' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:41:01.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:41:00 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/3363122735' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:41:01.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:41:00 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/57107861' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:41:01.217 DEBUG:teuthology.orchestra.run.smithi053:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-25T14:41:02.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:41:01 smithi053 ceph-mon[27773]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:41:02.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:41:01 smithi183 ceph-mon[32584]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:41:02.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:41:01 smithi150 ceph-mon[32504]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:41:03.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:41:02 smithi053 ceph-mon[27773]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-04-25T14:41:03.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:41:02 smithi183 ceph-mon[32584]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-04-25T14:41:03.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:41:02 smithi150 ceph-mon[32504]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-04-25T14:41:03.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:41:03 smithi150 ceph-mon[32504]: mgrmap e26: x(active, since 22s), standbys: y, z 2024-04-25T14:41:04.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:41:03 smithi053 ceph-mon[27773]: mgrmap e26: x(active, since 22s), standbys: y, z 2024-04-25T14:41:04.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:41:03 smithi183 ceph-mon[32584]: mgrmap e26: x(active, since 22s), standbys: y, z 2024-04-25T14:41:05.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:41:04 smithi053 ceph-mon[27773]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-25T14:41:05.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:41:04 smithi183 ceph-mon[32584]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-25T14:41:05.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:41:04 smithi150 ceph-mon[32504]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-25T14:41:07.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:41:07 smithi053 ceph-mon[27773]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-25T14:41:07.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:41:07 smithi183 ceph-mon[32584]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-25T14:41:07.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:41:07 smithi150 ceph-mon[32504]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-25T14:41:09.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:41:09 smithi053 ceph-mon[27773]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-04-25T14:41:09.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:41:09 smithi183 ceph-mon[32584]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-04-25T14:41:09.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:41:09 smithi150 ceph-mon[32504]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-04-25T14:41:11.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:41:11 smithi053 ceph-mon[27773]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-25T14:41:11.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:41:11 smithi183 ceph-mon[32584]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-25T14:41:11.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:41:11 smithi150 ceph-mon[32504]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-25T14:41:13.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:41:13 smithi053 ceph-mon[27773]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-04-25T14:41:13.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:41:13 smithi183 ceph-mon[32584]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-04-25T14:41:13.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:41:13 smithi150 ceph-mon[32504]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-04-25T14:41:15.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:41:15 smithi183 ceph-mon[32584]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T14:41:15.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:41:15 smithi150 ceph-mon[32504]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T14:41:15.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:41:15 smithi053 ceph-mon[27773]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T14:41:17.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:41:17 smithi183 ceph-mon[32584]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T14:41:17.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:41:17 smithi150 ceph-mon[32504]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T14:41:17.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:41:17 smithi053 ceph-mon[27773]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T14:41:19.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:41:19 smithi183 ceph-mon[32584]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T14:41:19.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:41:19 smithi150 ceph-mon[32504]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T14:41:19.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:41:19 smithi053 ceph-mon[27773]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T14:41:21.532 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:41:21 smithi053 ceph-mon[27773]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T14:41:21.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:41:21 smithi183 ceph-mon[32584]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T14:41:21.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:41:21 smithi150 ceph-mon[32504]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T14:41:23.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:41:23 smithi183 ceph-mon[32584]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T14:41:23.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:41:23 smithi150 ceph-mon[32504]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T14:41:23.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:41:23 smithi053 ceph-mon[27773]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T14:41:25.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:41:25 smithi183 ceph-mon[32584]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:41:25.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:41:25 smithi150 ceph-mon[32504]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:41:25.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:41:25 smithi053 ceph-mon[27773]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:41:27.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:41:27 smithi183 ceph-mon[32584]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:41:27.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:41:27 smithi150 ceph-mon[32504]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:41:27.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:41:27 smithi053 ceph-mon[27773]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:41:29.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:41:29 smithi183 ceph-mon[32584]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:41:29.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:41:29 smithi150 ceph-mon[32504]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:41:29.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:41:29 smithi053 ceph-mon[27773]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:41:31.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:41:31 smithi183 ceph-mon[32584]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:41:31.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:41:31 smithi150 ceph-mon[32504]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:41:31.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:41:31 smithi053 ceph-mon[27773]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:41:33.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:41:33 smithi183 ceph-mon[32584]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:41:33.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:41:33 smithi150 ceph-mon[32504]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:41:33.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:41:33 smithi053 ceph-mon[27773]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:41:35.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:41:35 smithi183 ceph-mon[32584]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:41:35.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:41:35 smithi150 ceph-mon[32504]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:41:35.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:41:35 smithi053 ceph-mon[27773]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:41:37.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:41:37 smithi183 ceph-mon[32584]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T14:41:37.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:41:37 smithi150 ceph-mon[32504]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T14:41:37.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:41:37 smithi053 ceph-mon[27773]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T14:41:39.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:41:39 smithi183 ceph-mon[32584]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:41:39.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:41:39 smithi150 ceph-mon[32504]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:41:39.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:41:39 smithi053 ceph-mon[27773]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:41:41.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:41:41 smithi183 ceph-mon[32584]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:41:41.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:41:41 smithi150 ceph-mon[32504]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:41:41.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:41:41 smithi053 ceph-mon[27773]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:41:43.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:41:43 smithi183 ceph-mon[32584]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:41:43.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:41:43 smithi150 ceph-mon[32504]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:41:43.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:41:43 smithi053 ceph-mon[27773]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:41:45.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:41:45 smithi183 ceph-mon[32584]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:41:45.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:41:45 smithi150 ceph-mon[32504]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:41:45.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:41:45 smithi053 ceph-mon[27773]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:41:47.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:41:47 smithi183 ceph-mon[32584]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T14:41:47.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:41:47 smithi150 ceph-mon[32504]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T14:41:47.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:41:47 smithi053 ceph-mon[27773]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T14:41:49.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:41:49 smithi183 ceph-mon[32584]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:41:49.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:41:49 smithi150 ceph-mon[32504]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:41:49.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:41:49 smithi053 ceph-mon[27773]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:41:51.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:41:51 smithi183 ceph-mon[32584]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:41:51.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:41:51 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:41:51.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:41:51 smithi150 ceph-mon[32504]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:41:51.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:41:51 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:41:51.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:41:51 smithi053 ceph-mon[27773]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:41:51.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:41:51 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:41:53.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:41:52 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:41:53.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:41:52 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:41:53.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:41:52 smithi183 ceph-mon[32584]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:41:53.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:41:52 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:41:53.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:41:52 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:41:53.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:41:52 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:41:53.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:41:52 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:41:53.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:41:52 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:41:53.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:41:52 smithi150 ceph-mon[32504]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:41:53.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:41:52 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:41:53.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:41:52 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:41:53.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:41:52 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:41:53.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:41:52 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:41:53.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:41:52 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:41:53.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:41:52 smithi053 ceph-mon[27773]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:41:53.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:41:52 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:41:53.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:41:52 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:41:53.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:41:52 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:41:55.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:41:55 smithi183 ceph-mon[32584]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:41:55.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:41:55 smithi150 ceph-mon[32504]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:41:55.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:41:55 smithi053 ceph-mon[27773]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:41:57.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:41:57 smithi183 ceph-mon[32584]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-25T14:41:57.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:41:57 smithi150 ceph-mon[32504]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-25T14:41:57.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:41:57 smithi053 ceph-mon[27773]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-25T14:41:59.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:41:59 smithi183 ceph-mon[32584]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:41:59.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:41:59 smithi150 ceph-mon[32504]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:41:59.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:41:59 smithi053 ceph-mon[27773]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:42:01.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:42:01 smithi183 ceph-mon[32584]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:42:01.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:42:01 smithi150 ceph-mon[32504]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:42:01.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:42:01 smithi053 ceph-mon[27773]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:42:03.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:42:03 smithi183 ceph-mon[32584]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T14:42:03.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:42:03 smithi150 ceph-mon[32504]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T14:42:03.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:42:03 smithi053 ceph-mon[27773]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T14:42:05.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:42:05 smithi183 ceph-mon[32584]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-25T14:42:05.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:42:05 smithi150 ceph-mon[32504]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-25T14:42:05.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:42:05 smithi053 ceph-mon[27773]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-25T14:42:07.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:42:07 smithi183 ceph-mon[32584]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-25T14:42:07.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:42:07 smithi150 ceph-mon[32504]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-25T14:42:07.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:42:07 smithi053 ceph-mon[27773]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-25T14:42:09.585 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:42:09 smithi053 ceph-mon[27773]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T14:42:09.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:42:09 smithi183 ceph-mon[32584]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T14:42:09.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:42:09 smithi150 ceph-mon[32504]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T14:42:11.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:42:11 smithi183 ceph-mon[32584]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-25T14:42:11.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:42:11 smithi150 ceph-mon[32504]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-25T14:42:11.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:42:11 smithi053 ceph-mon[27773]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-25T14:42:13.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:42:13 smithi183 ceph-mon[32584]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-25T14:42:13.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:42:13 smithi183 ceph-mon[32584]: mgrmap e27: x(active, since 92s), standbys: y, z 2024-04-25T14:42:13.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:42:13 smithi150 ceph-mon[32504]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-25T14:42:13.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:42:13 smithi150 ceph-mon[32504]: mgrmap e27: x(active, since 92s), standbys: y, z 2024-04-25T14:42:13.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:42:13 smithi053 ceph-mon[27773]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-25T14:42:13.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:42:13 smithi053 ceph-mon[27773]: mgrmap e27: x(active, since 92s), standbys: y, z 2024-04-25T14:42:15.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:42:15 smithi183 ceph-mon[32584]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-25T14:42:15.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:42:15 smithi150 ceph-mon[32504]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-25T14:42:15.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:42:15 smithi053 ceph-mon[27773]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-25T14:42:17.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:42:17 smithi183 ceph-mon[32584]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-25T14:42:17.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:42:17 smithi150 ceph-mon[32504]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-25T14:42:17.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:42:17 smithi053 ceph-mon[27773]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-25T14:42:19.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:42:19 smithi183 ceph-mon[32584]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-25T14:42:19.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:42:19 smithi150 ceph-mon[32504]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-25T14:42:19.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:42:19 smithi053 ceph-mon[27773]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-25T14:42:21.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:42:21 smithi183 ceph-mon[32584]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-25T14:42:21.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:42:21 smithi150 ceph-mon[32504]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-25T14:42:21.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:42:21 smithi053 ceph-mon[27773]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-25T14:42:23.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:42:23 smithi183 ceph-mon[32584]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 682 B/s wr, 8 op/s 2024-04-25T14:42:23.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:42:23 smithi150 ceph-mon[32504]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 682 B/s wr, 8 op/s 2024-04-25T14:42:23.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:42:23 smithi053 ceph-mon[27773]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 682 B/s wr, 8 op/s 2024-04-25T14:42:25.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:42:25 smithi183 ceph-mon[32584]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T14:42:25.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:42:25 smithi150 ceph-mon[32504]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T14:42:25.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:42:25 smithi053 ceph-mon[27773]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T14:42:27.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:42:27 smithi183 ceph-mon[32584]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T14:42:27.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:42:27 smithi150 ceph-mon[32504]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T14:42:27.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:42:27 smithi053 ceph-mon[27773]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T14:42:29.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:42:29 smithi183 ceph-mon[32584]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:42:29.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:42:29 smithi150 ceph-mon[32504]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:42:29.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:42:29 smithi053 ceph-mon[27773]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:42:31.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:42:31 smithi183 ceph-mon[32584]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:42:31.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:42:31 smithi150 ceph-mon[32504]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:42:31.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:42:31 smithi053 ceph-mon[27773]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:42:33.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:42:33 smithi183 ceph-mon[32584]: pgmap v59: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:42:33.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:42:33 smithi150 ceph-mon[32504]: pgmap v59: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:42:33.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:42:33 smithi053 ceph-mon[27773]: pgmap v59: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:42:35.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:42:35 smithi183 ceph-mon[32584]: pgmap v60: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T14:42:35.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:42:35 smithi150 ceph-mon[32504]: pgmap v60: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T14:42:35.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:42:35 smithi053 ceph-mon[27773]: pgmap v60: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T14:42:37.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:42:37 smithi183 ceph-mon[32584]: pgmap v61: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:42:37.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:42:37 smithi150 ceph-mon[32504]: pgmap v61: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:42:37.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:42:37 smithi053 ceph-mon[27773]: pgmap v61: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:42:39.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:42:39 smithi183 ceph-mon[32584]: pgmap v62: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T14:42:39.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:42:39 smithi150 ceph-mon[32504]: pgmap v62: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T14:42:39.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:42:39 smithi053 ceph-mon[27773]: pgmap v62: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T14:42:41.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:42:41 smithi183 ceph-mon[32584]: pgmap v63: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T14:42:41.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:42:41 smithi150 ceph-mon[32504]: pgmap v63: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T14:42:41.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:42:41 smithi053 ceph-mon[27773]: pgmap v63: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T14:42:43.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:42:43 smithi183 ceph-mon[32584]: pgmap v64: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:42:43.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:42:43 smithi150 ceph-mon[32504]: pgmap v64: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:42:43.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:42:43 smithi053 ceph-mon[27773]: pgmap v64: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:42:45.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:42:45 smithi183 ceph-mon[32584]: pgmap v65: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:42:45.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:42:45 smithi150 ceph-mon[32504]: pgmap v65: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:42:45.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:42:45 smithi053 ceph-mon[27773]: pgmap v65: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:42:47.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:42:47 smithi183 ceph-mon[32584]: pgmap v66: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:42:47.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:42:47 smithi150 ceph-mon[32504]: pgmap v66: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:42:47.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:42:47 smithi053 ceph-mon[27773]: pgmap v66: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:42:49.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:42:49 smithi183 ceph-mon[32584]: pgmap v67: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:42:49.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:42:49 smithi150 ceph-mon[32504]: pgmap v67: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:42:49.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:42:49 smithi053 ceph-mon[27773]: pgmap v67: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:42:51.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:42:51 smithi183 ceph-mon[32584]: pgmap v68: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:42:51.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:42:51 smithi150 ceph-mon[32504]: pgmap v68: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:42:51.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:42:51 smithi053 ceph-mon[27773]: pgmap v68: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:42:53.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:42:53 smithi183 ceph-mon[32584]: pgmap v69: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:42:53.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:42:53 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:42:53.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:42:53 smithi150 ceph-mon[32504]: pgmap v69: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:42:53.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:42:53 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:42:53.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:42:53 smithi053 ceph-mon[27773]: pgmap v69: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:42:53.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:42:53 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:42:54.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:42:54 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:42:54.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:42:54 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:42:54.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:42:54 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:42:54.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:42:54 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:42:54.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:42:54 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:42:54.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:42:54 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:42:54.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:42:54 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:42:54.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:42:54 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:42:54.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:42:54 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:42:55.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:42:55 smithi150 ceph-mon[32504]: pgmap v70: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:42:55.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:42:55 smithi053 ceph-mon[27773]: pgmap v70: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:42:55.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:42:55 smithi183 ceph-mon[32584]: pgmap v70: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:42:57.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:42:57 smithi150 ceph-mon[32504]: pgmap v71: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:42:57.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:42:57 smithi053 ceph-mon[27773]: pgmap v71: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:42:57.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:42:57 smithi183 ceph-mon[32584]: pgmap v71: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:42:59.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:42:59 smithi053 ceph-mon[27773]: pgmap v72: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:42:59.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:42:59 smithi183 ceph-mon[32584]: pgmap v72: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:42:59.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:42:59 smithi150 ceph-mon[32504]: pgmap v72: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:43:01.756 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:43:01 smithi183 ceph-mon[32584]: pgmap v73: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:43:01.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:43:01 smithi053 ceph-mon[27773]: pgmap v73: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:43:01.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:43:01 smithi150 ceph-mon[32504]: pgmap v73: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:43:03.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:43:03 smithi053 ceph-mon[27773]: pgmap v74: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T14:43:03.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:43:03 smithi183 ceph-mon[32584]: pgmap v74: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T14:43:03.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:43:03 smithi150 ceph-mon[32504]: pgmap v74: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T14:43:05.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:43:05 smithi053 ceph-mon[27773]: pgmap v75: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-25T14:43:05.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:43:05 smithi183 ceph-mon[32584]: pgmap v75: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-25T14:43:05.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:43:05 smithi150 ceph-mon[32504]: pgmap v75: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-25T14:43:07.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:43:07 smithi053 ceph-mon[27773]: pgmap v76: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T14:43:07.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:43:07 smithi183 ceph-mon[32584]: pgmap v76: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T14:43:07.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:43:07 smithi150 ceph-mon[32504]: pgmap v76: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T14:43:09.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:43:09 smithi053 ceph-mon[27773]: pgmap v77: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T14:43:09.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:43:09 smithi183 ceph-mon[32584]: pgmap v77: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T14:43:09.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:43:09 smithi150 ceph-mon[32504]: pgmap v77: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T14:43:11.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:43:11 smithi053 ceph-mon[27773]: pgmap v78: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T14:43:11.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:43:11 smithi183 ceph-mon[32584]: pgmap v78: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T14:43:11.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:43:11 smithi150 ceph-mon[32504]: pgmap v78: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T14:43:13.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:43:13 smithi053 ceph-mon[27773]: pgmap v79: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T14:43:13.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:43:13 smithi183 ceph-mon[32584]: pgmap v79: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T14:43:13.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:43:13 smithi150 ceph-mon[32504]: pgmap v79: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T14:43:15.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:43:15 smithi053 ceph-mon[27773]: pgmap v80: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T14:43:15.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:43:15 smithi183 ceph-mon[32584]: pgmap v80: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T14:43:15.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:43:15 smithi150 ceph-mon[32504]: pgmap v80: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T14:43:17.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:43:17 smithi053 ceph-mon[27773]: pgmap v81: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T14:43:17.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:43:17 smithi183 ceph-mon[32584]: pgmap v81: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T14:43:17.969 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:43:17 smithi150 ceph-mon[32504]: pgmap v81: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T14:43:19.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:43:19 smithi053 ceph-mon[27773]: pgmap v82: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T14:43:19.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:43:19 smithi183 ceph-mon[32584]: pgmap v82: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T14:43:19.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:43:19 smithi150 ceph-mon[32504]: pgmap v82: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T14:43:21.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:43:21 smithi053 ceph-mon[27773]: pgmap v83: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T14:43:21.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:43:21 smithi183 ceph-mon[32584]: pgmap v83: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T14:43:21.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:43:21 smithi150 ceph-mon[32504]: pgmap v83: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T14:43:23.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:43:23 smithi053 ceph-mon[27773]: pgmap v84: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T14:43:23.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:43:23 smithi183 ceph-mon[32584]: pgmap v84: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T14:43:23.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:43:23 smithi150 ceph-mon[32504]: pgmap v84: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T14:43:25.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:43:25 smithi053 ceph-mon[27773]: pgmap v85: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:43:25.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:43:25 smithi183 ceph-mon[32584]: pgmap v85: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:43:25.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:43:25 smithi150 ceph-mon[32504]: pgmap v85: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:43:27.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:43:27 smithi183 ceph-mon[32584]: pgmap v86: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:43:27.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:43:27 smithi150 ceph-mon[32504]: pgmap v86: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:43:28.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:43:27 smithi053 ceph-mon[27773]: pgmap v86: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:43:29.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:43:29 smithi183 ceph-mon[32584]: pgmap v87: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:43:29.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:43:29 smithi150 ceph-mon[32504]: pgmap v87: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:43:30.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:43:29 smithi053 ceph-mon[27773]: pgmap v87: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:43:31.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:43:31 smithi183 ceph-mon[32584]: pgmap v88: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:43:31.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:43:31 smithi150 ceph-mon[32504]: pgmap v88: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:43:32.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:43:31 smithi053 ceph-mon[27773]: pgmap v88: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:43:33.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:43:33 smithi183 ceph-mon[32584]: pgmap v89: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:43:33.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:43:33 smithi150 ceph-mon[32504]: pgmap v89: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:43:34.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:43:33 smithi053 ceph-mon[27773]: pgmap v89: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:43:35.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:43:35 smithi183 ceph-mon[32584]: pgmap v90: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:43:35.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:43:35 smithi150 ceph-mon[32504]: pgmap v90: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:43:36.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:43:35 smithi053 ceph-mon[27773]: pgmap v90: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:43:37.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:43:37 smithi183 ceph-mon[32584]: pgmap v91: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:43:37.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:43:37 smithi150 ceph-mon[32504]: pgmap v91: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:43:38.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:43:37 smithi053 ceph-mon[27773]: pgmap v91: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:43:39.836 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:43:39 smithi053 ceph-mon[27773]: pgmap v92: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:43:39.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:43:39 smithi183 ceph-mon[32584]: pgmap v92: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:43:39.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:43:39 smithi150 ceph-mon[32504]: pgmap v92: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:43:41.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:43:41 smithi183 ceph-mon[32584]: pgmap v93: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:43:41.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:43:41 smithi150 ceph-mon[32504]: pgmap v93: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:43:42.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:43:41 smithi053 ceph-mon[27773]: pgmap v93: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:43:43.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:43:43 smithi183 ceph-mon[32584]: pgmap v94: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:43:43.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:43:43 smithi150 ceph-mon[32504]: pgmap v94: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:43:44.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:43:43 smithi053 ceph-mon[27773]: pgmap v94: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:43:45.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:43:45 smithi183 ceph-mon[32584]: pgmap v95: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:43:45.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:43:45 smithi150 ceph-mon[32504]: pgmap v95: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:43:46.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:43:45 smithi053 ceph-mon[27773]: pgmap v95: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:43:47.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:43:47 smithi183 ceph-mon[32584]: pgmap v96: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:43:47.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:43:47 smithi150 ceph-mon[32504]: pgmap v96: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:43:48.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:43:47 smithi053 ceph-mon[27773]: pgmap v96: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:43:49.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:43:49 smithi183 ceph-mon[32584]: pgmap v97: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:43:49.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:43:49 smithi150 ceph-mon[32504]: pgmap v97: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:43:50.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:43:49 smithi053 ceph-mon[27773]: pgmap v97: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:43:51.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:43:51 smithi183 ceph-mon[32584]: pgmap v98: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:43:51.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:43:51 smithi150 ceph-mon[32504]: pgmap v98: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:43:52.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:43:51 smithi053 ceph-mon[27773]: pgmap v98: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:43:53.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:43:53 smithi183 ceph-mon[32584]: pgmap v99: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:43:53.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:43:53 smithi150 ceph-mon[32504]: pgmap v99: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:43:54.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:43:53 smithi053 ceph-mon[27773]: pgmap v99: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:43:55.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:43:55 smithi183 ceph-mon[32584]: pgmap v100: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:43:55.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:43:55 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:43:55.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:43:55 smithi150 ceph-mon[32504]: pgmap v100: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:43:55.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:43:55 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:43:56.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:43:55 smithi053 ceph-mon[27773]: pgmap v100: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:43:56.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:43:55 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:43:56.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:43:56 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:43:56.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:43:56 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:43:56.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:43:56 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:43:56.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:43:56 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:43:56.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:43:56 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:43:56.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:43:56 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:43:57.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:43:56 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:43:57.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:43:56 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:43:57.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:43:56 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:43:57.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:43:57 smithi183 ceph-mon[32584]: pgmap v101: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:43:57.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:43:57 smithi150 ceph-mon[32504]: pgmap v101: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:43:58.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:43:57 smithi053 ceph-mon[27773]: pgmap v101: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:43:59.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:43:59 smithi183 ceph-mon[32584]: pgmap v102: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:43:59.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:43:59 smithi150 ceph-mon[32504]: pgmap v102: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:44:00.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:43:59 smithi053 ceph-mon[27773]: pgmap v102: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:44:01.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:44:01 smithi183 ceph-mon[32584]: pgmap v103: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:44:01.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:44:01 smithi150 ceph-mon[32504]: pgmap v103: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:44:02.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:44:01 smithi053 ceph-mon[27773]: pgmap v103: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:44:03.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:44:03 smithi183 ceph-mon[32584]: pgmap v104: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T14:44:03.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:44:03 smithi150 ceph-mon[32504]: pgmap v104: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T14:44:04.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:44:03 smithi053 ceph-mon[27773]: pgmap v104: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T14:44:05.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:44:05 smithi183 ceph-mon[32584]: pgmap v105: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-25T14:44:05.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:44:05 smithi150 ceph-mon[32504]: pgmap v105: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-25T14:44:06.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:44:05 smithi053 ceph-mon[27773]: pgmap v105: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-25T14:44:07.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:44:07 smithi183 ceph-mon[32584]: pgmap v106: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T14:44:07.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:44:07 smithi150 ceph-mon[32504]: pgmap v106: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T14:44:08.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:44:07 smithi053 ceph-mon[27773]: pgmap v106: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T14:44:09.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:44:09 smithi183 ceph-mon[32584]: pgmap v107: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T14:44:09.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:44:09 smithi150 ceph-mon[32504]: pgmap v107: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T14:44:10.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:44:09 smithi053 ceph-mon[27773]: pgmap v107: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T14:44:11.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:44:11 smithi183 ceph-mon[32584]: pgmap v108: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T14:44:11.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:44:11 smithi150 ceph-mon[32504]: pgmap v108: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T14:44:12.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:44:11 smithi053 ceph-mon[27773]: pgmap v108: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T14:44:13.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:44:13 smithi183 ceph-mon[32584]: pgmap v109: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T14:44:13.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:44:13 smithi150 ceph-mon[32504]: pgmap v109: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T14:44:14.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:44:13 smithi053 ceph-mon[27773]: pgmap v109: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T14:44:15.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:44:15 smithi183 ceph-mon[32584]: pgmap v110: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T14:44:15.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:44:15 smithi150 ceph-mon[32504]: pgmap v110: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T14:44:16.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:44:15 smithi053 ceph-mon[27773]: pgmap v110: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T14:44:17.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:44:17 smithi183 ceph-mon[32584]: pgmap v111: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T14:44:17.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:44:17 smithi150 ceph-mon[32504]: pgmap v111: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T14:44:18.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:44:17 smithi053 ceph-mon[27773]: pgmap v111: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T14:44:19.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:44:19 smithi183 ceph-mon[32584]: pgmap v112: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T14:44:19.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:44:19 smithi150 ceph-mon[32504]: pgmap v112: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T14:44:20.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:44:19 smithi053 ceph-mon[27773]: pgmap v112: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T14:44:21.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:44:21 smithi183 ceph-mon[32584]: pgmap v113: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T14:44:21.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:44:21 smithi150 ceph-mon[32504]: pgmap v113: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T14:44:22.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:44:21 smithi053 ceph-mon[27773]: pgmap v113: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T14:44:23.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:44:23 smithi183 ceph-mon[32584]: pgmap v114: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T14:44:23.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:44:23 smithi150 ceph-mon[32504]: pgmap v114: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T14:44:24.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:44:23 smithi053 ceph-mon[27773]: pgmap v114: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T14:44:25.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:44:25 smithi183 ceph-mon[32584]: pgmap v115: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:44:25.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:44:25 smithi150 ceph-mon[32504]: pgmap v115: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:44:26.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:44:25 smithi053 ceph-mon[27773]: pgmap v115: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:44:27.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:44:27 smithi183 ceph-mon[32584]: pgmap v116: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:44:27.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:44:27 smithi150 ceph-mon[32504]: pgmap v116: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:44:28.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:44:27 smithi053 ceph-mon[27773]: pgmap v116: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:44:29.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:44:29 smithi183 ceph-mon[32584]: pgmap v117: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:44:29.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:44:29 smithi150 ceph-mon[32504]: pgmap v117: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:44:30.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:44:29 smithi053 ceph-mon[27773]: pgmap v117: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:44:31.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:44:31 smithi183 ceph-mon[32584]: pgmap v118: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:44:31.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:44:31 smithi150 ceph-mon[32504]: pgmap v118: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:44:32.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:44:31 smithi053 ceph-mon[27773]: pgmap v118: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:44:33.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:44:33 smithi150 ceph-mon[32504]: pgmap v119: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:44:34.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:44:33 smithi053 ceph-mon[27773]: pgmap v119: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:44:34.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:44:33 smithi183 ceph-mon[32584]: pgmap v119: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:44:36.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:44:35 smithi053 ceph-mon[27773]: pgmap v120: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:44:36.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:44:35 smithi183 ceph-mon[32584]: pgmap v120: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:44:36.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:44:35 smithi150 ceph-mon[32504]: pgmap v120: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:44:38.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:44:37 smithi053 ceph-mon[27773]: pgmap v121: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:44:38.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:44:37 smithi183 ceph-mon[32584]: pgmap v121: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:44:38.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:44:37 smithi150 ceph-mon[32504]: pgmap v121: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:44:40.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:44:39 smithi053 ceph-mon[27773]: pgmap v122: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:44:40.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:44:39 smithi183 ceph-mon[32584]: pgmap v122: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:44:40.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:44:39 smithi150 ceph-mon[32504]: pgmap v122: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:44:42.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:44:41 smithi053 ceph-mon[27773]: pgmap v123: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:44:42.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:44:41 smithi183 ceph-mon[32584]: pgmap v123: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:44:42.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:44:41 smithi150 ceph-mon[32504]: pgmap v123: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:44:44.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:44:43 smithi053 ceph-mon[27773]: pgmap v124: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:44:44.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:44:43 smithi183 ceph-mon[32584]: pgmap v124: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:44:44.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:44:43 smithi150 ceph-mon[32504]: pgmap v124: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:44:46.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:44:45 smithi053 ceph-mon[27773]: pgmap v125: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:44:46.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:44:45 smithi183 ceph-mon[32584]: pgmap v125: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:44:46.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:44:45 smithi150 ceph-mon[32504]: pgmap v125: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:44:48.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:44:47 smithi053 ceph-mon[27773]: pgmap v126: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:44:48.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:44:47 smithi183 ceph-mon[32584]: pgmap v126: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:44:48.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:44:47 smithi150 ceph-mon[32504]: pgmap v126: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:44:50.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:44:49 smithi053 ceph-mon[27773]: pgmap v127: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:44:50.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:44:49 smithi183 ceph-mon[32584]: pgmap v127: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:44:50.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:44:49 smithi150 ceph-mon[32504]: pgmap v127: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:44:52.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:44:51 smithi053 ceph-mon[27773]: pgmap v128: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:44:52.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:44:51 smithi183 ceph-mon[32584]: pgmap v128: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:44:52.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:44:51 smithi150 ceph-mon[32504]: pgmap v128: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:44:54.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:44:53 smithi053 ceph-mon[27773]: pgmap v129: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:44:54.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:44:53 smithi183 ceph-mon[32584]: pgmap v129: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:44:54.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:44:53 smithi150 ceph-mon[32504]: pgmap v129: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:44:55.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:44:54 smithi183 ceph-mon[32584]: pgmap v130: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:44:55.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:44:54 smithi150 ceph-mon[32504]: pgmap v130: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:44:55.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:44:54 smithi053 ceph-mon[27773]: pgmap v130: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:44:57.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:44:56 smithi183 ceph-mon[32584]: pgmap v131: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:44:57.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:44:56 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:44:57.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:44:56 smithi150 ceph-mon[32504]: pgmap v131: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:44:57.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:44:56 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:44:57.282 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:44:56 smithi053 ceph-mon[27773]: pgmap v131: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:44:57.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:44:56 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:44:58.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:44:57 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:44:58.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:44:57 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:44:58.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:44:57 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:44:58.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:44:57 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:44:58.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:44:57 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:44:58.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:44:57 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:44:58.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:44:57 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:44:58.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:44:57 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:44:58.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:44:57 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:44:59.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:44:58 smithi183 ceph-mon[32584]: pgmap v132: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:44:59.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:44:58 smithi150 ceph-mon[32504]: pgmap v132: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:44:59.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:44:58 smithi053 ceph-mon[27773]: pgmap v132: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:01.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:45:01 smithi183 ceph-mon[32584]: pgmap v133: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:01.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:45:01 smithi150 ceph-mon[32504]: pgmap v133: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:01.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:45:01 smithi053 ceph-mon[27773]: pgmap v133: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:03.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:45:03 smithi183 ceph-mon[32584]: pgmap v134: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T14:45:03.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:45:03 smithi150 ceph-mon[32504]: pgmap v134: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T14:45:03.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:45:03 smithi053 ceph-mon[27773]: pgmap v134: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T14:45:05.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:45:05 smithi183 ceph-mon[32584]: pgmap v135: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-25T14:45:05.713 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:45:05 smithi150 ceph-mon[32504]: pgmap v135: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-25T14:45:05.786 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:45:05 smithi053 ceph-mon[27773]: pgmap v135: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-25T14:45:07.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:45:07 smithi183 ceph-mon[32584]: pgmap v136: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T14:45:07.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:45:07 smithi150 ceph-mon[32504]: pgmap v136: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T14:45:07.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:45:07 smithi053 ceph-mon[27773]: pgmap v136: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T14:45:09.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:45:09 smithi183 ceph-mon[32584]: pgmap v137: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T14:45:09.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:45:09 smithi150 ceph-mon[32504]: pgmap v137: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T14:45:09.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:45:09 smithi053 ceph-mon[27773]: pgmap v137: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T14:45:11.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:45:11 smithi183 ceph-mon[32584]: pgmap v138: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T14:45:11.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:45:11 smithi150 ceph-mon[32504]: pgmap v138: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T14:45:11.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:45:11 smithi053 ceph-mon[27773]: pgmap v138: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T14:45:13.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:45:13 smithi183 ceph-mon[32584]: pgmap v139: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T14:45:13.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:45:13 smithi150 ceph-mon[32504]: pgmap v139: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T14:45:13.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:45:13 smithi053 ceph-mon[27773]: pgmap v139: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T14:45:15.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:45:15 smithi183 ceph-mon[32584]: pgmap v140: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T14:45:15.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:45:15 smithi150 ceph-mon[32504]: pgmap v140: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T14:45:15.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:45:15 smithi053 ceph-mon[27773]: pgmap v140: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T14:45:17.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:45:17 smithi183 ceph-mon[32584]: pgmap v141: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T14:45:17.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:45:17 smithi150 ceph-mon[32504]: pgmap v141: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T14:45:17.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:45:17 smithi053 ceph-mon[27773]: pgmap v141: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T14:45:19.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:45:19 smithi183 ceph-mon[32584]: pgmap v142: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:19.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:45:19 smithi150 ceph-mon[32504]: pgmap v142: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:19.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:45:19 smithi053 ceph-mon[27773]: pgmap v142: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:21.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:45:21 smithi183 ceph-mon[32584]: pgmap v143: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:21.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:45:21 smithi150 ceph-mon[32504]: pgmap v143: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:21.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:45:21 smithi053 ceph-mon[27773]: pgmap v143: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:23.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:45:23 smithi183 ceph-mon[32584]: pgmap v144: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:45:23.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:45:23 smithi150 ceph-mon[32504]: pgmap v144: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:45:23.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:45:23 smithi053 ceph-mon[27773]: pgmap v144: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:45:25.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:45:25 smithi183 ceph-mon[32584]: pgmap v145: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:25.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:45:25 smithi150 ceph-mon[32504]: pgmap v145: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:25.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:45:25 smithi053 ceph-mon[27773]: pgmap v145: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:27.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:45:27 smithi183 ceph-mon[32584]: pgmap v146: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:45:27.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:45:27 smithi150 ceph-mon[32504]: pgmap v146: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:45:27.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:45:27 smithi053 ceph-mon[27773]: pgmap v146: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:45:29.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:45:29 smithi183 ceph-mon[32584]: pgmap v147: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:29.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:45:29 smithi150 ceph-mon[32504]: pgmap v147: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:29.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:45:29 smithi053 ceph-mon[27773]: pgmap v147: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:31.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:45:31 smithi183 ceph-mon[32584]: pgmap v148: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:31.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:45:31 smithi150 ceph-mon[32504]: pgmap v148: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:31.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:45:31 smithi053 ceph-mon[27773]: pgmap v148: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:33.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:45:33 smithi183 ceph-mon[32584]: pgmap v149: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:45:33.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:45:33 smithi150 ceph-mon[32504]: pgmap v149: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:45:33.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:45:33 smithi053 ceph-mon[27773]: pgmap v149: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:45:35.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:45:35 smithi150 ceph-mon[32504]: pgmap v150: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:35.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:45:35 smithi053 ceph-mon[27773]: pgmap v150: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:35.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:45:35 smithi183 ceph-mon[32584]: pgmap v150: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:37.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:45:37 smithi150 ceph-mon[32504]: pgmap v151: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:45:37.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:45:37 smithi053 ceph-mon[27773]: pgmap v151: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:45:37.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:45:37 smithi183 ceph-mon[32584]: pgmap v151: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:45:39.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:45:39 smithi053 ceph-mon[27773]: pgmap v152: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:39.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:45:39 smithi183 ceph-mon[32584]: pgmap v152: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:39.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:45:39 smithi150 ceph-mon[32504]: pgmap v152: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:41.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:45:41 smithi053 ceph-mon[27773]: pgmap v153: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:41.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:45:41 smithi183 ceph-mon[32584]: pgmap v153: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:41.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:45:41 smithi150 ceph-mon[32504]: pgmap v153: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:43.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:45:43 smithi053 ceph-mon[27773]: pgmap v154: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:45:43.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:45:43 smithi183 ceph-mon[32584]: pgmap v154: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:45:43.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:45:43 smithi150 ceph-mon[32504]: pgmap v154: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:45:45.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:45:45 smithi053 ceph-mon[27773]: pgmap v155: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:45.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:45:45 smithi183 ceph-mon[32584]: pgmap v155: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:45.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:45:45 smithi150 ceph-mon[32504]: pgmap v155: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:47.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:45:47 smithi053 ceph-mon[27773]: pgmap v156: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:45:47.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:45:47 smithi183 ceph-mon[32584]: pgmap v156: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:45:47.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:45:47 smithi150 ceph-mon[32504]: pgmap v156: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:45:49.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:45:49 smithi053 ceph-mon[27773]: pgmap v157: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:49.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:45:49 smithi183 ceph-mon[32584]: pgmap v157: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:49.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:45:49 smithi150 ceph-mon[32504]: pgmap v157: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:51.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:45:51 smithi053 ceph-mon[27773]: pgmap v158: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:51.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:45:51 smithi183 ceph-mon[32584]: pgmap v158: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:51.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:45:51 smithi150 ceph-mon[32504]: pgmap v158: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:53.763 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:45:53 smithi150 ceph-mon[32504]: pgmap v159: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:45:53.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:45:53 smithi053 ceph-mon[27773]: pgmap v159: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:45:53.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:45:53 smithi183 ceph-mon[32584]: pgmap v159: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:45:55.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:45:55 smithi053 ceph-mon[27773]: pgmap v160: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:55.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:45:55 smithi183 ceph-mon[32584]: pgmap v160: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:55.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:45:55 smithi150 ceph-mon[32504]: pgmap v160: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:57.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:45:57 smithi053 ceph-mon[27773]: pgmap v161: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:45:57.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:45:57 smithi183 ceph-mon[32584]: pgmap v161: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:45:57.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:45:57 smithi150 ceph-mon[32504]: pgmap v161: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:45:58.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:45:58 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:45:58.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:45:58 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:45:58.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:45:58 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:45:59.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:45:59 smithi053 ceph-mon[27773]: pgmap v162: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:59.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:45:59 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:45:59.784 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:45:59 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:45:59.784 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:45:59 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:45:59.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:45:59 smithi183 ceph-mon[32584]: pgmap v162: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:59.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:45:59 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:45:59.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:45:59 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:45:59.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:45:59 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:45:59.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:45:59 smithi150 ceph-mon[32504]: pgmap v162: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:45:59.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:45:59 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:45:59.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:45:59 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:45:59.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:45:59 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:46:01.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:46:01 smithi053 ceph-mon[27773]: pgmap v163: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:46:01.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:46:01 smithi183 ceph-mon[32584]: pgmap v163: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:46:01.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:46:01 smithi150 ceph-mon[32504]: pgmap v163: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:46:03.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:46:03 smithi183 ceph-mon[32584]: pgmap v164: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T14:46:03.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:46:03 smithi150 ceph-mon[32504]: pgmap v164: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T14:46:04.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:46:03 smithi053 ceph-mon[27773]: pgmap v164: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T14:46:05.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:46:05 smithi183 ceph-mon[32584]: pgmap v165: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-25T14:46:05.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:46:05 smithi150 ceph-mon[32504]: pgmap v165: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-25T14:46:06.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:46:05 smithi053 ceph-mon[27773]: pgmap v165: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-25T14:46:07.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:46:07 smithi183 ceph-mon[32584]: pgmap v166: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T14:46:07.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:46:07 smithi150 ceph-mon[32504]: pgmap v166: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T14:46:08.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:46:07 smithi053 ceph-mon[27773]: pgmap v166: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T14:46:09.836 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:46:09 smithi053 ceph-mon[27773]: pgmap v167: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T14:46:09.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:46:09 smithi183 ceph-mon[32584]: pgmap v167: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T14:46:09.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:46:09 smithi150 ceph-mon[32504]: pgmap v167: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T14:46:11.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:46:11 smithi183 ceph-mon[32584]: pgmap v168: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T14:46:11.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:46:11 smithi150 ceph-mon[32504]: pgmap v168: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T14:46:12.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:46:11 smithi053 ceph-mon[27773]: pgmap v168: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T14:46:13.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:46:13 smithi183 ceph-mon[32584]: pgmap v169: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T14:46:13.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:46:13 smithi150 ceph-mon[32504]: pgmap v169: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T14:46:14.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:46:13 smithi053 ceph-mon[27773]: pgmap v169: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T14:46:15.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:46:15 smithi183 ceph-mon[32584]: pgmap v170: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T14:46:15.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:46:15 smithi150 ceph-mon[32504]: pgmap v170: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T14:46:16.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:46:15 smithi053 ceph-mon[27773]: pgmap v170: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T14:46:17.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:46:17 smithi183 ceph-mon[32584]: pgmap v171: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T14:46:17.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:46:17 smithi150 ceph-mon[32504]: pgmap v171: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T14:46:18.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:46:17 smithi053 ceph-mon[27773]: pgmap v171: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T14:46:19.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:46:19 smithi183 ceph-mon[32584]: pgmap v172: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T14:46:19.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:46:19 smithi150 ceph-mon[32504]: pgmap v172: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T14:46:20.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:46:19 smithi053 ceph-mon[27773]: pgmap v172: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T14:46:21.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:46:21 smithi183 ceph-mon[32584]: pgmap v173: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T14:46:21.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:46:21 smithi150 ceph-mon[32504]: pgmap v173: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T14:46:22.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:46:21 smithi053 ceph-mon[27773]: pgmap v173: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T14:46:23.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:46:23 smithi183 ceph-mon[32584]: pgmap v174: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T14:46:23.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:46:23 smithi150 ceph-mon[32504]: pgmap v174: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T14:46:24.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:46:23 smithi053 ceph-mon[27773]: pgmap v174: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T14:46:25.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:46:25 smithi183 ceph-mon[32584]: pgmap v175: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:46:25.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:46:25 smithi150 ceph-mon[32504]: pgmap v175: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:46:26.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:46:25 smithi053 ceph-mon[27773]: pgmap v175: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:46:27.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:46:27 smithi183 ceph-mon[32584]: pgmap v176: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:46:27.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:46:27 smithi150 ceph-mon[32504]: pgmap v176: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:46:28.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:46:27 smithi053 ceph-mon[27773]: pgmap v176: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:46:29.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:46:29 smithi183 ceph-mon[32584]: pgmap v177: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:46:29.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:46:29 smithi150 ceph-mon[32504]: pgmap v177: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:46:30.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:46:29 smithi053 ceph-mon[27773]: pgmap v177: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:46:31.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:46:31 smithi183 ceph-mon[32584]: pgmap v178: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:46:31.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:46:31 smithi150 ceph-mon[32504]: pgmap v178: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:46:32.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:46:31 smithi053 ceph-mon[27773]: pgmap v178: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:46:33.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:46:33 smithi183 ceph-mon[32584]: pgmap v179: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:46:33.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:46:33 smithi150 ceph-mon[32504]: pgmap v179: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:46:34.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:46:33 smithi053 ceph-mon[27773]: pgmap v179: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:46:35.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:46:35 smithi183 ceph-mon[32584]: pgmap v180: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:46:35.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:46:35 smithi150 ceph-mon[32504]: pgmap v180: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:46:36.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:46:35 smithi053 ceph-mon[27773]: pgmap v180: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:46:37.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:46:37 smithi183 ceph-mon[32584]: pgmap v181: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:46:37.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:46:37 smithi150 ceph-mon[32504]: pgmap v181: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:46:38.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:46:37 smithi053 ceph-mon[27773]: pgmap v181: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:46:39.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:46:39 smithi183 ceph-mon[32584]: pgmap v182: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:46:39.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:46:39 smithi150 ceph-mon[32504]: pgmap v182: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:46:40.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:46:39 smithi053 ceph-mon[27773]: pgmap v182: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:46:41.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:46:41 smithi183 ceph-mon[32584]: pgmap v183: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:46:41.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:46:41 smithi150 ceph-mon[32504]: pgmap v183: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:46:42.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:46:41 smithi053 ceph-mon[27773]: pgmap v183: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:46:43.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:46:43 smithi183 ceph-mon[32584]: pgmap v184: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:46:43.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:46:43 smithi150 ceph-mon[32504]: pgmap v184: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:46:44.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:46:43 smithi053 ceph-mon[27773]: pgmap v184: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:46:45.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:46:45 smithi183 ceph-mon[32584]: pgmap v185: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:46:45.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:46:45 smithi150 ceph-mon[32504]: pgmap v185: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:46:46.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:46:45 smithi053 ceph-mon[27773]: pgmap v185: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:46:47.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:46:47 smithi183 ceph-mon[32584]: pgmap v186: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:46:47.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:46:47 smithi150 ceph-mon[32504]: pgmap v186: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:46:48.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:46:47 smithi053 ceph-mon[27773]: pgmap v186: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:46:49.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:46:49 smithi183 ceph-mon[32584]: pgmap v187: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:46:49.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:46:49 smithi150 ceph-mon[32504]: pgmap v187: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:46:50.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:46:49 smithi053 ceph-mon[27773]: pgmap v187: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:46:51.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:46:51 smithi183 ceph-mon[32584]: pgmap v188: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:46:51.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:46:51 smithi150 ceph-mon[32504]: pgmap v188: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:46:52.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:46:51 smithi053 ceph-mon[27773]: pgmap v188: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:46:53.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:46:53 smithi183 ceph-mon[32584]: pgmap v189: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:46:53.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:46:53 smithi150 ceph-mon[32504]: pgmap v189: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:46:54.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:46:53 smithi053 ceph-mon[27773]: pgmap v189: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:46:55.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:46:55 smithi183 ceph-mon[32584]: pgmap v190: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:46:55.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:46:55 smithi150 ceph-mon[32504]: pgmap v190: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:46:56.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:46:55 smithi053 ceph-mon[27773]: pgmap v190: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:46:57.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:46:57 smithi183 ceph-mon[32584]: pgmap v191: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:46:57.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:46:57 smithi150 ceph-mon[32504]: pgmap v191: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:46:58.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:46:57 smithi053 ceph-mon[27773]: pgmap v191: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:46:59.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:46:59 smithi183 ceph-mon[32584]: pgmap v192: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:46:59.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:46:59 smithi150 ceph-mon[32504]: pgmap v192: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:47:00.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:46:59 smithi053 ceph-mon[27773]: pgmap v192: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:47:00.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:47:00 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:47:00.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:47:00 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:47:01.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:47:00 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:47:01.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:47:01 smithi183 ceph-mon[32584]: pgmap v193: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:47:01.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:47:01 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:47:01.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:47:01 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:47:01.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:47:01 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:47:01.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:47:01 smithi150 ceph-mon[32504]: pgmap v193: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:47:01.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:47:01 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:47:01.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:47:01 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:47:01.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:47:01 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:47:02.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:47:01 smithi053 ceph-mon[27773]: pgmap v193: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:47:02.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:47:01 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:47:02.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:47:01 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:47:02.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:47:01 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:47:03.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:47:03 smithi183 ceph-mon[32584]: pgmap v194: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T14:47:03.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:47:03 smithi150 ceph-mon[32504]: pgmap v194: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T14:47:04.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:47:03 smithi053 ceph-mon[27773]: pgmap v194: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T14:47:05.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:47:05 smithi183 ceph-mon[32584]: pgmap v195: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:47:05.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:47:05 smithi150 ceph-mon[32504]: pgmap v195: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:47:06.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:47:05 smithi053 ceph-mon[27773]: pgmap v195: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:47:07.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:47:07 smithi183 ceph-mon[32584]: pgmap v196: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T14:47:07.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:47:07 smithi150 ceph-mon[32504]: pgmap v196: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T14:47:08.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:47:07 smithi053 ceph-mon[27773]: pgmap v196: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T14:47:09.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:47:09 smithi150 ceph-mon[32504]: pgmap v197: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:47:10.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:47:09 smithi053 ceph-mon[27773]: pgmap v197: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:47:10.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:47:09 smithi183 ceph-mon[32584]: pgmap v197: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:47:12.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:47:11 smithi053 ceph-mon[27773]: pgmap v198: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:47:12.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:47:11 smithi053 ceph-mon[27773]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T14:47:12.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:47:11 smithi183 ceph-mon[32584]: pgmap v198: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:47:12.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:47:11 smithi183 ceph-mon[32584]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T14:47:12.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:47:11 smithi150 ceph-mon[32504]: pgmap v198: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:47:12.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:47:11 smithi150 ceph-mon[32504]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T14:47:14.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:47:13 smithi053 ceph-mon[27773]: pgmap v200: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T14:47:14.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:47:13 smithi183 ceph-mon[32584]: pgmap v200: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T14:47:14.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:47:13 smithi150 ceph-mon[32504]: pgmap v200: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T14:47:16.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:47:15 smithi053 ceph-mon[27773]: pgmap v201: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T14:47:16.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:47:15 smithi183 ceph-mon[32584]: pgmap v201: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T14:47:16.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:47:15 smithi150 ceph-mon[32504]: pgmap v201: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T14:47:17.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:47:16 smithi183 ceph-mon[32584]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T14:47:17.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:47:16 smithi183 ceph-mon[32584]: pgmap v203: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T14:47:17.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:47:16 smithi150 ceph-mon[32504]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T14:47:17.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:47:16 smithi150 ceph-mon[32504]: pgmap v203: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T14:47:17.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:47:16 smithi053 ceph-mon[27773]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T14:47:17.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:47:16 smithi053 ceph-mon[27773]: pgmap v203: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T14:47:19.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:47:19 smithi183 ceph-mon[32584]: pgmap v204: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T14:47:19.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:47:19 smithi150 ceph-mon[32504]: pgmap v204: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T14:47:19.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:47:19 smithi053 ceph-mon[27773]: pgmap v204: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T14:47:21.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:47:21 smithi183 ceph-mon[32584]: pgmap v205: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 212 B/s wr, 6 op/s 2024-04-25T14:47:21.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:47:21 smithi150 ceph-mon[32504]: pgmap v205: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 212 B/s wr, 6 op/s 2024-04-25T14:47:21.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:47:21 smithi053 ceph-mon[27773]: pgmap v205: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 212 B/s wr, 6 op/s 2024-04-25T14:47:23.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:47:23 smithi183 ceph-mon[32584]: pgmap v206: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T14:47:23.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:47:23 smithi150 ceph-mon[32504]: pgmap v206: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T14:47:23.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:47:23 smithi053 ceph-mon[27773]: pgmap v206: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T14:47:25.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:47:25 smithi183 ceph-mon[32584]: pgmap v207: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T14:47:25.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:47:25 smithi150 ceph-mon[32504]: pgmap v207: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T14:47:25.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:47:25 smithi053 ceph-mon[27773]: pgmap v207: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T14:47:27.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:47:27 smithi183 ceph-mon[32584]: pgmap v208: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 97 B/s wr, 8 op/s 2024-04-25T14:47:27.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:47:27 smithi150 ceph-mon[32504]: pgmap v208: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 97 B/s wr, 8 op/s 2024-04-25T14:47:27.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:47:27 smithi053 ceph-mon[27773]: pgmap v208: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 97 B/s wr, 8 op/s 2024-04-25T14:47:29.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:47:29 smithi150 ceph-mon[32504]: pgmap v209: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:47:29.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:47:29 smithi053 ceph-mon[27773]: pgmap v209: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:47:29.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:47:29 smithi183 ceph-mon[32584]: pgmap v209: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:47:31.756 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:47:31 smithi183 ceph-mon[32584]: pgmap v210: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:47:31.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:47:31 smithi053 ceph-mon[27773]: pgmap v210: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:47:31.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:47:31 smithi150 ceph-mon[32504]: pgmap v210: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:47:33.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:47:33 smithi053 ceph-mon[27773]: pgmap v211: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:47:33.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:47:33 smithi183 ceph-mon[32584]: pgmap v211: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:47:33.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:47:33 smithi150 ceph-mon[32504]: pgmap v211: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:47:35.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:47:35 smithi053 ceph-mon[27773]: pgmap v212: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:47:35.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:47:35 smithi183 ceph-mon[32584]: pgmap v212: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:47:35.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:47:35 smithi150 ceph-mon[32504]: pgmap v212: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:47:37.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:47:36 smithi183 ceph-mon[32584]: pgmap v213: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:47:37.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:47:36 smithi150 ceph-mon[32504]: pgmap v213: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:47:37.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:47:36 smithi053 ceph-mon[27773]: pgmap v213: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:47:39.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:47:39 smithi183 ceph-mon[32584]: pgmap v214: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:47:39.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:47:39 smithi150 ceph-mon[32504]: pgmap v214: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:47:39.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:47:39 smithi053 ceph-mon[27773]: pgmap v214: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:47:41.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:47:41 smithi183 ceph-mon[32584]: pgmap v215: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:47:41.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:47:41 smithi150 ceph-mon[32504]: pgmap v215: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:47:41.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:47:41 smithi053 ceph-mon[27773]: pgmap v215: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:47:43.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:47:43 smithi183 ceph-mon[32584]: pgmap v216: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:47:43.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:47:43 smithi150 ceph-mon[32504]: pgmap v216: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:47:43.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:47:43 smithi053 ceph-mon[27773]: pgmap v216: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:47:45.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:47:45 smithi183 ceph-mon[32584]: pgmap v217: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:47:45.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:47:45 smithi150 ceph-mon[32504]: pgmap v217: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:47:45.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:47:45 smithi053 ceph-mon[27773]: pgmap v217: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:47:47.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:47:47 smithi150 ceph-mon[32504]: pgmap v218: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:47:47.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:47:47 smithi053 ceph-mon[27773]: pgmap v218: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:47:47.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:47:47 smithi183 ceph-mon[32584]: pgmap v218: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:47:49.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:47:49 smithi053 ceph-mon[27773]: pgmap v219: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:47:49.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:47:49 smithi183 ceph-mon[32584]: pgmap v219: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:47:49.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:47:49 smithi150 ceph-mon[32504]: pgmap v219: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:47:51.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:47:51 smithi053 ceph-mon[27773]: pgmap v220: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:47:51.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:47:51 smithi183 ceph-mon[32584]: pgmap v220: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:47:51.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:47:51 smithi150 ceph-mon[32504]: pgmap v220: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:47:53.763 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:47:53 smithi150 ceph-mon[32504]: pgmap v221: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:47:53.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:47:53 smithi053 ceph-mon[27773]: pgmap v221: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:47:53.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:47:53 smithi183 ceph-mon[32584]: pgmap v221: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:47:55.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:47:55 smithi053 ceph-mon[27773]: pgmap v222: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:47:55.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:47:55 smithi183 ceph-mon[32584]: pgmap v222: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:47:55.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:47:55 smithi150 ceph-mon[32504]: pgmap v222: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:47:57.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:47:57 smithi053 ceph-mon[27773]: pgmap v223: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:47:57.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:47:57 smithi183 ceph-mon[32584]: pgmap v223: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:47:57.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:47:57 smithi150 ceph-mon[32504]: pgmap v223: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:47:59.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:47:59 smithi053 ceph-mon[27773]: pgmap v224: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:47:59.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:47:59 smithi183 ceph-mon[32584]: pgmap v224: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:47:59.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:47:59 smithi150 ceph-mon[32504]: pgmap v224: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:48:01.756 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:48:01 smithi183 ceph-mon[32584]: pgmap v225: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:48:01.756 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:48:01 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:48:01.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:48:01 smithi053 ceph-mon[27773]: pgmap v225: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:48:01.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:48:01 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:48:01.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:48:01 smithi150 ceph-mon[32504]: pgmap v225: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:48:01.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:48:01 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:48:03.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:48:03 smithi053 ceph-mon[27773]: pgmap v226: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T14:48:03.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:48:03 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:48:03.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:48:03 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:48:03.784 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:48:03 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:48:03.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:48:03 smithi183 ceph-mon[32584]: pgmap v226: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T14:48:03.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:48:03 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:48:03.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:48:03 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:48:03.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:48:03 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:48:03.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:48:03 smithi150 ceph-mon[32504]: pgmap v226: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T14:48:03.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:48:03 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:48:03.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:48:03 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:48:03.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:48:03 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:48:05.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:48:05 smithi053 ceph-mon[27773]: pgmap v227: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:48:05.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:48:05 smithi183 ceph-mon[32584]: pgmap v227: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:48:05.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:48:05 smithi150 ceph-mon[32504]: pgmap v227: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:48:07.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:48:07 smithi053 ceph-mon[27773]: pgmap v228: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T14:48:07.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:48:07 smithi183 ceph-mon[32584]: pgmap v228: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T14:48:07.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:48:07 smithi150 ceph-mon[32504]: pgmap v228: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T14:48:09.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:48:09 smithi053 ceph-mon[27773]: pgmap v229: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:48:09.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:48:09 smithi183 ceph-mon[32584]: pgmap v229: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:48:09.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:48:09 smithi150 ceph-mon[32504]: pgmap v229: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:48:11.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:48:11 smithi053 ceph-mon[27773]: pgmap v230: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:48:11.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:48:11 smithi053 ceph-mon[27773]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T14:48:11.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:48:11 smithi183 ceph-mon[32584]: pgmap v230: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:48:11.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:48:11 smithi183 ceph-mon[32584]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T14:48:11.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:48:11 smithi150 ceph-mon[32504]: pgmap v230: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:48:11.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:48:11 smithi150 ceph-mon[32504]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T14:48:13.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:48:13 smithi183 ceph-mon[32584]: pgmap v232: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T14:48:13.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:48:13 smithi150 ceph-mon[32504]: pgmap v232: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T14:48:14.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:48:13 smithi053 ceph-mon[27773]: pgmap v232: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T14:48:15.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:48:15 smithi183 ceph-mon[32584]: pgmap v233: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T14:48:15.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:48:15 smithi150 ceph-mon[32504]: pgmap v233: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T14:48:16.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:48:15 smithi053 ceph-mon[27773]: pgmap v233: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T14:48:17.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:48:16 smithi183 ceph-mon[32584]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T14:48:17.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:48:16 smithi183 ceph-mon[32584]: pgmap v235: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T14:48:17.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:48:16 smithi150 ceph-mon[32504]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T14:48:17.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:48:16 smithi150 ceph-mon[32504]: pgmap v235: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T14:48:17.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:48:16 smithi053 ceph-mon[27773]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T14:48:17.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:48:16 smithi053 ceph-mon[27773]: pgmap v235: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T14:48:19.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:48:19 smithi150 ceph-mon[32504]: pgmap v236: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T14:48:19.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:48:19 smithi053 ceph-mon[27773]: pgmap v236: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T14:48:19.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:48:19 smithi183 ceph-mon[32584]: pgmap v236: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T14:48:21.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:48:21 smithi053 ceph-mon[27773]: pgmap v237: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-04-25T14:48:21.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:48:21 smithi183 ceph-mon[32584]: pgmap v237: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-04-25T14:48:21.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:48:21 smithi150 ceph-mon[32504]: pgmap v237: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-04-25T14:48:23.763 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:48:23 smithi150 ceph-mon[32504]: pgmap v238: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T14:48:23.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:48:23 smithi053 ceph-mon[27773]: pgmap v238: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T14:48:23.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:48:23 smithi183 ceph-mon[32584]: pgmap v238: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T14:48:25.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:48:25 smithi053 ceph-mon[27773]: pgmap v239: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T14:48:25.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:48:25 smithi183 ceph-mon[32584]: pgmap v239: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T14:48:25.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:48:25 smithi150 ceph-mon[32504]: pgmap v239: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T14:48:27.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:48:27 smithi053 ceph-mon[27773]: pgmap v240: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 97 B/s wr, 8 op/s 2024-04-25T14:48:27.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:48:27 smithi183 ceph-mon[32584]: pgmap v240: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 97 B/s wr, 8 op/s 2024-04-25T14:48:27.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:48:27 smithi150 ceph-mon[32504]: pgmap v240: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 97 B/s wr, 8 op/s 2024-04-25T14:48:29.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:48:29 smithi053 ceph-mon[27773]: pgmap v241: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:48:29.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:48:29 smithi183 ceph-mon[32584]: pgmap v241: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:48:29.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:48:29 smithi150 ceph-mon[32504]: pgmap v241: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:48:31.756 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:48:31 smithi183 ceph-mon[32584]: pgmap v242: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:48:31.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:48:31 smithi053 ceph-mon[27773]: pgmap v242: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:48:31.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:48:31 smithi150 ceph-mon[32504]: pgmap v242: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:48:33.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:48:33 smithi053 ceph-mon[27773]: pgmap v243: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:48:33.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:48:33 smithi183 ceph-mon[32584]: pgmap v243: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:48:33.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:48:33 smithi150 ceph-mon[32504]: pgmap v243: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:48:35.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:48:35 smithi053 ceph-mon[27773]: pgmap v244: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:48:35.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:48:35 smithi183 ceph-mon[32584]: pgmap v244: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:48:35.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:48:35 smithi150 ceph-mon[32504]: pgmap v244: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:48:37.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:48:37 smithi053 ceph-mon[27773]: pgmap v245: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:48:37.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:48:37 smithi183 ceph-mon[32584]: pgmap v245: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:48:37.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:48:37 smithi150 ceph-mon[32504]: pgmap v245: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:48:39.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:48:39 smithi053 ceph-mon[27773]: pgmap v246: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:48:39.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:48:39 smithi183 ceph-mon[32584]: pgmap v246: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:48:39.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:48:39 smithi150 ceph-mon[32504]: pgmap v246: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:48:41.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:48:41 smithi053 ceph-mon[27773]: pgmap v247: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:48:41.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:48:41 smithi183 ceph-mon[32584]: pgmap v247: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:48:41.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:48:41 smithi150 ceph-mon[32504]: pgmap v247: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:48:43.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:48:43 smithi053 ceph-mon[27773]: pgmap v248: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:48:43.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:48:43 smithi183 ceph-mon[32584]: pgmap v248: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:48:43.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:48:43 smithi150 ceph-mon[32504]: pgmap v248: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:48:45.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:48:45 smithi053 ceph-mon[27773]: pgmap v249: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:48:45.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:48:45 smithi183 ceph-mon[32584]: pgmap v249: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:48:45.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:48:45 smithi150 ceph-mon[32504]: pgmap v249: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:48:47.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:48:47 smithi183 ceph-mon[32584]: pgmap v250: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:48:47.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:48:47 smithi150 ceph-mon[32504]: pgmap v250: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:48:48.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:48:47 smithi053 ceph-mon[27773]: pgmap v250: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:48:49.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:48:49 smithi183 ceph-mon[32584]: pgmap v251: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:48:49.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:48:49 smithi150 ceph-mon[32504]: pgmap v251: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:48:50.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:48:49 smithi053 ceph-mon[27773]: pgmap v251: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:48:51.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:48:51 smithi183 ceph-mon[32584]: pgmap v252: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:48:51.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:48:51 smithi150 ceph-mon[32504]: pgmap v252: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:48:52.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:48:51 smithi053 ceph-mon[27773]: pgmap v252: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:48:53.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:48:53 smithi183 ceph-mon[32584]: pgmap v253: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:48:53.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:48:53 smithi150 ceph-mon[32504]: pgmap v253: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:48:54.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:48:53 smithi053 ceph-mon[27773]: pgmap v253: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:48:55.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:48:55 smithi183 ceph-mon[32584]: pgmap v254: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:48:55.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:48:55 smithi150 ceph-mon[32504]: pgmap v254: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:48:56.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:48:55 smithi053 ceph-mon[27773]: pgmap v254: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:48:57.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:48:56 smithi183 ceph-mon[32584]: pgmap v255: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:48:57.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:48:56 smithi150 ceph-mon[32504]: pgmap v255: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:48:57.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:48:56 smithi053 ceph-mon[27773]: pgmap v255: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:48:59.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:48:59 smithi053 ceph-mon[27773]: pgmap v256: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:48:59.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:48:59 smithi183 ceph-mon[32584]: pgmap v256: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:48:59.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:48:59 smithi150 ceph-mon[32504]: pgmap v256: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:49:01.756 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:49:01 smithi183 ceph-mon[32584]: pgmap v257: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:49:01.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:49:01 smithi053 ceph-mon[27773]: pgmap v257: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:49:01.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:49:01 smithi150 ceph-mon[32504]: pgmap v257: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:49:03.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:49:03 smithi053 ceph-mon[27773]: pgmap v258: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T14:49:03.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:49:03 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:49:03.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:49:03 smithi183 ceph-mon[32584]: pgmap v258: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T14:49:03.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:49:03 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:49:03.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:49:03 smithi150 ceph-mon[32504]: pgmap v258: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T14:49:03.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:49:03 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:49:04.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:49:04 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:49:04.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:49:04 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:49:04.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:49:04 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:49:04.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:49:04 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:49:04.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:49:04 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:49:04.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:49:04 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:49:04.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:49:04 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:49:04.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:49:04 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:49:04.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:49:04 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:49:05.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:49:05 smithi053 ceph-mon[27773]: pgmap v259: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:49:05.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:49:05 smithi183 ceph-mon[32584]: pgmap v259: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:49:05.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:49:05 smithi150 ceph-mon[32504]: pgmap v259: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:49:07.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:49:07 smithi053 ceph-mon[27773]: pgmap v260: 97 pgs: 97 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T14:49:07.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:49:07 smithi183 ceph-mon[32584]: pgmap v260: 97 pgs: 97 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T14:49:07.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:49:07 smithi150 ceph-mon[32504]: pgmap v260: 97 pgs: 97 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T14:49:09.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:49:09 smithi053 ceph-mon[27773]: pgmap v261: 97 pgs: 97 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:49:09.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:49:09 smithi183 ceph-mon[32584]: pgmap v261: 97 pgs: 97 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:49:09.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:49:09 smithi150 ceph-mon[32504]: pgmap v261: 97 pgs: 97 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:49:11.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:49:11 smithi053 ceph-mon[27773]: pgmap v262: 97 pgs: 97 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:49:11.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:49:11 smithi053 ceph-mon[27773]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T14:49:11.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:49:11 smithi183 ceph-mon[32584]: pgmap v262: 97 pgs: 97 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:49:11.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:49:11 smithi183 ceph-mon[32584]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T14:49:11.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:49:11 smithi150 ceph-mon[32504]: pgmap v262: 97 pgs: 97 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:49:11.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:49:11 smithi150 ceph-mon[32504]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T14:49:13.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:49:13 smithi053 ceph-mon[27773]: pgmap v264: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T14:49:13.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:49:13 smithi183 ceph-mon[32584]: pgmap v264: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T14:49:13.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:49:13 smithi150 ceph-mon[32504]: pgmap v264: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T14:49:15.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:49:15 smithi053 ceph-mon[27773]: pgmap v265: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T14:49:15.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:49:15 smithi183 ceph-mon[32584]: pgmap v265: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T14:49:15.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:49:15 smithi150 ceph-mon[32504]: pgmap v265: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T14:49:17.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:49:16 smithi183 ceph-mon[32584]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T14:49:17.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:49:16 smithi183 ceph-mon[32584]: pgmap v267: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T14:49:17.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:49:16 smithi150 ceph-mon[32504]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T14:49:17.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:49:16 smithi150 ceph-mon[32504]: pgmap v267: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T14:49:17.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:49:16 smithi053 ceph-mon[27773]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T14:49:17.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:49:16 smithi053 ceph-mon[27773]: pgmap v267: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T14:49:19.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:49:19 smithi053 ceph-mon[27773]: pgmap v268: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T14:49:19.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:49:19 smithi183 ceph-mon[32584]: pgmap v268: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T14:49:19.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:49:19 smithi150 ceph-mon[32504]: pgmap v268: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T14:49:21.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:49:21 smithi053 ceph-mon[27773]: pgmap v269: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-04-25T14:49:21.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:49:21 smithi183 ceph-mon[32584]: pgmap v269: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-04-25T14:49:21.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:49:21 smithi150 ceph-mon[32504]: pgmap v269: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-04-25T14:49:23.763 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:49:23 smithi150 ceph-mon[32504]: pgmap v270: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T14:49:23.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:49:23 smithi053 ceph-mon[27773]: pgmap v270: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T14:49:23.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:49:23 smithi183 ceph-mon[32584]: pgmap v270: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T14:49:25.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:49:25 smithi053 ceph-mon[27773]: pgmap v271: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T14:49:25.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:49:25 smithi183 ceph-mon[32584]: pgmap v271: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T14:49:25.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:49:25 smithi150 ceph-mon[32504]: pgmap v271: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T14:49:27.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:49:27 smithi053 ceph-mon[27773]: pgmap v272: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 97 B/s wr, 8 op/s 2024-04-25T14:49:27.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:49:27 smithi183 ceph-mon[32584]: pgmap v272: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 97 B/s wr, 8 op/s 2024-04-25T14:49:27.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:49:27 smithi150 ceph-mon[32504]: pgmap v272: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 97 B/s wr, 8 op/s 2024-04-25T14:49:29.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:49:29 smithi053 ceph-mon[27773]: pgmap v273: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T14:49:29.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:49:29 smithi183 ceph-mon[32584]: pgmap v273: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T14:49:29.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:49:29 smithi150 ceph-mon[32504]: pgmap v273: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T14:49:31.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:49:31 smithi053 ceph-mon[27773]: pgmap v274: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T14:49:31.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:49:31 smithi183 ceph-mon[32584]: pgmap v274: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T14:49:31.990 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:49:31 smithi150 ceph-mon[32504]: pgmap v274: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T14:49:33.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:49:33 smithi053 ceph-mon[27773]: pgmap v275: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:49:33.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:49:33 smithi183 ceph-mon[32584]: pgmap v275: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:49:33.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:49:33 smithi150 ceph-mon[32504]: pgmap v275: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:49:35.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:49:35 smithi053 ceph-mon[27773]: pgmap v276: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:49:35.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:49:35 smithi183 ceph-mon[32584]: pgmap v276: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:49:35.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:49:35 smithi150 ceph-mon[32504]: pgmap v276: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:49:37.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:49:37 smithi053 ceph-mon[27773]: pgmap v277: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:49:37.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:49:37 smithi183 ceph-mon[32584]: pgmap v277: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:49:37.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:49:37 smithi150 ceph-mon[32504]: pgmap v277: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:49:39.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:49:39 smithi183 ceph-mon[32584]: pgmap v278: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:49:39.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:49:39 smithi150 ceph-mon[32504]: pgmap v278: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:49:40.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:49:39 smithi053 ceph-mon[27773]: pgmap v278: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:49:41.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:49:41 smithi183 ceph-mon[32584]: pgmap v279: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:49:41.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:49:41 smithi150 ceph-mon[32504]: pgmap v279: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:49:42.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:49:41 smithi053 ceph-mon[27773]: pgmap v279: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:49:43.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:49:43 smithi183 ceph-mon[32584]: pgmap v280: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:49:43.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:49:43 smithi150 ceph-mon[32504]: pgmap v280: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:49:44.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:49:43 smithi053 ceph-mon[27773]: pgmap v280: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:49:45.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:49:45 smithi183 ceph-mon[32584]: pgmap v281: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:49:45.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:49:45 smithi150 ceph-mon[32504]: pgmap v281: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:49:46.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:49:45 smithi053 ceph-mon[27773]: pgmap v281: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:49:47.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:49:46 smithi183 ceph-mon[32584]: pgmap v282: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:49:47.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:49:46 smithi150 ceph-mon[32504]: pgmap v282: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:49:47.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:49:46 smithi053 ceph-mon[27773]: pgmap v282: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:49:49.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:49:49 smithi053 ceph-mon[27773]: pgmap v283: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:49:49.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:49:49 smithi183 ceph-mon[32584]: pgmap v283: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:49:49.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:49:49 smithi150 ceph-mon[32504]: pgmap v283: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:49:51.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:49:51 smithi053 ceph-mon[27773]: pgmap v284: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:49:51.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:49:51 smithi183 ceph-mon[32584]: pgmap v284: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:49:51.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:49:51 smithi150 ceph-mon[32504]: pgmap v284: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:49:53.763 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:49:53 smithi150 ceph-mon[32504]: pgmap v285: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:49:53.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:49:53 smithi053 ceph-mon[27773]: pgmap v285: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:49:53.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:49:53 smithi183 ceph-mon[32584]: pgmap v285: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:49:55.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:49:55 smithi053 ceph-mon[27773]: pgmap v286: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:49:55.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:49:55 smithi183 ceph-mon[32584]: pgmap v286: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:49:55.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:49:55 smithi150 ceph-mon[32504]: pgmap v286: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:49:57.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:49:57 smithi053 ceph-mon[27773]: pgmap v287: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:49:57.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:49:57 smithi183 ceph-mon[32584]: pgmap v287: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:49:57.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:49:57 smithi150 ceph-mon[32504]: pgmap v287: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:49:59.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:49:59 smithi053 ceph-mon[27773]: pgmap v288: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:49:59.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:49:59 smithi183 ceph-mon[32584]: pgmap v288: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:49:59.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:49:59 smithi150 ceph-mon[32504]: pgmap v288: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:50:00.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:00 smithi053 ceph-mon[27773]: overall HEALTH_OK 2024-04-25T14:50:00.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:00 smithi183 ceph-mon[32584]: overall HEALTH_OK 2024-04-25T14:50:00.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:00 smithi150 ceph-mon[32504]: overall HEALTH_OK 2024-04-25T14:50:01.855 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:01 smithi183 ceph-mon[32584]: pgmap v289: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:50:01.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:01 smithi150 ceph-mon[32504]: pgmap v289: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:50:02.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:01 smithi053 ceph-mon[27773]: pgmap v289: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:50:03.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:03 smithi183 ceph-mon[32584]: pgmap v290: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T14:50:03.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:03 smithi150 ceph-mon[32504]: pgmap v290: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T14:50:04.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:03 smithi053 ceph-mon[27773]: pgmap v290: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T14:50:05.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:05 smithi183 ceph-mon[32584]: pgmap v291: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:50:05.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:05 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:50:05.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:05 smithi150 ceph-mon[32504]: pgmap v291: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:50:05.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:05 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:50:06.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:05 smithi053 ceph-mon[27773]: pgmap v291: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:50:06.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:05 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:50:06.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:06 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:50:06.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:06 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:50:06.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:06 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:50:06.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:06 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:50:06.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:06 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:50:06.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:06 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:50:07.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:06 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:50:07.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:06 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:50:07.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:06 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:50:07.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:07 smithi183 ceph-mon[32584]: pgmap v292: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T14:50:07.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:07 smithi150 ceph-mon[32504]: pgmap v292: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T14:50:08.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:07 smithi053 ceph-mon[27773]: pgmap v292: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T14:50:09.836 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:09 smithi053 ceph-mon[27773]: pgmap v293: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:50:09.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:09 smithi183 ceph-mon[32584]: pgmap v293: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:50:09.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:09 smithi150 ceph-mon[32504]: pgmap v293: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:50:11.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:11 smithi183 ceph-mon[32584]: pgmap v294: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:50:11.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:11 smithi183 ceph-mon[32584]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T14:50:11.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:11 smithi150 ceph-mon[32504]: pgmap v294: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:50:11.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:11 smithi150 ceph-mon[32504]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T14:50:12.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:11 smithi053 ceph-mon[27773]: pgmap v294: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:50:12.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:11 smithi053 ceph-mon[27773]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T14:50:13.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:13 smithi183 ceph-mon[32584]: pgmap v296: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T14:50:13.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:13 smithi150 ceph-mon[32504]: pgmap v296: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T14:50:14.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:13 smithi053 ceph-mon[27773]: pgmap v296: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T14:50:15.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:15 smithi183 ceph-mon[32584]: pgmap v297: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T14:50:15.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:15 smithi150 ceph-mon[32504]: pgmap v297: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T14:50:16.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:15 smithi053 ceph-mon[27773]: pgmap v297: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T14:50:17.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:16 smithi183 ceph-mon[32584]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T14:50:17.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:16 smithi183 ceph-mon[32584]: pgmap v299: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T14:50:17.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:16 smithi150 ceph-mon[32504]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T14:50:17.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:16 smithi150 ceph-mon[32504]: pgmap v299: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T14:50:17.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:16 smithi053 ceph-mon[27773]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T14:50:17.283 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:16 smithi053 ceph-mon[27773]: pgmap v299: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T14:50:19.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:19 smithi053 ceph-mon[27773]: pgmap v300: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T14:50:19.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:19 smithi183 ceph-mon[32584]: pgmap v300: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T14:50:19.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:19 smithi150 ceph-mon[32504]: pgmap v300: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T14:50:21.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:21 smithi053 ceph-mon[27773]: pgmap v301: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-04-25T14:50:21.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:21 smithi183 ceph-mon[32584]: pgmap v301: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-04-25T14:50:21.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:21 smithi150 ceph-mon[32504]: pgmap v301: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-04-25T14:50:23.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:23 smithi053 ceph-mon[27773]: pgmap v302: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T14:50:23.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:23 smithi183 ceph-mon[32584]: pgmap v302: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T14:50:23.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:23 smithi150 ceph-mon[32504]: pgmap v302: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T14:50:25.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:25 smithi053 ceph-mon[27773]: pgmap v303: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T14:50:25.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:25 smithi183 ceph-mon[32584]: pgmap v303: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T14:50:25.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:25 smithi150 ceph-mon[32504]: pgmap v303: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T14:50:27.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:27 smithi053 ceph-mon[27773]: pgmap v304: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 97 B/s wr, 8 op/s 2024-04-25T14:50:27.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:27 smithi183 ceph-mon[32584]: pgmap v304: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 97 B/s wr, 8 op/s 2024-04-25T14:50:27.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:27 smithi150 ceph-mon[32504]: pgmap v304: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 97 B/s wr, 8 op/s 2024-04-25T14:50:29.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:29 smithi183 ceph-mon[32584]: pgmap v305: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:50:29.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:29 smithi150 ceph-mon[32504]: pgmap v305: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:50:30.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:29 smithi053 ceph-mon[27773]: pgmap v305: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:50:31.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:31 smithi183 ceph-mon[32584]: pgmap v306: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:50:31.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:31 smithi150 ceph-mon[32504]: pgmap v306: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:50:32.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:31 smithi053 ceph-mon[27773]: pgmap v306: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:50:33.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:33 smithi183 ceph-mon[32584]: pgmap v307: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:50:33.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:33 smithi150 ceph-mon[32504]: pgmap v307: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:50:34.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:33 smithi053 ceph-mon[27773]: pgmap v307: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:50:35.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:35 smithi183 ceph-mon[32584]: pgmap v308: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:50:35.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:35 smithi150 ceph-mon[32504]: pgmap v308: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:50:36.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:35 smithi053 ceph-mon[27773]: pgmap v308: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:50:37.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:37 smithi183 ceph-mon[32584]: pgmap v309: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:50:37.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:37 smithi150 ceph-mon[32504]: pgmap v309: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:50:38.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:37 smithi053 ceph-mon[27773]: pgmap v309: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:50:39.836 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:39 smithi053 ceph-mon[27773]: pgmap v310: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:50:39.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:39 smithi183 ceph-mon[32584]: pgmap v310: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:50:39.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:39 smithi150 ceph-mon[32504]: pgmap v310: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:50:41.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:41 smithi183 ceph-mon[32584]: pgmap v311: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:50:41.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:41 smithi150 ceph-mon[32504]: pgmap v311: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:50:42.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:41 smithi053 ceph-mon[27773]: pgmap v311: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:50:43.318 DEBUG:teuthology.orchestra.run.smithi053:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T14:50:43.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.533+0000 7fa8c5400640 1 -- 172.21.15.53:0/1925594548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8c0108560 msgr2=0x7fa8c01089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:50:43.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.533+0000 7fa8c5400640 1 --2- 172.21.15.53:0/1925594548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8c0108560 0x7fa8c01089e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac009900 tx=0x7fa8ac02a3f0 comp rx=0 tx=0).stop 2024-04-25T14:50:43.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.533+0000 7fa8c5400640 1 -- 172.21.15.53:0/1925594548 shutdown_connections 2024-04-25T14:50:43.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.533+0000 7fa8c5400640 1 --2- 172.21.15.53:0/1925594548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8c0108f20 0x7fa8c01153e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:43.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.533+0000 7fa8c5400640 1 --2- 172.21.15.53:0/1925594548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8c0108560 0x7fa8c01089e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:43.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.533+0000 7fa8c5400640 1 --2- 172.21.15.53:0/1925594548 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8c010e620 0x7fa8c010ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:43.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.533+0000 7fa8c5400640 1 -- 172.21.15.53:0/1925594548 >> 172.21.15.53:0/1925594548 conn(0x7fa8c00764e0 msgr2=0x7fa8c0078900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:50:43.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.533+0000 7fa8c5400640 1 -- 172.21.15.53:0/1925594548 shutdown_connections 2024-04-25T14:50:43.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.533+0000 7fa8c5400640 1 -- 172.21.15.53:0/1925594548 wait complete. 2024-04-25T14:50:43.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.534+0000 7fa8c5400640 1 Processor -- start 2024-04-25T14:50:43.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.534+0000 7fa8c5400640 1 -- start start 2024-04-25T14:50:43.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.535+0000 7fa8c5400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8c0108560 0x7fa8c01a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:50:43.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.535+0000 7fa8c5400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8c0108f20 0x7fa8c01a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:50:43.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.535+0000 7fa8c5400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8c010e620 0x7fa8c019ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:50:43.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.535+0000 7fa8c5400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c0118790 con 0x7fa8c010e620 2024-04-25T14:50:43.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.535+0000 7fa8c5400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c0118610 con 0x7fa8c0108560 2024-04-25T14:50:43.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.535+0000 7fa8c5400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c0118910 con 0x7fa8c0108f20 2024-04-25T14:50:43.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.535+0000 7fa8bf200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8c010e620 0x7fa8c019ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:50:43.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.535+0000 7fa8bde00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8c0108f20 0x7fa8c01a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:50:43.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.535+0000 7fa8bde00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8c0108f20 0x7fa8c01a4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:56440/0 (socket says 172.21.15.53:56440) 2024-04-25T14:50:43.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.535+0000 7fa8bde00640 1 -- 172.21.15.53:0/1222567002 learned_addr learned my addr 172.21.15.53:0/1222567002 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:50:43.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.535+0000 7fa8bf200640 1 --2- 172.21.15.53:0/1222567002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8c010e620 0x7fa8c019ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55892/0 (socket says 172.21.15.53:55892) 2024-04-25T14:50:43.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.535+0000 7fa8be800640 1 --2- 172.21.15.53:0/1222567002 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8c0108560 0x7fa8c01a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:50:43.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.536+0000 7fa8bde00640 1 -- 172.21.15.53:0/1222567002 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8c0108560 msgr2=0x7fa8c01a48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:50:43.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.536+0000 7fa8bde00640 1 --2- 172.21.15.53:0/1222567002 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8c0108560 0x7fa8c01a48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:43.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.536+0000 7fa8bde00640 1 -- 172.21.15.53:0/1222567002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8c010e620 msgr2=0x7fa8c019ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:50:43.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.536+0000 7fa8bde00640 1 --2- 172.21.15.53:0/1222567002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8c010e620 0x7fa8c019ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:43.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.536+0000 7fa8bde00640 1 -- 172.21.15.53:0/1222567002 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c019f1d0 con 0x7fa8c0108f20 2024-04-25T14:50:43.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.536+0000 7fa8bf200640 1 --2- 172.21.15.53:0/1222567002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8c010e620 0x7fa8c019ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:50:43.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.536+0000 7fa8be800640 1 --2- 172.21.15.53:0/1222567002 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8c0108560 0x7fa8c01a48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:50:43.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.536+0000 7fa8bde00640 1 --2- 172.21.15.53:0/1222567002 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8c0108f20 0x7fa8c01a4de0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac006010 tx=0x7fa8ac0040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:50:43.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.536+0000 7fa8b7400640 1 -- 172.21.15.53:0/1222567002 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac03f030 con 0x7fa8c0108f20 2024-04-25T14:50:43.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.536+0000 7fa8b7400640 1 -- 172.21.15.53:0/1222567002 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8ac0044c0 con 0x7fa8c0108f20 2024-04-25T14:50:43.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.536+0000 7fa8b7400640 1 -- 172.21.15.53:0/1222567002 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac035070 con 0x7fa8c0108f20 2024-04-25T14:50:43.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.536+0000 7fa8c5400640 1 -- 172.21.15.53:0/1222567002 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8c019f460 con 0x7fa8c0108f20 2024-04-25T14:50:43.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.537+0000 7fa8c5400640 1 -- 172.21.15.53:0/1222567002 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8c01ab760 con 0x7fa8c0108f20 2024-04-25T14:50:43.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.538+0000 7fa8c5400640 1 -- 172.21.15.53:0/1222567002 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8c004fce0 con 0x7fa8c0108f20 2024-04-25T14:50:43.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.540+0000 7fa8b7400640 1 -- 172.21.15.53:0/1222567002 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fa8ac042030 con 0x7fa8c0108f20 2024-04-25T14:50:43.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.541+0000 7fa8b7400640 1 --2- 172.21.15.53:0/1222567002 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fa8a40a3dc0 0x7fa8a40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:50:43.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.542+0000 7fa8b7400640 1 -- 172.21.15.53:0/1222567002 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8587+0+0 (secure 0 0 0) 0x7fa8ac0f0e00 con 0x7fa8c0108f20 2024-04-25T14:50:43.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.542+0000 7fa8be800640 1 --2- 172.21.15.53:0/1222567002 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fa8a40a3dc0 0x7fa8a40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:50:43.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.542+0000 7fa8be800640 1 --2- 172.21.15.53:0/1222567002 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fa8a40a3dc0 0x7fa8a40a6280 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa8c019fb90 tx=0x7fa8b0005f90 comp rx=0 tx=0).ready entity=mgr.15093 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:50:43.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.544+0000 7fa8b7400640 1 -- 172.21.15.53:0/1222567002 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa8ac0bd400 con 0x7fa8c0108f20 2024-04-25T14:50:43.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.774+0000 7fa8c5400640 1 -- 172.21.15.53:0/1222567002 --> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fa8c01109d0 con 0x7fa8a40a3dc0 2024-04-25T14:50:43.792 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.791+0000 7fa8b7400640 1 -- 172.21.15.53:0/1222567002 <== mgr.15093 v2:172.21.15.53:6800/802683714 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7fa8c01109d0 con 0x7fa8a40a3dc0 2024-04-25T14:50:43.792 INFO:teuthology.orchestra.run.smithi053.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T14:40:41", "first": "2024-04-25T14:41:00", "last": "2024-04-25T14:50:00", "last_pruned": "2024-04-25T14:50:01", "created_count": 10, "pruned_count": 4, "active": true} 2024-04-25T14:50:43.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.794+0000 7fa8c5400640 1 -- 172.21.15.53:0/1222567002 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fa8a40a3dc0 msgr2=0x7fa8a40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:50:43.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.794+0000 7fa8c5400640 1 --2- 172.21.15.53:0/1222567002 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fa8a40a3dc0 0x7fa8a40a6280 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa8c019fb90 tx=0x7fa8b0005f90 comp rx=0 tx=0).stop 2024-04-25T14:50:43.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.795+0000 7fa8c5400640 1 -- 172.21.15.53:0/1222567002 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8c0108f20 msgr2=0x7fa8c01a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:50:43.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.795+0000 7fa8c5400640 1 --2- 172.21.15.53:0/1222567002 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8c0108f20 0x7fa8c01a4de0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac006010 tx=0x7fa8ac0040c0 comp rx=0 tx=0).stop 2024-04-25T14:50:43.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.795+0000 7fa8c5400640 1 -- 172.21.15.53:0/1222567002 shutdown_connections 2024-04-25T14:50:43.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.795+0000 7fa8c5400640 1 --2- 172.21.15.53:0/1222567002 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fa8a40a3dc0 0x7fa8a40a6280 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:43.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.795+0000 7fa8c5400640 1 --2- 172.21.15.53:0/1222567002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8c010e620 0x7fa8c019ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:43.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.795+0000 7fa8c5400640 1 --2- 172.21.15.53:0/1222567002 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8c0108f20 0x7fa8c01a4de0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:43.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.795+0000 7fa8c5400640 1 --2- 172.21.15.53:0/1222567002 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8c0108560 0x7fa8c01a48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:43.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.795+0000 7fa8c5400640 1 -- 172.21.15.53:0/1222567002 >> 172.21.15.53:0/1222567002 conn(0x7fa8c00764e0 msgr2=0x7fa8c0077de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:50:43.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.795+0000 7fa8c5400640 1 -- 172.21.15.53:0/1222567002 shutdown_connections 2024-04-25T14:50:43.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:43.795+0000 7fa8c5400640 1 -- 172.21.15.53:0/1222567002 wait complete. 2024-04-25T14:50:43.815 DEBUG:teuthology.orchestra.run.smithi053:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T14:50:43.890 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:43 smithi053 ceph-mon[27773]: pgmap v312: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:50:43.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:43 smithi183 ceph-mon[32584]: pgmap v312: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:50:43.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:43 smithi150 ceph-mon[32504]: pgmap v312: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T14:50:44.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.027+0000 7f58e3e00640 1 -- 172.21.15.53:0/1171725825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58dc104d60 msgr2=0x7f58dc108da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:50:44.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.027+0000 7f58e3e00640 1 --2- 172.21.15.53:0/1171725825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58dc104d60 0x7f58dc108da0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f58c8009900 tx=0x7f58c802a410 comp rx=0 tx=0).stop 2024-04-25T14:50:44.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.028+0000 7f58e3e00640 1 -- 172.21.15.53:0/1171725825 shutdown_connections 2024-04-25T14:50:44.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.028+0000 7f58e3e00640 1 --2- 172.21.15.53:0/1171725825 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58dc109480 0x7f58dc111050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:44.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.028+0000 7f58e3e00640 1 --2- 172.21.15.53:0/1171725825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58dc104d60 0x7f58dc108da0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:44.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.028+0000 7f58e3e00640 1 --2- 172.21.15.53:0/1171725825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58dc1043b0 0x7f58dc104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:44.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.028+0000 7f58e3e00640 1 -- 172.21.15.53:0/1171725825 >> 172.21.15.53:0/1171725825 conn(0x7f58dc0ffc50 msgr2=0x7f58dc102070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:50:44.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.028+0000 7f58e3e00640 1 -- 172.21.15.53:0/1171725825 shutdown_connections 2024-04-25T14:50:44.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.028+0000 7f58e3e00640 1 -- 172.21.15.53:0/1171725825 wait complete. 2024-04-25T14:50:44.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.029+0000 7f58e3e00640 1 Processor -- start 2024-04-25T14:50:44.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.029+0000 7f58e3e00640 1 -- start start 2024-04-25T14:50:44.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.029+0000 7f58e3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58dc1043b0 0x7f58dc1a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:50:44.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.029+0000 7f58e3e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58dc109480 0x7f58dc1a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:50:44.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.030+0000 7f58e3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58dc19a760 0x7f58dc19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:50:44.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.030+0000 7f58e3e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc1143c0 con 0x7f58dc19a760 2024-04-25T14:50:44.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.030+0000 7f58e3e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc114240 con 0x7f58dc109480 2024-04-25T14:50:44.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.030+0000 7f58e3e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc114540 con 0x7f58dc1043b0 2024-04-25T14:50:44.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.030+0000 7f58e1400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58dc1043b0 0x7f58dc1a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:50:44.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.030+0000 7f58e1400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58dc1043b0 0x7f58dc1a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:56448/0 (socket says 172.21.15.53:56448) 2024-04-25T14:50:44.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.030+0000 7f58e1400640 1 -- 172.21.15.53:0/4176622932 learned_addr learned my addr 172.21.15.53:0/4176622932 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:50:44.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.030+0000 7f58e0a00640 1 --2- 172.21.15.53:0/4176622932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58dc109480 0x7f58dc1a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:50:44.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.030+0000 7f58e1e00640 1 --2- 172.21.15.53:0/4176622932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58dc19a760 0x7f58dc19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:50:44.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.030+0000 7f58e1400640 1 -- 172.21.15.53:0/4176622932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58dc109480 msgr2=0x7f58dc1a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:50:44.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.030+0000 7f58e1400640 1 --2- 172.21.15.53:0/4176622932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58dc109480 0x7f58dc1a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:44.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.030+0000 7f58e1400640 1 -- 172.21.15.53:0/4176622932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58dc19a760 msgr2=0x7f58dc19abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:50:44.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.030+0000 7f58e1400640 1 --2- 172.21.15.53:0/4176622932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58dc19a760 0x7f58dc19abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:44.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.030+0000 7f58e1400640 1 -- 172.21.15.53:0/4176622932 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58dc19b360 con 0x7f58dc1043b0 2024-04-25T14:50:44.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.031+0000 7f58e1400640 1 --2- 172.21.15.53:0/4176622932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58dc1043b0 0x7f58dc1a0650 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f58d000e9f0 tx=0x7f58d000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:50:44.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.031+0000 7f58da000640 1 -- 172.21.15.53:0/4176622932 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d000ccc0 con 0x7f58dc1043b0 2024-04-25T14:50:44.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.031+0000 7f58da000640 1 -- 172.21.15.53:0/4176622932 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58d000ce60 con 0x7f58dc1043b0 2024-04-25T14:50:44.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.031+0000 7f58e3e00640 1 -- 172.21.15.53:0/4176622932 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58dc1a73f0 con 0x7f58dc1043b0 2024-04-25T14:50:44.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.031+0000 7f58da000640 1 -- 172.21.15.53:0/4176622932 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d0010430 con 0x7f58dc1043b0 2024-04-25T14:50:44.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.031+0000 7f58e3e00640 1 -- 172.21.15.53:0/4176622932 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58dc1a77a0 con 0x7f58dc1043b0 2024-04-25T14:50:44.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.033+0000 7f58e3e00640 1 -- 172.21.15.53:0/4176622932 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58dc105880 con 0x7f58dc1043b0 2024-04-25T14:50:44.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.034+0000 7f58da000640 1 -- 172.21.15.53:0/4176622932 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f58d00046c0 con 0x7f58dc1043b0 2024-04-25T14:50:44.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.036+0000 7f58da000640 1 --2- 172.21.15.53:0/4176622932 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f58c40a3e40 0x7f58c40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:50:44.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.036+0000 7f58da000640 1 -- 172.21.15.53:0/4176622932 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8587+0+0 (secure 0 0 0) 0x7f58d00d0bd0 con 0x7f58dc1043b0 2024-04-25T14:50:44.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.036+0000 7f58e0a00640 1 --2- 172.21.15.53:0/4176622932 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f58c40a3e40 0x7f58c40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:50:44.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.037+0000 7f58e0a00640 1 --2- 172.21.15.53:0/4176622932 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f58c40a3e40 0x7f58c40a6300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f58c8000c00 tx=0x7f58c8031400 comp rx=0 tx=0).ready entity=mgr.15093 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:50:44.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.040+0000 7f58da000640 1 -- 172.21.15.53:0/4176622932 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f58d009d1d0 con 0x7f58dc1043b0 2024-04-25T14:50:44.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.268+0000 7f58e3e00640 1 -- 172.21.15.53:0/4176622932 --> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f58dc104d60 con 0x7f58c40a3e40 2024-04-25T14:50:44.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.283+0000 7f58da000640 1 -- 172.21.15.53:0/4176622932 <== mgr.15093 v2:172.21.15.53:6800/802683714 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f58dc104d60 con 0x7f58c40a3e40 2024-04-25T14:50:44.284 INFO:teuthology.orchestra.run.smithi053.stdout:/ 1m 6m3h 2024-04-25T14:50:44.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.285+0000 7f58e3e00640 1 -- 172.21.15.53:0/4176622932 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f58c40a3e40 msgr2=0x7f58c40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:50:44.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.285+0000 7f58e3e00640 1 --2- 172.21.15.53:0/4176622932 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f58c40a3e40 0x7f58c40a6300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f58c8000c00 tx=0x7f58c8031400 comp rx=0 tx=0).stop 2024-04-25T14:50:44.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.285+0000 7f58e3e00640 1 -- 172.21.15.53:0/4176622932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58dc1043b0 msgr2=0x7f58dc1a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:50:44.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.285+0000 7f58e3e00640 1 --2- 172.21.15.53:0/4176622932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58dc1043b0 0x7f58dc1a0650 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f58d000e9f0 tx=0x7f58d000edf0 comp rx=0 tx=0).stop 2024-04-25T14:50:44.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.285+0000 7f58e3e00640 1 -- 172.21.15.53:0/4176622932 shutdown_connections 2024-04-25T14:50:44.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.285+0000 7f58e3e00640 1 --2- 172.21.15.53:0/4176622932 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f58c40a3e40 0x7f58c40a6300 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:44.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.285+0000 7f58e3e00640 1 --2- 172.21.15.53:0/4176622932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58dc19a760 0x7f58dc19abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:44.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.285+0000 7f58e3e00640 1 --2- 172.21.15.53:0/4176622932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58dc109480 0x7f58dc1a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:44.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.285+0000 7f58e3e00640 1 --2- 172.21.15.53:0/4176622932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58dc1043b0 0x7f58dc1a0650 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:44.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.285+0000 7f58e3e00640 1 -- 172.21.15.53:0/4176622932 >> 172.21.15.53:0/4176622932 conn(0x7f58dc0ffc50 msgr2=0x7f58dc100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:50:44.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.286+0000 7f58e3e00640 1 -- 172.21.15.53:0/4176622932 shutdown_connections 2024-04-25T14:50:44.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.286+0000 7f58e3e00640 1 -- 172.21.15.53:0/4176622932 wait complete. 2024-04-25T14:50:44.306 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-25T14:50:44.319 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:50:44.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.537+0000 7fa4dae00640 1 -- 172.21.15.53:0/822927116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4d410e460 msgr2=0x7fa4d410e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:50:44.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.537+0000 7fa4dae00640 1 --2- 172.21.15.53:0/822927116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4d410e460 0x7fa4d410e840 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8009900 tx=0x7fa4c802a480 comp rx=0 tx=0).stop 2024-04-25T14:50:44.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.537+0000 7fa4dae00640 1 -- 172.21.15.53:0/822927116 shutdown_connections 2024-04-25T14:50:44.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.537+0000 7fa4dae00640 1 --2- 172.21.15.53:0/822927116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4d4108d90 0x7fa4d4115200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:44.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.537+0000 7fa4dae00640 1 --2- 172.21.15.53:0/822927116 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4d41083f0 0x7fa4d4108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:44.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.537+0000 7fa4dae00640 1 --2- 172.21.15.53:0/822927116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4d410e460 0x7fa4d410e840 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:44.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.537+0000 7fa4dae00640 1 -- 172.21.15.53:0/822927116 >> 172.21.15.53:0/822927116 conn(0x7fa4d4077ad0 msgr2=0x7fa4d4079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:50:44.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.538+0000 7fa4dae00640 1 -- 172.21.15.53:0/822927116 shutdown_connections 2024-04-25T14:50:44.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.538+0000 7fa4dae00640 1 -- 172.21.15.53:0/822927116 wait complete. 2024-04-25T14:50:44.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.539+0000 7fa4dae00640 1 Processor -- start 2024-04-25T14:50:44.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.539+0000 7fa4dae00640 1 -- start start 2024-04-25T14:50:44.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.539+0000 7fa4dae00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4d41083f0 0x7fa4d41a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:50:44.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.539+0000 7fa4dae00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4d4108d90 0x7fa4d41a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:50:44.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.539+0000 7fa4dae00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4d410e460 0x7fa4d419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:50:44.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.539+0000 7fa4dae00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d41185d0 con 0x7fa4d41083f0 2024-04-25T14:50:44.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.539+0000 7fa4dae00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d4118450 con 0x7fa4d410e460 2024-04-25T14:50:44.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.539+0000 7fa4dae00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d4118750 con 0x7fa4d4108d90 2024-04-25T14:50:44.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.540+0000 7fa4d9a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4d41083f0 0x7fa4d41a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:50:44.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.540+0000 7fa4d9000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4d4108d90 0x7fa4d41a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:50:44.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.540+0000 7fa4da400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4d410e460 0x7fa4d419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:50:44.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.540+0000 7fa4d9a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4d41083f0 0x7fa4d41a46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55932/0 (socket says 172.21.15.53:55932) 2024-04-25T14:50:44.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.540+0000 7fa4d9a00640 1 -- 172.21.15.53:0/1396114528 learned_addr learned my addr 172.21.15.53:0/1396114528 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:50:44.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.540+0000 7fa4da400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4d410e460 0x7fa4d419e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.53:50774/0 (socket says 172.21.15.53:50774) 2024-04-25T14:50:44.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.540+0000 7fa4d9000640 1 -- 172.21.15.53:0/1396114528 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4d410e460 msgr2=0x7fa4d419e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:50:44.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.540+0000 7fa4d9000640 1 --2- 172.21.15.53:0/1396114528 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4d410e460 0x7fa4d419e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:44.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.540+0000 7fa4d9000640 1 -- 172.21.15.53:0/1396114528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4d41083f0 msgr2=0x7fa4d41a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:50:44.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.540+0000 7fa4d9000640 1 --2- 172.21.15.53:0/1396114528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4d41083f0 0x7fa4d41a46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:44.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.540+0000 7fa4d9000640 1 -- 172.21.15.53:0/1396114528 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4d419f010 con 0x7fa4d4108d90 2024-04-25T14:50:44.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.540+0000 7fa4da400640 1 --2- 172.21.15.53:0/1396114528 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4d410e460 0x7fa4d419e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:50:44.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.540+0000 7fa4d9a00640 1 --2- 172.21.15.53:0/1396114528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4d41083f0 0x7fa4d41a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:50:44.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.541+0000 7fa4d9000640 1 --2- 172.21.15.53:0/1396114528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4d4108d90 0x7fa4d41a4c20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa4c000e750 tx=0x7fa4c000ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:50:44.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.541+0000 7fa4d2000640 1 -- 172.21.15.53:0/1396114528 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c000ccc0 con 0x7fa4d4108d90 2024-04-25T14:50:44.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.541+0000 7fa4d2000640 1 -- 172.21.15.53:0/1396114528 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4c000ee60 con 0x7fa4d4108d90 2024-04-25T14:50:44.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.541+0000 7fa4dae00640 1 -- 172.21.15.53:0/1396114528 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4d419f300 con 0x7fa4d4108d90 2024-04-25T14:50:44.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.541+0000 7fa4d2000640 1 -- 172.21.15.53:0/1396114528 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c0004a00 con 0x7fa4d4108d90 2024-04-25T14:50:44.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.541+0000 7fa4dae00640 1 -- 172.21.15.53:0/1396114528 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4d41ab5a0 con 0x7fa4d4108d90 2024-04-25T14:50:44.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.542+0000 7fa4dae00640 1 -- 172.21.15.53:0/1396114528 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4d40690e0 con 0x7fa4d4108d90 2024-04-25T14:50:44.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.545+0000 7fa4d2000640 1 -- 172.21.15.53:0/1396114528 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fa4c0004ba0 con 0x7fa4d4108d90 2024-04-25T14:50:44.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.547+0000 7fa4d2000640 1 --2- 172.21.15.53:0/1396114528 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fa4bc0a3d20 0x7fa4bc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:50:44.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.547+0000 7fa4d2000640 1 -- 172.21.15.53:0/1396114528 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8587+0+0 (secure 0 0 0) 0x7fa4c00d0870 con 0x7fa4d4108d90 2024-04-25T14:50:44.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.547+0000 7fa4d9a00640 1 --2- 172.21.15.53:0/1396114528 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fa4bc0a3d20 0x7fa4bc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:50:44.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.548+0000 7fa4d9a00640 1 --2- 172.21.15.53:0/1396114528 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fa4bc0a3d20 0x7fa4bc0a61e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8009230 tx=0x7fa4c80058d0 comp rx=0 tx=0).ready entity=mgr.15093 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:50:44.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.548+0000 7fa4d2000640 1 -- 172.21.15.53:0/1396114528 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa4c009ce70 con 0x7fa4d4108d90 2024-04-25T14:50:44.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.758+0000 7fa4dae00640 1 -- 172.21.15.53:0/1396114528 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa4d402d5c0 con 0x7fa4d4108d90 2024-04-25T14:50:44.761 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.760+0000 7fa4d2000640 1 -- 172.21.15.53:0/1396114528 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+23201 (secure 0 0 0) 0x7fa4c009c770 con 0x7fa4d4108d90 2024-04-25T14:50:44.762 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:50:44.762 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":101,"fsid":"fd30e6ce-030f-11ef-bc93-c7b262605968","created":"2024-04-25T14:28:54.330326+0000","modified":"2024-04-25T14:50:15.909422+0000","last_up_change":"2024-04-25T14:37:10.921148+0000","last_in_change":"2024-04-25T14:36:49.566459+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:33:02.338115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:38:07.517974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:38:08.994122+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":100,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aecfbaf0-39da-4749-a49b-c7205dabd4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6803","nonce":464176015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6805","nonce":464176015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6809","nonce":464176015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6807","nonce":464176015}]},"public_addr":"172.21.15.53:6803/464176015","cluster_addr":"172.21.15.53:6805/464176015","heartbeat_back_addr":"172.21.15.53:6809/464176015","heartbeat_front_addr":"172.21.15.53:6807/464176015","state":["exists","up"]},{"osd":1,"uuid":"2c4a21ae-c43e-4839-8a56-c3fc63aeb7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6801","nonce":1304621982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6803","nonce":1304621982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6807","nonce":1304621982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6805","nonce":1304621982}]},"public_addr":"172.21.15.150:6801/1304621982","cluster_addr":"172.21.15.150:6803/1304621982","heartbeat_back_addr":"172.21.15.150:6807/1304621982","heartbeat_front_addr":"172.21.15.150:6805/1304621982","state":["exists","up"]},{"osd":2,"uuid":"e8effd43-adc0-472a-9eed-a9273b06dc0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6801","nonce":574509311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6803","nonce":574509311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6807","nonce":574509311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6805","nonce":574509311}]},"public_addr":"172.21.15.183:6801/574509311","cluster_addr":"172.21.15.183:6803/574509311","heartbeat_back_addr":"172.21.15.183:6807/574509311","heartbeat_front_addr":"172.21.15.183:6805/574509311","state":["exists","up"]},{"osd":3,"uuid":"e7cfced4-9b40-40de-a303-75ea8d5df4a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6811","nonce":570672219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6813","nonce":570672219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6817","nonce":570672219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6815","nonce":570672219}]},"public_addr":"172.21.15.53:6811/570672219","cluster_addr":"172.21.15.53:6813/570672219","heartbeat_back_addr":"172.21.15.53:6817/570672219","heartbeat_front_addr":"172.21.15.53:6815/570672219","state":["exists","up"]},{"osd":4,"uuid":"44a9e070-2282-43f1-9f05-b1f8ff81e540","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6809","nonce":1526850103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6811","nonce":1526850103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6815","nonce":1526850103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6813","nonce":1526850103}]},"public_addr":"172.21.15.150:6809/1526850103","cluster_addr":"172.21.15.150:6811/1526850103","heartbeat_back_addr":"172.21.15.150:6815/1526850103","heartbeat_front_addr":"172.21.15.150:6813/1526850103","state":["exists","up"]},{"osd":5,"uuid":"2c5619d2-c0f9-47d6-bf7e-2f4afd1b09f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6809","nonce":773726576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6811","nonce":773726576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6815","nonce":773726576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6813","nonce":773726576}]},"public_addr":"172.21.15.183:6809/773726576","cluster_addr":"172.21.15.183:6811/773726576","heartbeat_back_addr":"172.21.15.183:6815/773726576","heartbeat_front_addr":"172.21.15.183:6813/773726576","state":["exists","up"]},{"osd":6,"uuid":"1cf74d94-c118-4738-b6db-040af458a8eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6819","nonce":355884319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6821","nonce":355884319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6825","nonce":355884319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6823","nonce":355884319}]},"public_addr":"172.21.15.53:6819/355884319","cluster_addr":"172.21.15.53:6821/355884319","heartbeat_back_addr":"172.21.15.53:6825/355884319","heartbeat_front_addr":"172.21.15.53:6823/355884319","state":["exists","up"]},{"osd":7,"uuid":"0ce97c04-cd41-4080-bbbc-be9cf5fadc02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6817","nonce":229022562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6819","nonce":229022562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6823","nonce":229022562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6821","nonce":229022562}]},"public_addr":"172.21.15.150:6817/229022562","cluster_addr":"172.21.15.150:6819/229022562","heartbeat_back_addr":"172.21.15.150:6823/229022562","heartbeat_front_addr":"172.21.15.150:6821/229022562","state":["exists","up"]},{"osd":8,"uuid":"5d57787d-2a7c-4514-bd01-357ced413f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6817","nonce":301003844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6819","nonce":301003844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6823","nonce":301003844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6821","nonce":301003844}]},"public_addr":"172.21.15.183:6817/301003844","cluster_addr":"172.21.15.183:6819/301003844","heartbeat_back_addr":"172.21.15.183:6823/301003844","heartbeat_front_addr":"172.21.15.183:6821/301003844","state":["exists","up"]},{"osd":9,"uuid":"e11dd8eb-c55a-4d70-8c58-457540b7a50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6827","nonce":3449260978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6829","nonce":3449260978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6833","nonce":3449260978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6831","nonce":3449260978}]},"public_addr":"172.21.15.53:6827/3449260978","cluster_addr":"172.21.15.53:6829/3449260978","heartbeat_back_addr":"172.21.15.53:6833/3449260978","heartbeat_front_addr":"172.21.15.53:6831/3449260978","state":["exists","up"]},{"osd":10,"uuid":"5de4803d-ad37-4a90-add5-0048f44fc4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6825","nonce":3435992308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6827","nonce":3435992308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6831","nonce":3435992308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6829","nonce":3435992308}]},"public_addr":"172.21.15.150:6825/3435992308","cluster_addr":"172.21.15.150:6827/3435992308","heartbeat_back_addr":"172.21.15.150:6831/3435992308","heartbeat_front_addr":"172.21.15.150:6829/3435992308","state":["exists","up"]},{"osd":11,"uuid":"0e819a68-ed56-477e-b9c0-81e8cc0bce11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6825","nonce":1957842686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6827","nonce":1957842686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6831","nonce":1957842686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6829","nonce":1957842686}]},"public_addr":"172.21.15.183:6825/1957842686","cluster_addr":"172.21.15.183:6827/1957842686","heartbeat_back_addr":"172.21.15.183:6831/1957842686","heartbeat_front_addr":"172.21.15.183:6829/1957842686","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:09.840629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:34.769537+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:58.112969+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:33:29.656790+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:33:55.181287+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:34:19.772645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:34:53.833921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:35:19.076733+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:35:43.163390+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:36:16.568340+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:36:42.492768+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:37:07.944639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"3.38","mappings":[{"from":5,"to":7},{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6800/4154933108":"2024-04-26T14:40:40.168940+0000","172.21.15.53:0/3916258673":"2024-04-26T14:40:40.168940+0000","172.21.15.53:0/309091034":"2024-04-26T14:40:40.168940+0000","172.21.15.53:0/4267178590":"2024-04-26T14:40:40.168940+0000","172.21.15.53:0/920465993":"2024-04-26T14:40:40.168940+0000","172.21.15.53:6800/3887954889":"2024-04-26T14:30:14.162383+0000","172.21.15.53:6801/4154933108":"2024-04-26T14:40:40.168940+0000","172.21.15.53:0/1014501022":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/2576091441":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/1233527569":"2024-04-26T14:29:38.106242+0000","172.21.15.53:6801/3887954889":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/931889810":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/177196141":"2024-04-26T14:30:14.162383+0000","172.21.15.53:6800/1722497390":"2024-04-26T14:29:38.106242+0000","172.21.15.53:6801/1722497390":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/1927549198":"2024-04-26T14:30:14.162383+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:50:44.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.763+0000 7fa4dae00640 1 -- 172.21.15.53:0/1396114528 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fa4bc0a3d20 msgr2=0x7fa4bc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:50:44.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.763+0000 7fa4dae00640 1 --2- 172.21.15.53:0/1396114528 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fa4bc0a3d20 0x7fa4bc0a61e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8009230 tx=0x7fa4c80058d0 comp rx=0 tx=0).stop 2024-04-25T14:50:44.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.763+0000 7fa4dae00640 1 -- 172.21.15.53:0/1396114528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4d4108d90 msgr2=0x7fa4d41a4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:50:44.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.763+0000 7fa4dae00640 1 --2- 172.21.15.53:0/1396114528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4d4108d90 0x7fa4d41a4c20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa4c000e750 tx=0x7fa4c000ec10 comp rx=0 tx=0).stop 2024-04-25T14:50:44.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.763+0000 7fa4dae00640 1 -- 172.21.15.53:0/1396114528 shutdown_connections 2024-04-25T14:50:44.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.763+0000 7fa4dae00640 1 --2- 172.21.15.53:0/1396114528 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fa4bc0a3d20 0x7fa4bc0a61e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:44.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.763+0000 7fa4dae00640 1 --2- 172.21.15.53:0/1396114528 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4d410e460 0x7fa4d419e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:44.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.763+0000 7fa4dae00640 1 --2- 172.21.15.53:0/1396114528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4d4108d90 0x7fa4d41a4c20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:44.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.763+0000 7fa4dae00640 1 --2- 172.21.15.53:0/1396114528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4d41083f0 0x7fa4d41a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:44.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.763+0000 7fa4dae00640 1 -- 172.21.15.53:0/1396114528 >> 172.21.15.53:0/1396114528 conn(0x7fa4d4077ad0 msgr2=0x7fa4d4079220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:50:44.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.763+0000 7fa4dae00640 1 -- 172.21.15.53:0/1396114528 shutdown_connections 2024-04-25T14:50:44.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:44.764+0000 7fa4dae00640 1 -- 172.21.15.53:0/1396114528 wait complete. 2024-04-25T14:50:44.783 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:33:02.338115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:38:07.517974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:38:08.994122+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 100, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:50:44.784 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:50:44.857 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:44 smithi053 ceph-mon[27773]: from='client.25435 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:50:44.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:44 smithi183 ceph-mon[32584]: from='client.25435 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:50:44.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:44 smithi150 ceph-mon[32504]: from='client.25435 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:50:45.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.002+0000 7f1aa8c00640 1 -- 172.21.15.53:0/381818103 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1aa41043a0 msgr2=0x7f1aa4104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:50:45.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.002+0000 7f1aa8c00640 1 --2- 172.21.15.53:0/381818103 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1aa41043a0 0x7f1aa4104780 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1a94009ca0 tx=0x7f1a9402a8b0 comp rx=0 tx=0).stop 2024-04-25T14:50:45.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.002+0000 7f1aa8c00640 1 -- 172.21.15.53:0/381818103 shutdown_connections 2024-04-25T14:50:45.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.002+0000 7f1aa8c00640 1 --2- 172.21.15.53:0/381818103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1aa4109470 0x7f1aa4111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:45.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.002+0000 7f1aa8c00640 1 --2- 172.21.15.53:0/381818103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1aa4104d50 0x7f1aa4108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:45.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.002+0000 7f1aa8c00640 1 --2- 172.21.15.53:0/381818103 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1aa41043a0 0x7f1aa4104780 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:45.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.002+0000 7f1aa8c00640 1 -- 172.21.15.53:0/381818103 >> 172.21.15.53:0/381818103 conn(0x7f1aa40ffbc0 msgr2=0x7f1aa4101fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:50:45.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.002+0000 7f1aa8c00640 1 -- 172.21.15.53:0/381818103 shutdown_connections 2024-04-25T14:50:45.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.002+0000 7f1aa8c00640 1 -- 172.21.15.53:0/381818103 wait complete. 2024-04-25T14:50:45.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.003+0000 7f1aa8c00640 1 Processor -- start 2024-04-25T14:50:45.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.004+0000 7f1aa8c00640 1 -- start start 2024-04-25T14:50:45.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.004+0000 7f1aa8c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1aa41043a0 0x7f1aa41a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:50:45.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.004+0000 7f1aa8c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1aa4109470 0x7f1aa41a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:50:45.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.004+0000 7f1aa8c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1aa419a830 0x7f1aa419ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:50:45.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.004+0000 7f1aa8c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa41143d0 con 0x7f1aa41043a0 2024-04-25T14:50:45.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.004+0000 7f1aa8c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa4114250 con 0x7f1aa419a830 2024-04-25T14:50:45.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.004+0000 7f1aa8c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa4114550 con 0x7f1aa4109470 2024-04-25T14:50:45.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.005+0000 7f1aa1e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1aa41043a0 0x7f1aa41a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:50:45.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.005+0000 7f1aa1400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1aa4109470 0x7f1aa41a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:50:45.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.005+0000 7f1aa1e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1aa41043a0 0x7f1aa41a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55948/0 (socket says 172.21.15.53:55948) 2024-04-25T14:50:45.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.005+0000 7f1aa1e00640 1 -- 172.21.15.53:0/2998956296 learned_addr learned my addr 172.21.15.53:0/2998956296 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:50:45.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.005+0000 7f1aa2800640 1 --2- 172.21.15.53:0/2998956296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1aa419a830 0x7f1aa419ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:50:45.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.005+0000 7f1aa1e00640 1 -- 172.21.15.53:0/2998956296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1aa4109470 msgr2=0x7f1aa41a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:50:45.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.005+0000 7f1aa1e00640 1 --2- 172.21.15.53:0/2998956296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1aa4109470 0x7f1aa41a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:45.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.005+0000 7f1aa1e00640 1 -- 172.21.15.53:0/2998956296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1aa419a830 msgr2=0x7f1aa419ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:50:45.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.005+0000 7f1aa1e00640 1 --2- 172.21.15.53:0/2998956296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1aa419a830 0x7f1aa419ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:45.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.005+0000 7f1aa1e00640 1 -- 172.21.15.53:0/2998956296 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa419b3c0 con 0x7f1aa41043a0 2024-04-25T14:50:45.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.005+0000 7f1aa1e00640 1 --2- 172.21.15.53:0/2998956296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1aa41043a0 0x7f1aa41a0670 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1a94006010 tx=0x7f1a94030cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:50:45.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.006+0000 7f1a9aa00640 1 -- 172.21.15.53:0/2998956296 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a94004040 con 0x7f1aa41043a0 2024-04-25T14:50:45.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.006+0000 7f1a9aa00640 1 -- 172.21.15.53:0/2998956296 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a94040070 con 0x7f1aa41043a0 2024-04-25T14:50:45.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.006+0000 7f1a9aa00640 1 -- 172.21.15.53:0/2998956296 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a94045b50 con 0x7f1aa41043a0 2024-04-25T14:50:45.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.006+0000 7f1aa8c00640 1 -- 172.21.15.53:0/2998956296 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aa419b650 con 0x7f1aa41043a0 2024-04-25T14:50:45.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.006+0000 7f1aa8c00640 1 -- 172.21.15.53:0/2998956296 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aa41a7830 con 0x7f1aa41043a0 2024-04-25T14:50:45.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.007+0000 7f1aa8c00640 1 -- 172.21.15.53:0/2998956296 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1aa4105870 con 0x7f1aa41043a0 2024-04-25T14:50:45.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.009+0000 7f1a9aa00640 1 -- 172.21.15.53:0/2998956296 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f1a94045cf0 con 0x7f1aa41043a0 2024-04-25T14:50:45.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.011+0000 7f1a9aa00640 1 --2- 172.21.15.53:0/2998956296 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f1a880a3da0 0x7f1a880a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:50:45.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.011+0000 7f1a9aa00640 1 -- 172.21.15.53:0/2998956296 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8587+0+0 (secure 0 0 0) 0x7f1a940f1980 con 0x7f1aa41043a0 2024-04-25T14:50:45.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.011+0000 7f1aa1400640 1 --2- 172.21.15.53:0/2998956296 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f1a880a3da0 0x7f1a880a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:50:45.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.012+0000 7f1aa1400640 1 --2- 172.21.15.53:0/2998956296 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f1a880a3da0 0x7f1a880a6260 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1aa410a980 tx=0x7f1a8c005f90 comp rx=0 tx=0).ready entity=mgr.15093 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:50:45.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.014+0000 7f1a9aa00640 1 -- 172.21.15.53:0/2998956296 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1a940be120 con 0x7f1aa41043a0 2024-04-25T14:50:45.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.218+0000 7f1aa8c00640 1 -- 172.21.15.53:0/2998956296 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1aa40691c0 con 0x7f1aa41043a0 2024-04-25T14:50:45.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.219+0000 7f1a9aa00640 1 -- 172.21.15.53:0/2998956296 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) ==== 93+0+10 (secure 0 0 0) 0x7f1a940bda20 con 0x7f1aa41043a0 2024-04-25T14:50:45.221 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2024-04-25T14:50:45.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.222+0000 7f1aa8c00640 1 -- 172.21.15.53:0/2998956296 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f1a880a3da0 msgr2=0x7f1a880a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:50:45.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.222+0000 7f1aa8c00640 1 --2- 172.21.15.53:0/2998956296 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f1a880a3da0 0x7f1a880a6260 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1aa410a980 tx=0x7f1a8c005f90 comp rx=0 tx=0).stop 2024-04-25T14:50:45.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.222+0000 7f1aa8c00640 1 -- 172.21.15.53:0/2998956296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1aa41043a0 msgr2=0x7f1aa41a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:50:45.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.222+0000 7f1aa8c00640 1 --2- 172.21.15.53:0/2998956296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1aa41043a0 0x7f1aa41a0670 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1a94006010 tx=0x7f1a94030cf0 comp rx=0 tx=0).stop 2024-04-25T14:50:45.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.222+0000 7f1aa8c00640 1 -- 172.21.15.53:0/2998956296 shutdown_connections 2024-04-25T14:50:45.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.222+0000 7f1aa8c00640 1 --2- 172.21.15.53:0/2998956296 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f1a880a3da0 0x7f1a880a6260 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:45.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.222+0000 7f1aa8c00640 1 --2- 172.21.15.53:0/2998956296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1aa419a830 0x7f1aa419ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:45.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.222+0000 7f1aa8c00640 1 --2- 172.21.15.53:0/2998956296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1aa4109470 0x7f1aa41a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:45.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.222+0000 7f1aa8c00640 1 --2- 172.21.15.53:0/2998956296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1aa41043a0 0x7f1aa41a0670 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:45.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.222+0000 7f1aa8c00640 1 -- 172.21.15.53:0/2998956296 >> 172.21.15.53:0/2998956296 conn(0x7f1aa40ffbc0 msgr2=0x7f1aa4101010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:50:45.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.223+0000 7f1aa8c00640 1 -- 172.21.15.53:0/2998956296 shutdown_connections 2024-04-25T14:50:45.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.223+0000 7f1aa8c00640 1 -- 172.21.15.53:0/2998956296 wait complete. 2024-04-25T14:50:45.242 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:50:45.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.463+0000 7f87a6600640 1 -- 172.21.15.53:0/867131375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87a0109330 msgr2=0x7f87a0110f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:50:45.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.463+0000 7f87a6600640 1 --2- 172.21.15.53:0/867131375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87a0109330 0x7f87a0110f00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8798009ca0 tx=0x7f879802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:50:45.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.463+0000 7f87a6600640 1 -- 172.21.15.53:0/867131375 shutdown_connections 2024-04-25T14:50:45.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.463+0000 7f87a6600640 1 --2- 172.21.15.53:0/867131375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87a0109330 0x7f87a0110f00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:45.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.463+0000 7f87a6600640 1 --2- 172.21.15.53:0/867131375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87a0104c10 0x7f87a0108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:45.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.463+0000 7f87a6600640 1 --2- 172.21.15.53:0/867131375 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87a0104260 0x7f87a0104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:45.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.463+0000 7f87a6600640 1 -- 172.21.15.53:0/867131375 >> 172.21.15.53:0/867131375 conn(0x7f87a00ffac0 msgr2=0x7f87a0101ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:50:45.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.463+0000 7f87a6600640 1 -- 172.21.15.53:0/867131375 shutdown_connections 2024-04-25T14:50:45.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.463+0000 7f87a6600640 1 -- 172.21.15.53:0/867131375 wait complete. 2024-04-25T14:50:45.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.464+0000 7f87a6600640 1 Processor -- start 2024-04-25T14:50:45.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.464+0000 7f87a6600640 1 -- start start 2024-04-25T14:50:45.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.465+0000 7f87a6600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87a0104260 0x7f87a01a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:50:45.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.465+0000 7f87a6600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87a0109330 0x7f87a01a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:50:45.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.465+0000 7f87a6600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87a019a5f0 0x7f87a019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:50:45.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.465+0000 7f87a6600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f87a01142d0 con 0x7f87a0104260 2024-04-25T14:50:45.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.465+0000 7f87a6600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f87a0114150 con 0x7f87a0109330 2024-04-25T14:50:45.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.465+0000 7f87a6600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f87a0114450 con 0x7f87a019a5f0 2024-04-25T14:50:45.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.465+0000 7f87a5200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87a0104260 0x7f87a01a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:50:45.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.465+0000 7f87a5200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87a0104260 0x7f87a01a0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55964/0 (socket says 172.21.15.53:55964) 2024-04-25T14:50:45.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.465+0000 7f87a5200640 1 -- 172.21.15.53:0/1033125475 learned_addr learned my addr 172.21.15.53:0/1033125475 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:50:45.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.465+0000 7f879fe00640 1 --2- 172.21.15.53:0/1033125475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87a0109330 0x7f87a01a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:50:45.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.465+0000 7f87a5c00640 1 --2- 172.21.15.53:0/1033125475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87a019a5f0 0x7f87a019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:50:45.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.465+0000 7f879fe00640 1 -- 172.21.15.53:0/1033125475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87a019a5f0 msgr2=0x7f87a019aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:50:45.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.466+0000 7f879fe00640 1 --2- 172.21.15.53:0/1033125475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87a019a5f0 0x7f87a019aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:45.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.466+0000 7f879fe00640 1 -- 172.21.15.53:0/1033125475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87a0104260 msgr2=0x7f87a01a0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:50:45.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.466+0000 7f879fe00640 1 --2- 172.21.15.53:0/1033125475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87a0104260 0x7f87a01a0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:45.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.466+0000 7f879fe00640 1 -- 172.21.15.53:0/1033125475 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a019b110 con 0x7f87a0109330 2024-04-25T14:50:45.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.466+0000 7f87a5200640 1 --2- 172.21.15.53:0/1033125475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87a0104260 0x7f87a01a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:50:45.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.466+0000 7f879fe00640 1 --2- 172.21.15.53:0/1033125475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87a0109330 0x7f87a01a0980 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f878c00e9c0 tx=0x7f878c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:50:45.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.466+0000 7f879d600640 1 -- 172.21.15.53:0/1033125475 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c00cd80 con 0x7f87a0109330 2024-04-25T14:50:45.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.466+0000 7f879d600640 1 -- 172.21.15.53:0/1033125475 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f878c0035f0 con 0x7f87a0109330 2024-04-25T14:50:45.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.466+0000 7f87a6600640 1 -- 172.21.15.53:0/1033125475 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87a019b400 con 0x7f87a0109330 2024-04-25T14:50:45.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.466+0000 7f879d600640 1 -- 172.21.15.53:0/1033125475 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c010690 con 0x7f87a0109330 2024-04-25T14:50:45.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.466+0000 7f87a6600640 1 -- 172.21.15.53:0/1033125475 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87a01a76a0 con 0x7f87a0109330 2024-04-25T14:50:45.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.467+0000 7f87a6600640 1 -- 172.21.15.53:0/1033125475 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87a00690e0 con 0x7f87a0109330 2024-04-25T14:50:45.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.470+0000 7f879d600640 1 -- 172.21.15.53:0/1033125475 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f878c0037a0 con 0x7f87a0109330 2024-04-25T14:50:45.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.472+0000 7f879d600640 1 --2- 172.21.15.53:0/1033125475 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f87880a3da0 0x7f87880a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:50:45.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.472+0000 7f879d600640 1 -- 172.21.15.53:0/1033125475 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8587+0+0 (secure 0 0 0) 0x7f878c0d0b90 con 0x7f87a0109330 2024-04-25T14:50:45.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.472+0000 7f87a5200640 1 --2- 172.21.15.53:0/1033125475 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f87880a3da0 0x7f87880a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:50:45.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.473+0000 7f87a5200640 1 --2- 172.21.15.53:0/1033125475 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f87880a3da0 0x7f87880a6260 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8794011550 tx=0x7f8794005f90 comp rx=0 tx=0).ready entity=mgr.15093 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:50:45.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.474+0000 7f879d600640 1 -- 172.21.15.53:0/1033125475 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f878c09d190 con 0x7f87a0109330 2024-04-25T14:50:45.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.677+0000 7f87a6600640 1 -- 172.21.15.53:0/1033125475 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f87a010c5c0 con 0x7f87a0109330 2024-04-25T14:50:45.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.678+0000 7f879d600640 1 -- 172.21.15.53:0/1033125475 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v101) ==== 104+0+11 (secure 0 0 0) 0x7f878c09ca90 con 0x7f87a0109330 2024-04-25T14:50:45.680 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2024-04-25T14:50:45.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.681+0000 7f87a6600640 1 -- 172.21.15.53:0/1033125475 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f87880a3da0 msgr2=0x7f87880a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:50:45.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.681+0000 7f87a6600640 1 --2- 172.21.15.53:0/1033125475 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f87880a3da0 0x7f87880a6260 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8794011550 tx=0x7f8794005f90 comp rx=0 tx=0).stop 2024-04-25T14:50:45.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.681+0000 7f87a6600640 1 -- 172.21.15.53:0/1033125475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87a0109330 msgr2=0x7f87a01a0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:50:45.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.681+0000 7f87a6600640 1 --2- 172.21.15.53:0/1033125475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87a0109330 0x7f87a01a0980 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f878c00e9c0 tx=0x7f878c00ee80 comp rx=0 tx=0).stop 2024-04-25T14:50:45.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.682+0000 7f87a6600640 1 -- 172.21.15.53:0/1033125475 shutdown_connections 2024-04-25T14:50:45.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.682+0000 7f87a6600640 1 --2- 172.21.15.53:0/1033125475 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f87880a3da0 0x7f87880a6260 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:45.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.682+0000 7f87a6600640 1 --2- 172.21.15.53:0/1033125475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87a019a5f0 0x7f87a019aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:45.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.682+0000 7f87a6600640 1 --2- 172.21.15.53:0/1033125475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87a0109330 0x7f87a01a0980 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:45.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.682+0000 7f87a6600640 1 --2- 172.21.15.53:0/1033125475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87a0104260 0x7f87a01a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:45.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.682+0000 7f87a6600640 1 -- 172.21.15.53:0/1033125475 >> 172.21.15.53:0/1033125475 conn(0x7f87a00ffac0 msgr2=0x7f87a0101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:50:45.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.682+0000 7f87a6600640 1 -- 172.21.15.53:0/1033125475 shutdown_connections 2024-04-25T14:50:45.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.682+0000 7f87a6600640 1 -- 172.21.15.53:0/1033125475 wait complete. 2024-04-25T14:50:45.702 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:50:45.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.916+0000 7ffa49800640 1 -- 172.21.15.53:0/1156967070 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa4410d3c0 msgr2=0x7ffa4410d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:50:45.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.916+0000 7ffa49800640 1 --2- 172.21.15.53:0/1156967070 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa4410d3c0 0x7ffa4410d7a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ffa34009be0 tx=0x7ffa3402a750 comp rx=0 tx=0).stop 2024-04-25T14:50:45.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.917+0000 7ffa49800640 1 -- 172.21.15.53:0/1156967070 shutdown_connections 2024-04-25T14:50:45.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.917+0000 7ffa49800640 1 --2- 172.21.15.53:0/1156967070 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa44108e90 0x7ffa441152b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:45.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.917+0000 7ffa49800640 1 --2- 172.21.15.53:0/1156967070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa441084f0 0x7ffa44108950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:45.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.917+0000 7ffa49800640 1 --2- 172.21.15.53:0/1156967070 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa4410d3c0 0x7ffa4410d7a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:45.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.917+0000 7ffa49800640 1 -- 172.21.15.53:0/1156967070 >> 172.21.15.53:0/1156967070 conn(0x7ffa44077d70 msgr2=0x7ffa44106d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:50:45.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.917+0000 7ffa49800640 1 -- 172.21.15.53:0/1156967070 shutdown_connections 2024-04-25T14:50:45.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.917+0000 7ffa49800640 1 -- 172.21.15.53:0/1156967070 wait complete. 2024-04-25T14:50:45.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.918+0000 7ffa49800640 1 Processor -- start 2024-04-25T14:50:45.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.918+0000 7ffa49800640 1 -- start start 2024-04-25T14:50:45.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.919+0000 7ffa49800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa441084f0 0x7ffa441a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:50:45.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.919+0000 7ffa49800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa44108e90 0x7ffa441a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:50:45.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.919+0000 7ffa49800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa4410d3c0 0x7ffa4419ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:50:45.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.919+0000 7ffa49800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ffa441186b0 con 0x7ffa441084f0 2024-04-25T14:50:45.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.919+0000 7ffa49800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ffa44118530 con 0x7ffa44108e90 2024-04-25T14:50:45.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.919+0000 7ffa49800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffa44118830 con 0x7ffa4410d3c0 2024-04-25T14:50:45.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.919+0000 7ffa43e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa441084f0 0x7ffa441a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:50:45.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.919+0000 7ffa43400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa44108e90 0x7ffa441a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:50:45.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.919+0000 7ffa43e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa441084f0 0x7ffa441a4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55980/0 (socket says 172.21.15.53:55980) 2024-04-25T14:50:45.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.919+0000 7ffa43e00640 1 -- 172.21.15.53:0/2497011596 learned_addr learned my addr 172.21.15.53:0/2497011596 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:50:45.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.919+0000 7ffa48e00640 1 --2- 172.21.15.53:0/2497011596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa4410d3c0 0x7ffa4419ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:50:45.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.920+0000 7ffa43e00640 1 -- 172.21.15.53:0/2497011596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa4410d3c0 msgr2=0x7ffa4419ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:50:45.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.920+0000 7ffa43e00640 1 --2- 172.21.15.53:0/2497011596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa4410d3c0 0x7ffa4419ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:45.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.920+0000 7ffa43e00640 1 -- 172.21.15.53:0/2497011596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa44108e90 msgr2=0x7ffa441a4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:50:45.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.920+0000 7ffa43e00640 1 --2- 172.21.15.53:0/2497011596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa44108e90 0x7ffa441a4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:45.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.920+0000 7ffa43e00640 1 -- 172.21.15.53:0/2497011596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa4419f1b0 con 0x7ffa441084f0 2024-04-25T14:50:45.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.920+0000 7ffa43e00640 1 --2- 172.21.15.53:0/2497011596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa441084f0 0x7ffa441a4880 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ffa34034040 tx=0x7ffa34003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:50:45.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.920+0000 7ffa40c00640 1 -- 172.21.15.53:0/2497011596 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa34041020 con 0x7ffa441084f0 2024-04-25T14:50:45.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.921+0000 7ffa40c00640 1 -- 172.21.15.53:0/2497011596 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa340318f0 con 0x7ffa441084f0 2024-04-25T14:50:45.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.921+0000 7ffa49800640 1 -- 172.21.15.53:0/2497011596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa4419f440 con 0x7ffa441084f0 2024-04-25T14:50:45.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.921+0000 7ffa40c00640 1 -- 172.21.15.53:0/2497011596 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa34036040 con 0x7ffa441084f0 2024-04-25T14:50:45.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.921+0000 7ffa49800640 1 -- 172.21.15.53:0/2497011596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa441ab740 con 0x7ffa441084f0 2024-04-25T14:50:45.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.922+0000 7ffa49800640 1 -- 172.21.15.53:0/2497011596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa440690e0 con 0x7ffa441084f0 2024-04-25T14:50:45.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.924+0000 7ffa40c00640 1 -- 172.21.15.53:0/2497011596 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7ffa34044020 con 0x7ffa441084f0 2024-04-25T14:50:45.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.925+0000 7ffa40c00640 1 --2- 172.21.15.53:0/2497011596 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7ffa280a3d20 0x7ffa280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:50:45.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.926+0000 7ffa40c00640 1 -- 172.21.15.53:0/2497011596 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8587+0+0 (secure 0 0 0) 0x7ffa340f0ff0 con 0x7ffa441084f0 2024-04-25T14:50:45.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.926+0000 7ffa43400640 1 --2- 172.21.15.53:0/2497011596 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7ffa280a3d20 0x7ffa280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:50:45.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.927+0000 7ffa43400640 1 --2- 172.21.15.53:0/2497011596 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7ffa280a3d20 0x7ffa280a61e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ffa44108000 tx=0x7ffa2c009260 comp rx=0 tx=0).ready entity=mgr.15093 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:50:45.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:45.931+0000 7ffa40c00640 1 -- 172.21.15.53:0/2497011596 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffa3404b030 con 0x7ffa441084f0 2024-04-25T14:50:45.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:45 smithi183 ceph-mon[32584]: from='client.25441 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:50:45.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:45 smithi183 ceph-mon[32584]: pgmap v313: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:50:45.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:45 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/1396114528' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:50:45.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:45 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/2998956296' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:50:45.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:45 smithi150 ceph-mon[32504]: from='client.25441 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:50:45.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:45 smithi150 ceph-mon[32504]: pgmap v313: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:50:45.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:45 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/1396114528' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:50:45.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:45 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/2998956296' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:50:46.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:45 smithi053 ceph-mon[27773]: from='client.25441 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:50:46.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:45 smithi053 ceph-mon[27773]: pgmap v313: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:50:46.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:45 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1396114528' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:50:46.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:45 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/2998956296' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:50:46.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.142+0000 7ffa49800640 1 -- 172.21.15.53:0/2497011596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ffa44110900 con 0x7ffa441084f0 2024-04-25T14:50:46.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.142+0000 7ffa40c00640 1 -- 172.21.15.53:0/2497011596 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v101) ==== 100+0+11 (secure 0 0 0) 0x7ffa340bd5f0 con 0x7ffa441084f0 2024-04-25T14:50:46.144 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2024-04-25T14:50:46.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.145+0000 7ffa49800640 1 -- 172.21.15.53:0/2497011596 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7ffa280a3d20 msgr2=0x7ffa280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:50:46.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.145+0000 7ffa49800640 1 --2- 172.21.15.53:0/2497011596 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7ffa280a3d20 0x7ffa280a61e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ffa44108000 tx=0x7ffa2c009260 comp rx=0 tx=0).stop 2024-04-25T14:50:46.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.145+0000 7ffa49800640 1 -- 172.21.15.53:0/2497011596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa441084f0 msgr2=0x7ffa441a4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:50:46.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.145+0000 7ffa49800640 1 --2- 172.21.15.53:0/2497011596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa441084f0 0x7ffa441a4880 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ffa34034040 tx=0x7ffa34003ba0 comp rx=0 tx=0).stop 2024-04-25T14:50:46.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.145+0000 7ffa49800640 1 -- 172.21.15.53:0/2497011596 shutdown_connections 2024-04-25T14:50:46.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.145+0000 7ffa49800640 1 --2- 172.21.15.53:0/2497011596 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7ffa280a3d20 0x7ffa280a61e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:46.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.145+0000 7ffa49800640 1 --2- 172.21.15.53:0/2497011596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa4410d3c0 0x7ffa4419ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:46.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.145+0000 7ffa49800640 1 --2- 172.21.15.53:0/2497011596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa44108e90 0x7ffa441a4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:46.148 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.145+0000 7ffa49800640 1 --2- 172.21.15.53:0/2497011596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa441084f0 0x7ffa441a4880 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:46.148 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.145+0000 7ffa49800640 1 -- 172.21.15.53:0/2497011596 >> 172.21.15.53:0/2497011596 conn(0x7ffa44077d70 msgr2=0x7ffa440783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:50:46.148 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.146+0000 7ffa49800640 1 -- 172.21.15.53:0/2497011596 shutdown_connections 2024-04-25T14:50:46.148 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.146+0000 7ffa49800640 1 -- 172.21.15.53:0/2497011596 wait complete. 2024-04-25T14:50:46.168 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-25T14:50:46.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.370+0000 7f8b46600640 1 -- 172.21.15.53:0/2038367446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b40076480 msgr2=0x7f8b40076860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:50:46.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.370+0000 7f8b46600640 1 --2- 172.21.15.53:0/2038367446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b40076480 0x7f8b40076860 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8b34009930 tx=0x7f8b3402a440 comp rx=0 tx=0).stop 2024-04-25T14:50:46.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.370+0000 7f8b46600640 1 -- 172.21.15.53:0/2038367446 shutdown_connections 2024-04-25T14:50:46.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.370+0000 7f8b46600640 1 --2- 172.21.15.53:0/2038367446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b40112bf0 0x7f8b40114fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:46.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.370+0000 7f8b46600640 1 --2- 172.21.15.53:0/2038367446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b40076da0 0x7f8b401126b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:46.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.370+0000 7f8b46600640 1 --2- 172.21.15.53:0/2038367446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b40076480 0x7f8b40076860 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:46.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.370+0000 7f8b46600640 1 -- 172.21.15.53:0/2038367446 >> 172.21.15.53:0/2038367446 conn(0x7f8b400ffa20 msgr2=0x7f8b40101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:50:46.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.370+0000 7f8b46600640 1 -- 172.21.15.53:0/2038367446 shutdown_connections 2024-04-25T14:50:46.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.370+0000 7f8b46600640 1 -- 172.21.15.53:0/2038367446 wait complete. 2024-04-25T14:50:46.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.372+0000 7f8b46600640 1 Processor -- start 2024-04-25T14:50:46.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.372+0000 7f8b46600640 1 -- start start 2024-04-25T14:50:46.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.372+0000 7f8b46600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b40076480 0x7f8b4019ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:50:46.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.372+0000 7f8b46600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b40076da0 0x7f8b4019f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:50:46.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.372+0000 7f8b46600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b40112bf0 0x7f8b401a24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:50:46.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.372+0000 7f8b46600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8b401185f0 con 0x7f8b40076480 2024-04-25T14:50:46.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.372+0000 7f8b46600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8b40118470 con 0x7f8b40112bf0 2024-04-25T14:50:46.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.372+0000 7f8b46600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8b40118770 con 0x7f8b40076da0 2024-04-25T14:50:46.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.372+0000 7f8b45200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b40076480 0x7f8b4019ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:50:46.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.372+0000 7f8b45200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b40076480 0x7f8b4019ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55994/0 (socket says 172.21.15.53:55994) 2024-04-25T14:50:46.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.373+0000 7f8b45200640 1 -- 172.21.15.53:0/2739086090 learned_addr learned my addr 172.21.15.53:0/2739086090 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:50:46.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.373+0000 7f8b3fe00640 1 --2- 172.21.15.53:0/2739086090 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b40076da0 0x7f8b4019f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:50:46.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.373+0000 7f8b45c00640 1 --2- 172.21.15.53:0/2739086090 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b40112bf0 0x7f8b401a24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:50:46.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.373+0000 7f8b45200640 1 -- 172.21.15.53:0/2739086090 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b40076da0 msgr2=0x7f8b4019f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:50:46.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.373+0000 7f8b45200640 1 --2- 172.21.15.53:0/2739086090 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b40076da0 0x7f8b4019f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:46.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.373+0000 7f8b45200640 1 -- 172.21.15.53:0/2739086090 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b40112bf0 msgr2=0x7f8b401a24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:50:46.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.373+0000 7f8b45200640 1 --2- 172.21.15.53:0/2739086090 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b40112bf0 0x7f8b401a24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:46.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.373+0000 7f8b45200640 1 -- 172.21.15.53:0/2739086090 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b401a2c00 con 0x7f8b40076480 2024-04-25T14:50:46.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.373+0000 7f8b45c00640 1 --2- 172.21.15.53:0/2739086090 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b40112bf0 0x7f8b401a24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:50:46.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.373+0000 7f8b45200640 1 --2- 172.21.15.53:0/2739086090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b40076480 0x7f8b4019ecc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8b34005940 tx=0x7f8b340040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:50:46.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.374+0000 7f8b3d600640 1 -- 172.21.15.53:0/2739086090 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3403f030 con 0x7f8b40076480 2024-04-25T14:50:46.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.374+0000 7f8b46600640 1 -- 172.21.15.53:0/2739086090 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b401a2e90 con 0x7f8b40076480 2024-04-25T14:50:46.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.374+0000 7f8b3d600640 1 -- 172.21.15.53:0/2739086090 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b340044c0 con 0x7f8b40076480 2024-04-25T14:50:46.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.374+0000 7f8b3d600640 1 -- 172.21.15.53:0/2739086090 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b34035070 con 0x7f8b40076480 2024-04-25T14:50:46.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.374+0000 7f8b46600640 1 -- 172.21.15.53:0/2739086090 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b401ab860 con 0x7f8b40076480 2024-04-25T14:50:46.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.375+0000 7f8b46600640 1 -- 172.21.15.53:0/2739086090 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b400690e0 con 0x7f8b40076480 2024-04-25T14:50:46.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.377+0000 7f8b3d600640 1 -- 172.21.15.53:0/2739086090 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f8b34042030 con 0x7f8b40076480 2024-04-25T14:50:46.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.378+0000 7f8b3d600640 1 --2- 172.21.15.53:0/2739086090 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f8b280a3d20 0x7f8b280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:50:46.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.379+0000 7f8b3d600640 1 -- 172.21.15.53:0/2739086090 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8587+0+0 (secure 0 0 0) 0x7f8b340f08f0 con 0x7f8b40076480 2024-04-25T14:50:46.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.379+0000 7f8b3fe00640 1 --2- 172.21.15.53:0/2739086090 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f8b280a3d20 0x7f8b280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:50:46.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.379+0000 7f8b3fe00640 1 --2- 172.21.15.53:0/2739086090 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f8b280a3d20 0x7f8b280a61e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c0095f0 tx=0x7f8b2c005f90 comp rx=0 tx=0).ready entity=mgr.15093 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:50:46.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.383+0000 7f8b3d600640 1 -- 172.21.15.53:0/2739086090 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8b340bcef0 con 0x7f8b40076480 2024-04-25T14:50:46.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.642+0000 7f8b46600640 1 -- 172.21.15.53:0/2739086090 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f8b4010dd80 con 0x7f8b40076480 2024-04-25T14:50:46.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.675+0000 7f8b3d600640 1 -- 172.21.15.53:0/2739086090 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v27) ==== 81+0+746058 (secure 0 0 0) 0x7f8b340bc7f0 con 0x7f8b40076480 2024-04-25T14:50:46.689 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:50:46.689 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-04-25T14:50:46.689 INFO:teuthology.orchestra.run.smithi053.stdout: "epoch": 27, 2024-04-25T14:50:46.689 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.689 INFO:teuthology.orchestra.run.smithi053.stdout: "active_gid": 15093, 2024-04-25T14:50:46.689 INFO:teuthology.orchestra.run.smithi053.stdout: "active_name": "x", 2024-04-25T14:50:46.690 INFO:teuthology.orchestra.run.smithi053.stdout: "active_addrs": { 2024-04-25T14:50:46.690 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-04-25T14:50:46.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:46.690 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-04-25T14:50:46.690 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6800", 2024-04-25T14:50:46.690 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 802683714 2024-04-25T14:50:46.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:46.690 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-04-25T14:50:46.691 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6801", 2024-04-25T14:50:46.691 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 802683714 2024-04-25T14:50:46.691 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:46.691 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-04-25T14:50:46.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.691 INFO:teuthology.orchestra.run.smithi053.stdout: "active_addr": "172.21.15.53:6801/802683714", 2024-04-25T14:50:46.691 INFO:teuthology.orchestra.run.smithi053.stdout: "active_change": "2024-04-25T14:40:40.169422+0000", 2024-04-25T14:50:46.691 INFO:teuthology.orchestra.run.smithi053.stdout: "active_mgr_features": 4540701547738038271, 2024-04-25T14:50:46.691 INFO:teuthology.orchestra.run.smithi053.stdout: "available": true, 2024-04-25T14:50:46.692 INFO:teuthology.orchestra.run.smithi053.stdout: "standbys": [ 2024-04-25T14:50:46.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:46.692 INFO:teuthology.orchestra.run.smithi053.stdout: "gid": 25222, 2024-04-25T14:50:46.692 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "y", 2024-04-25T14:50:46.692 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_features": 4540701547738038271, 2024-04-25T14:50:46.692 INFO:teuthology.orchestra.run.smithi053.stdout: "available_modules": [ 2024-04-25T14:50:46.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:46.692 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "alerts", 2024-04-25T14:50:46.692 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:46.693 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:46.693 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:46.693 INFO:teuthology.orchestra.run.smithi053.stdout: "interval": { 2024-04-25T14:50:46.693 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "interval", 2024-04-25T14:50:46.693 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:46.693 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.693 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.693 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "60", 2024-04-25T14:50:46.694 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.694 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.694 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.694 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T14:50:46.694 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.694 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.694 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.694 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:46.695 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:46.695 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.695 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.695 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.695 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:46.695 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.695 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.695 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:46.695 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:46.696 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:46.696 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:46.696 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:46.696 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:46.696 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:46.696 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:46.696 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.696 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.696 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.697 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.697 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:46.697 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:46.697 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.697 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.697 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.697 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:46.697 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.698 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.698 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.698 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.698 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.698 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.698 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.698 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:46.699 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:46.699 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.699 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.699 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.699 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:46.699 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.699 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.699 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:46.699 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:46.700 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:46.700 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:46.700 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:46.700 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:46.700 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:46.700 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:46.700 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.700 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.700 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.701 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.701 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:46.701 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:46.701 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.701 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.701 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.701 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:46.701 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.701 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.702 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.702 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.702 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.702 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.702 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.702 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_destination": { 2024-04-25T14:50:46.702 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_destination", 2024-04-25T14:50:46.703 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.703 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.703 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.703 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:46.703 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.703 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.703 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.703 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Email address to send alerts to", 2024-04-25T14:50:46.703 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.704 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.704 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.704 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_from_name": { 2024-04-25T14:50:46.704 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_from_name", 2024-04-25T14:50:46.704 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.704 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.704 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.704 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "Ceph", 2024-04-25T14:50:46.704 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.705 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.705 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.705 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Email From: name", 2024-04-25T14:50:46.705 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.705 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.705 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.705 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_host": { 2024-04-25T14:50:46.705 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_host", 2024-04-25T14:50:46.706 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.706 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.706 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.706 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:46.706 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.706 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.706 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.706 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "SMTP server", 2024-04-25T14:50:46.707 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.707 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.707 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.707 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_password": { 2024-04-25T14:50:46.707 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_password", 2024-04-25T14:50:46.707 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.707 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.707 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.708 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:46.708 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.708 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.708 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.708 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Password to authenticate with", 2024-04-25T14:50:46.708 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.708 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.708 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.708 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_port": { 2024-04-25T14:50:46.709 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_port", 2024-04-25T14:50:46.709 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:46.709 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.709 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.709 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "465", 2024-04-25T14:50:46.709 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.709 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.709 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.709 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "SMTP port", 2024-04-25T14:50:46.709 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.710 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.710 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.710 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_sender": { 2024-04-25T14:50:46.710 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_sender", 2024-04-25T14:50:46.710 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.710 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.710 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.711 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:46.711 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.711 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.711 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.711 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "SMTP envelope sender", 2024-04-25T14:50:46.711 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.711 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.711 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.712 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_ssl": { 2024-04-25T14:50:46.712 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_ssl", 2024-04-25T14:50:46.712 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.712 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.712 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.712 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:46.712 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.712 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.713 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.713 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T14:50:46.713 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.713 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.713 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.713 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_user": { 2024-04-25T14:50:46.713 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_user", 2024-04-25T14:50:46.713 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.714 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.714 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.714 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:46.714 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.714 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.714 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.714 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "User to authenticate as", 2024-04-25T14:50:46.714 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.714 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.715 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.715 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:46.715 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:46.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:46.715 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "balancer", 2024-04-25T14:50:46.715 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:46.715 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:46.715 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:46.716 INFO:teuthology.orchestra.run.smithi053.stdout: "active": { 2024-04-25T14:50:46.716 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "active", 2024-04-25T14:50:46.716 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.716 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.716 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.716 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:46.716 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.716 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.716 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.717 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T14:50:46.717 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.717 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.717 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.717 INFO:teuthology.orchestra.run.smithi053.stdout: "begin_time": { 2024-04-25T14:50:46.717 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "begin_time", 2024-04-25T14:50:46.717 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.717 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.718 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.718 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0000", 2024-04-25T14:50:46.718 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.718 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.718 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.718 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T14:50:46.718 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T14:50:46.718 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.719 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.719 INFO:teuthology.orchestra.run.smithi053.stdout: "begin_weekday": { 2024-04-25T14:50:46.719 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "begin_weekday", 2024-04-25T14:50:46.719 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "uint", 2024-04-25T14:50:46.719 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.719 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.719 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0", 2024-04-25T14:50:46.719 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "0", 2024-04-25T14:50:46.720 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "6", 2024-04-25T14:50:46.720 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.720 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T14:50:46.720 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T14:50:46.720 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.720 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.720 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_compat_max_iterations": { 2024-04-25T14:50:46.721 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "crush_compat_max_iterations", 2024-04-25T14:50:46.721 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "uint", 2024-04-25T14:50:46.721 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.721 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.721 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "25", 2024-04-25T14:50:46.721 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "1", 2024-04-25T14:50:46.721 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "250", 2024-04-25T14:50:46.721 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.721 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T14:50:46.722 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.722 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.722 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.722 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_compat_metrics": { 2024-04-25T14:50:46.722 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "crush_compat_metrics", 2024-04-25T14:50:46.722 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.722 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.723 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.723 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T14:50:46.723 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.723 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.723 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.723 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T14:50:46.723 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T14:50:46.723 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.724 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.724 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_compat_step": { 2024-04-25T14:50:46.724 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "crush_compat_step", 2024-04-25T14:50:46.724 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2024-04-25T14:50:46.724 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.724 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.724 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0.5", 2024-04-25T14:50:46.724 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "0.001", 2024-04-25T14:50:46.725 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "0.999", 2024-04-25T14:50:46.725 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.725 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "aggressiveness of optimization", 2024-04-25T14:50:46.725 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T14:50:46.725 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.725 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.725 INFO:teuthology.orchestra.run.smithi053.stdout: "end_time": { 2024-04-25T14:50:46.725 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "end_time", 2024-04-25T14:50:46.726 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.726 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.726 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.726 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "2359", 2024-04-25T14:50:46.726 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.726 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.726 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.726 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T14:50:46.726 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T14:50:46.727 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.727 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.727 INFO:teuthology.orchestra.run.smithi053.stdout: "end_weekday": { 2024-04-25T14:50:46.727 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "end_weekday", 2024-04-25T14:50:46.727 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "uint", 2024-04-25T14:50:46.727 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.727 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.728 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0", 2024-04-25T14:50:46.728 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "0", 2024-04-25T14:50:46.728 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "6", 2024-04-25T14:50:46.728 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.728 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T14:50:46.728 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T14:50:46.728 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.728 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.729 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:46.729 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:46.729 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.729 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.729 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.729 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:46.729 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.729 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.729 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:46.730 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:46.730 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:46.730 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:46.730 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:46.730 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:46.730 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:46.730 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:46.730 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.731 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.731 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.731 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.731 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:46.731 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:46.731 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.731 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.731 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.732 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:46.732 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.732 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.732 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.732 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.732 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.732 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.733 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.733 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:46.733 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:46.733 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.733 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.733 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.733 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:46.734 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.734 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.734 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:46.734 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:46.734 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:46.734 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:46.734 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:46.734 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:46.735 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:46.735 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:46.735 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.735 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.735 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.735 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.735 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:46.735 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:46.736 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.736 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.736 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.736 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:46.736 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.736 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.736 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.736 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.736 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.737 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.737 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.737 INFO:teuthology.orchestra.run.smithi053.stdout: "min_score": { 2024-04-25T14:50:46.737 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "min_score", 2024-04-25T14:50:46.737 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2024-04-25T14:50:46.737 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.737 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.738 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0", 2024-04-25T14:50:46.738 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.738 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.738 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.738 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T14:50:46.738 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.738 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.738 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.739 INFO:teuthology.orchestra.run.smithi053.stdout: "mode": { 2024-04-25T14:50:46.739 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "mode", 2024-04-25T14:50:46.739 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.739 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.739 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.739 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "upmap", 2024-04-25T14:50:46.739 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.739 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.739 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:46.740 INFO:teuthology.orchestra.run.smithi053.stdout: "crush-compat", 2024-04-25T14:50:46.740 INFO:teuthology.orchestra.run.smithi053.stdout: "none", 2024-04-25T14:50:46.740 INFO:teuthology.orchestra.run.smithi053.stdout: "read", 2024-04-25T14:50:46.740 INFO:teuthology.orchestra.run.smithi053.stdout: "upmap", 2024-04-25T14:50:46.740 INFO:teuthology.orchestra.run.smithi053.stdout: "upmap-read" 2024-04-25T14:50:46.740 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:46.740 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Balancer mode", 2024-04-25T14:50:46.740 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.740 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.740 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.741 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_ids": { 2024-04-25T14:50:46.741 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "pool_ids", 2024-04-25T14:50:46.741 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.741 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.741 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.741 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:46.741 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.741 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.742 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.742 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T14:50:46.742 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.742 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.742 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.742 INFO:teuthology.orchestra.run.smithi053.stdout: "sleep_interval": { 2024-04-25T14:50:46.742 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "sleep_interval", 2024-04-25T14:50:46.742 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:46.743 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.743 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.743 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "60", 2024-04-25T14:50:46.743 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.743 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.743 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.743 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T14:50:46.743 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.743 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.744 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.744 INFO:teuthology.orchestra.run.smithi053.stdout: "upmap_max_deviation": { 2024-04-25T14:50:46.744 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "upmap_max_deviation", 2024-04-25T14:50:46.744 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:46.744 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.744 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.744 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "5", 2024-04-25T14:50:46.744 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "1", 2024-04-25T14:50:46.745 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.745 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.745 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T14:50:46.745 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T14:50:46.745 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.745 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.745 INFO:teuthology.orchestra.run.smithi053.stdout: "upmap_max_optimizations": { 2024-04-25T14:50:46.745 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "upmap_max_optimizations", 2024-04-25T14:50:46.745 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "uint", 2024-04-25T14:50:46.746 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.746 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.746 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2024-04-25T14:50:46.746 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.746 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.746 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.746 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T14:50:46.746 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.747 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.747 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.747 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:46.747 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:46.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:46.747 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "cephadm", 2024-04-25T14:50:46.747 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:46.747 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:46.748 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:46.748 INFO:teuthology.orchestra.run.smithi053.stdout: "agent_down_multiplier": { 2024-04-25T14:50:46.748 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "agent_down_multiplier", 2024-04-25T14:50:46.748 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2024-04-25T14:50:46.748 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.748 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.748 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "3.0", 2024-04-25T14:50:46.748 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.748 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.749 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.749 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T14:50:46.749 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.749 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.749 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.749 INFO:teuthology.orchestra.run.smithi053.stdout: "agent_refresh_rate": { 2024-04-25T14:50:46.749 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "agent_refresh_rate", 2024-04-25T14:50:46.749 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:46.750 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.750 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.750 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "20", 2024-04-25T14:50:46.750 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.750 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.750 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.750 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T14:50:46.750 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.750 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.751 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.751 INFO:teuthology.orchestra.run.smithi053.stdout: "agent_starting_port": { 2024-04-25T14:50:46.751 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "agent_starting_port", 2024-04-25T14:50:46.751 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:46.751 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.751 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.751 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "4721", 2024-04-25T14:50:46.751 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.752 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.752 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.752 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T14:50:46.752 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.752 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.752 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.752 INFO:teuthology.orchestra.run.smithi053.stdout: "allow_ptrace": { 2024-04-25T14:50:46.752 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "allow_ptrace", 2024-04-25T14:50:46.753 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.753 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.753 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.753 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:46.753 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.753 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.753 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.753 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T14:50:46.753 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T14:50:46.754 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.754 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.754 INFO:teuthology.orchestra.run.smithi053.stdout: "autotune_interval": { 2024-04-25T14:50:46.754 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "autotune_interval", 2024-04-25T14:50:46.754 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:46.754 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.754 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.755 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "600", 2024-04-25T14:50:46.755 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.755 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.755 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.755 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T14:50:46.755 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.755 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.755 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.755 INFO:teuthology.orchestra.run.smithi053.stdout: "autotune_memory_target_ratio": { 2024-04-25T14:50:46.756 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T14:50:46.756 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2024-04-25T14:50:46.756 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.756 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.756 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0.7", 2024-04-25T14:50:46.756 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.756 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.756 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.757 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T14:50:46.757 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.757 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.757 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.757 INFO:teuthology.orchestra.run.smithi053.stdout: "cephadm_log_destination": { 2024-04-25T14:50:46.757 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "cephadm_log_destination", 2024-04-25T14:50:46.757 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.757 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.757 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.758 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:46.758 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.758 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.758 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:46.758 INFO:teuthology.orchestra.run.smithi053.stdout: "file", 2024-04-25T14:50:46.758 INFO:teuthology.orchestra.run.smithi053.stdout: "file,syslog", 2024-04-25T14:50:46.758 INFO:teuthology.orchestra.run.smithi053.stdout: "syslog" 2024-04-25T14:50:46.758 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:46.759 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T14:50:46.759 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.759 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.759 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.759 INFO:teuthology.orchestra.run.smithi053.stdout: "cgroups_split": { 2024-04-25T14:50:46.759 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "cgroups_split", 2024-04-25T14:50:46.759 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.759 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.759 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.760 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:46.760 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.760 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.760 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.760 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T14:50:46.760 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.760 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.760 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.760 INFO:teuthology.orchestra.run.smithi053.stdout: "config_checks_enabled": { 2024-04-25T14:50:46.761 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "config_checks_enabled", 2024-04-25T14:50:46.761 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.761 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.761 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.761 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:46.761 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.761 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.761 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.762 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T14:50:46.762 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.762 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.762 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.762 INFO:teuthology.orchestra.run.smithi053.stdout: "config_dashboard": { 2024-04-25T14:50:46.762 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "config_dashboard", 2024-04-25T14:50:46.762 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.762 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.763 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.763 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:46.763 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.763 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.763 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.763 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T14:50:46.763 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.763 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.763 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.764 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_alertmanager": { 2024-04-25T14:50:46.764 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_alertmanager", 2024-04-25T14:50:46.764 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.764 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.764 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.764 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T14:50:46.764 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.764 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.764 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.764 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Prometheus container image", 2024-04-25T14:50:46.765 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.765 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.765 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.765 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_base": { 2024-04-25T14:50:46.765 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_base", 2024-04-25T14:50:46.765 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.765 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.765 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.766 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T14:50:46.766 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.766 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.766 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.766 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Container image name, without the tag", 2024-04-25T14:50:46.766 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.766 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.766 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.767 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_elasticsearch": { 2024-04-25T14:50:46.767 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_elasticsearch", 2024-04-25T14:50:46.767 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.767 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.767 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.767 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T14:50:46.767 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.767 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.767 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.767 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "elasticsearch container image", 2024-04-25T14:50:46.768 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.768 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.768 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.768 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_grafana": { 2024-04-25T14:50:46.768 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_grafana", 2024-04-25T14:50:46.768 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.768 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.768 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.768 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T14:50:46.769 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.769 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.769 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.769 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Prometheus container image", 2024-04-25T14:50:46.769 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.769 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.769 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.769 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_haproxy": { 2024-04-25T14:50:46.769 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_haproxy", 2024-04-25T14:50:46.770 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.770 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.770 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.770 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T14:50:46.770 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.770 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.770 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.770 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "HAproxy container image", 2024-04-25T14:50:46.771 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.771 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.771 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.771 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_jaeger_agent": { 2024-04-25T14:50:46.771 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_jaeger_agent", 2024-04-25T14:50:46.771 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.771 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.771 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.771 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T14:50:46.772 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.772 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.772 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.772 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Jaeger agent container image", 2024-04-25T14:50:46.772 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.772 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.772 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.772 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_jaeger_collector": { 2024-04-25T14:50:46.773 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_jaeger_collector", 2024-04-25T14:50:46.773 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.773 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.773 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.773 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T14:50:46.773 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.773 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.773 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.773 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Jaeger collector container image", 2024-04-25T14:50:46.773 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.774 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.774 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.774 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_jaeger_query": { 2024-04-25T14:50:46.774 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_jaeger_query", 2024-04-25T14:50:46.774 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.774 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.774 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.775 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T14:50:46.775 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.775 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.775 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.775 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Jaeger query container image", 2024-04-25T14:50:46.775 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.775 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.775 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.776 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_keepalived": { 2024-04-25T14:50:46.776 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_keepalived", 2024-04-25T14:50:46.776 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.776 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.776 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.776 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T14:50:46.776 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.776 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.777 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.777 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Keepalived container image", 2024-04-25T14:50:46.777 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.777 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.777 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.777 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_loki": { 2024-04-25T14:50:46.777 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_loki", 2024-04-25T14:50:46.777 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.778 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.778 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.778 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T14:50:46.778 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.778 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.778 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.778 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Loki container image", 2024-04-25T14:50:46.778 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.779 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.779 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.779 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_node_exporter": { 2024-04-25T14:50:46.779 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_node_exporter", 2024-04-25T14:50:46.779 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.779 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.779 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.779 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T14:50:46.780 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.780 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.780 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.780 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Prometheus container image", 2024-04-25T14:50:46.780 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.780 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.780 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.780 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_nvmeof": { 2024-04-25T14:50:46.780 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_nvmeof", 2024-04-25T14:50:46.781 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.781 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.781 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.781 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T14:50:46.781 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.781 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.781 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.781 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Nvme-of container image", 2024-04-25T14:50:46.782 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.782 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.782 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.782 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_prometheus": { 2024-04-25T14:50:46.782 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_prometheus", 2024-04-25T14:50:46.782 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.782 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.782 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.783 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T14:50:46.783 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.783 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.783 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.783 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Prometheus container image", 2024-04-25T14:50:46.783 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.783 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.783 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.784 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_promtail": { 2024-04-25T14:50:46.784 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_promtail", 2024-04-25T14:50:46.784 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.784 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.784 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.784 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T14:50:46.784 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.784 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.784 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.784 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Promtail container image", 2024-04-25T14:50:46.785 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.785 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.785 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.785 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_samba": { 2024-04-25T14:50:46.785 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_samba", 2024-04-25T14:50:46.785 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.785 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.785 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.785 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T14:50:46.786 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.786 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.786 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.786 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Samba/SMB container image", 2024-04-25T14:50:46.786 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.786 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.786 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.786 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_snmp_gateway": { 2024-04-25T14:50:46.787 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_snmp_gateway", 2024-04-25T14:50:46.787 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.787 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.787 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.787 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T14:50:46.787 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.787 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.787 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.787 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "SNMP Gateway container image", 2024-04-25T14:50:46.788 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.788 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.788 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.788 INFO:teuthology.orchestra.run.smithi053.stdout: "container_init": { 2024-04-25T14:50:46.788 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_init", 2024-04-25T14:50:46.788 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.788 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.788 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.788 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:46.789 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.789 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.789 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.789 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T14:50:46.789 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.789 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.789 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.789 INFO:teuthology.orchestra.run.smithi053.stdout: "daemon_cache_timeout": { 2024-04-25T14:50:46.790 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "daemon_cache_timeout", 2024-04-25T14:50:46.790 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:46.790 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.790 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.790 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "600", 2024-04-25T14:50:46.790 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.790 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.790 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.790 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T14:50:46.791 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.791 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.791 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.791 INFO:teuthology.orchestra.run.smithi053.stdout: "default_cephadm_command_timeout": { 2024-04-25T14:50:46.791 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T14:50:46.791 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:46.791 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.791 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.791 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "900", 2024-04-25T14:50:46.792 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.792 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.792 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.792 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T14:50:46.792 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.792 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.792 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.792 INFO:teuthology.orchestra.run.smithi053.stdout: "default_registry": { 2024-04-25T14:50:46.793 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "default_registry", 2024-04-25T14:50:46.793 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.793 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.793 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.793 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "docker.io", 2024-04-25T14:50:46.793 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.793 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.793 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.793 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T14:50:46.794 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.794 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.794 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.794 INFO:teuthology.orchestra.run.smithi053.stdout: "device_cache_timeout": { 2024-04-25T14:50:46.794 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "device_cache_timeout", 2024-04-25T14:50:46.794 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:46.794 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.795 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.795 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "1800", 2024-04-25T14:50:46.795 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.795 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.795 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.795 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "seconds to cache device inventory", 2024-04-25T14:50:46.795 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.795 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.795 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.796 INFO:teuthology.orchestra.run.smithi053.stdout: "device_enhanced_scan": { 2024-04-25T14:50:46.796 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "device_enhanced_scan", 2024-04-25T14:50:46.796 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.796 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.796 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.796 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:46.796 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.796 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.796 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.797 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T14:50:46.797 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.797 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.797 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.797 INFO:teuthology.orchestra.run.smithi053.stdout: "facts_cache_timeout": { 2024-04-25T14:50:46.797 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "facts_cache_timeout", 2024-04-25T14:50:46.797 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:46.797 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.798 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.798 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "60", 2024-04-25T14:50:46.798 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.798 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.798 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.798 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "seconds to cache host facts data", 2024-04-25T14:50:46.798 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.798 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.798 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.799 INFO:teuthology.orchestra.run.smithi053.stdout: "grafana_dashboards_path": { 2024-04-25T14:50:46.799 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "grafana_dashboards_path", 2024-04-25T14:50:46.799 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.799 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.799 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.799 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T14:50:46.799 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.799 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.800 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.800 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T14:50:46.800 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.800 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.800 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.800 INFO:teuthology.orchestra.run.smithi053.stdout: "host_check_interval": { 2024-04-25T14:50:46.800 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "host_check_interval", 2024-04-25T14:50:46.800 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:46.800 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.801 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.801 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "600", 2024-04-25T14:50:46.801 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.801 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.801 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.801 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how frequently to perform a host check", 2024-04-25T14:50:46.801 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.801 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.801 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.802 INFO:teuthology.orchestra.run.smithi053.stdout: "hw_monitoring": { 2024-04-25T14:50:46.802 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "hw_monitoring", 2024-04-25T14:50:46.802 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.802 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.802 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.802 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:46.802 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.802 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.802 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.803 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T14:50:46.803 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.803 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.803 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.803 INFO:teuthology.orchestra.run.smithi053.stdout: "inventory_list_all": { 2024-04-25T14:50:46.803 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "inventory_list_all", 2024-04-25T14:50:46.803 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.803 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.803 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.804 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:46.804 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.804 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.804 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.804 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T14:50:46.804 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.804 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.804 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.805 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:46.805 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:46.805 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.805 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.805 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.805 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:46.805 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.805 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.805 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:46.805 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:46.806 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:46.806 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:46.806 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:46.806 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:46.806 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:46.806 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:46.806 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.806 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.806 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.807 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.807 INFO:teuthology.orchestra.run.smithi053.stdout: "log_refresh_metadata": { 2024-04-25T14:50:46.807 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_refresh_metadata", 2024-04-25T14:50:46.807 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.807 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.807 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.807 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:46.807 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.808 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.808 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.808 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T14:50:46.808 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.808 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.808 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.808 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:46.808 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:46.808 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.809 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.809 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.809 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:46.809 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.809 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.809 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.809 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T14:50:46.809 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.809 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.809 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.810 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:46.810 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:46.810 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.810 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.810 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.810 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:46.810 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.810 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.811 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:46.811 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:46.811 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:46.811 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:46.811 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:46.811 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:46.811 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:46.811 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:46.811 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.812 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.812 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.812 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.812 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:46.812 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:46.812 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.812 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.812 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.812 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:46.813 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.813 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.813 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.813 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.813 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.813 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.813 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.813 INFO:teuthology.orchestra.run.smithi053.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T14:50:46.813 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T14:50:46.814 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.814 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.814 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.814 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:46.814 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.814 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.814 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.814 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T14:50:46.814 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.815 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.815 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.815 INFO:teuthology.orchestra.run.smithi053.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T14:50:46.815 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T14:50:46.815 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.815 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.815 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.815 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "*", 2024-04-25T14:50:46.815 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.816 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.816 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.816 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T14:50:46.816 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.816 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.816 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.816 INFO:teuthology.orchestra.run.smithi053.stdout: "max_count_per_host": { 2024-04-25T14:50:46.816 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "max_count_per_host", 2024-04-25T14:50:46.817 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:46.817 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.817 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.817 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2024-04-25T14:50:46.817 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.817 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.817 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.817 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "max number of daemons per service per host", 2024-04-25T14:50:46.817 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.817 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.818 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.818 INFO:teuthology.orchestra.run.smithi053.stdout: "max_osd_draining_count": { 2024-04-25T14:50:46.818 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "max_osd_draining_count", 2024-04-25T14:50:46.818 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:46.818 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.818 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.818 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2024-04-25T14:50:46.818 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.818 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.819 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.819 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T14:50:46.819 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.819 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.819 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.819 INFO:teuthology.orchestra.run.smithi053.stdout: "migration_current": { 2024-04-25T14:50:46.819 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "migration_current", 2024-04-25T14:50:46.819 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:46.820 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.820 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.820 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:46.820 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.820 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.820 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.820 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "internal - do not modify", 2024-04-25T14:50:46.820 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.820 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.820 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.821 INFO:teuthology.orchestra.run.smithi053.stdout: "mode": { 2024-04-25T14:50:46.821 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "mode", 2024-04-25T14:50:46.821 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.821 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.821 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.821 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "root", 2024-04-25T14:50:46.821 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.822 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.822 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:46.822 INFO:teuthology.orchestra.run.smithi053.stdout: "cephadm-package", 2024-04-25T14:50:46.822 INFO:teuthology.orchestra.run.smithi053.stdout: "root" 2024-04-25T14:50:46.822 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:46.822 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T14:50:46.822 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.822 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.822 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.823 INFO:teuthology.orchestra.run.smithi053.stdout: "oob_default_addr": { 2024-04-25T14:50:46.823 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "oob_default_addr", 2024-04-25T14:50:46.823 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.823 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.823 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.823 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "169.254.1.1", 2024-04-25T14:50:46.823 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.823 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.824 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.824 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T14:50:46.824 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.824 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.824 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.824 INFO:teuthology.orchestra.run.smithi053.stdout: "prometheus_alerts_path": { 2024-04-25T14:50:46.824 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "prometheus_alerts_path", 2024-04-25T14:50:46.824 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.824 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.825 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.825 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T14:50:46.825 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.825 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.825 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.825 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T14:50:46.825 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.825 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.825 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.826 INFO:teuthology.orchestra.run.smithi053.stdout: "registry_insecure": { 2024-04-25T14:50:46.826 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "registry_insecure", 2024-04-25T14:50:46.826 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.826 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.826 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.826 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:46.827 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.827 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.827 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.827 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T14:50:46.827 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.827 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.827 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.827 INFO:teuthology.orchestra.run.smithi053.stdout: "registry_password": { 2024-04-25T14:50:46.828 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "registry_password", 2024-04-25T14:50:46.828 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.828 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.828 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.828 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:46.828 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.828 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.828 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.828 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T14:50:46.829 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.829 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.829 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.829 INFO:teuthology.orchestra.run.smithi053.stdout: "registry_url": { 2024-04-25T14:50:46.829 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "registry_url", 2024-04-25T14:50:46.829 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.829 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.829 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.829 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:46.830 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.830 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.830 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.830 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T14:50:46.830 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.830 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.830 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.831 INFO:teuthology.orchestra.run.smithi053.stdout: "registry_username": { 2024-04-25T14:50:46.831 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "registry_username", 2024-04-25T14:50:46.831 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.831 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.831 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.831 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:46.831 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.831 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.831 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.832 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T14:50:46.832 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.832 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.832 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.832 INFO:teuthology.orchestra.run.smithi053.stdout: "secure_monitoring_stack": { 2024-04-25T14:50:46.832 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "secure_monitoring_stack", 2024-04-25T14:50:46.832 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.832 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.832 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.833 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:46.833 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.833 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.833 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.833 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T14:50:46.833 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.833 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.833 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.834 INFO:teuthology.orchestra.run.smithi053.stdout: "service_discovery_port": { 2024-04-25T14:50:46.834 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "service_discovery_port", 2024-04-25T14:50:46.834 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:46.834 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.834 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.834 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "8765", 2024-04-25T14:50:46.834 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.834 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.835 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.835 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "cephadm service discovery port", 2024-04-25T14:50:46.835 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.835 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.835 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.835 INFO:teuthology.orchestra.run.smithi053.stdout: "ssh_config_file": { 2024-04-25T14:50:46.835 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ssh_config_file", 2024-04-25T14:50:46.835 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.836 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.836 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.836 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:46.836 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.836 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.836 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.836 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T14:50:46.836 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.836 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.837 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.837 INFO:teuthology.orchestra.run.smithi053.stdout: "use_agent": { 2024-04-25T14:50:46.837 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "use_agent", 2024-04-25T14:50:46.837 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.837 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.837 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.837 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:46.837 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.837 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.838 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.838 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T14:50:46.838 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.838 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.838 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.838 INFO:teuthology.orchestra.run.smithi053.stdout: "use_repo_digest": { 2024-04-25T14:50:46.838 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "use_repo_digest", 2024-04-25T14:50:46.838 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.839 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.839 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.839 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:46.839 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.839 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.839 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.839 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T14:50:46.839 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.839 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.839 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.840 INFO:teuthology.orchestra.run.smithi053.stdout: "warn_on_failed_host_check": { 2024-04-25T14:50:46.840 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "warn_on_failed_host_check", 2024-04-25T14:50:46.840 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.840 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.840 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.840 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:46.840 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.840 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.840 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.841 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T14:50:46.841 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.841 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.841 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.841 INFO:teuthology.orchestra.run.smithi053.stdout: "warn_on_stray_daemons": { 2024-04-25T14:50:46.841 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "warn_on_stray_daemons", 2024-04-25T14:50:46.841 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.841 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.841 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.842 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:46.842 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.842 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.842 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.842 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T14:50:46.842 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.842 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.842 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.843 INFO:teuthology.orchestra.run.smithi053.stdout: "warn_on_stray_hosts": { 2024-04-25T14:50:46.843 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "warn_on_stray_hosts", 2024-04-25T14:50:46.843 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.843 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.843 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.843 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:46.843 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.843 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.843 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.843 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T14:50:46.844 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.844 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.844 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.844 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:46.844 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:46.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:46.844 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "crash", 2024-04-25T14:50:46.844 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:46.845 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:46.845 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:46.845 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:46.845 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:46.845 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.845 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.845 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.845 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:46.845 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.845 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.846 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:46.846 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:46.846 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:46.846 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:46.846 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:46.846 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:46.846 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:46.846 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:46.846 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.847 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.847 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.847 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.847 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:46.847 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:46.847 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.847 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.847 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.847 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:46.848 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.848 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.848 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.848 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.848 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.848 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.848 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.848 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:46.848 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:46.849 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.849 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.849 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.849 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:46.849 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.849 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.849 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:46.849 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:46.849 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:46.849 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:46.850 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:46.850 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:46.850 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:46.850 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:46.850 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.850 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.850 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.850 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.851 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:46.851 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:46.851 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.851 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.851 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.851 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:46.851 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.851 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.851 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.851 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.852 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.852 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.852 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.852 INFO:teuthology.orchestra.run.smithi053.stdout: "retain_interval": { 2024-04-25T14:50:46.852 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "retain_interval", 2024-04-25T14:50:46.852 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:46.852 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.852 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.852 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "31536000", 2024-04-25T14:50:46.853 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.853 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.853 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.853 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T14:50:46.853 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.853 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.853 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.853 INFO:teuthology.orchestra.run.smithi053.stdout: "warn_recent_interval": { 2024-04-25T14:50:46.853 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "warn_recent_interval", 2024-04-25T14:50:46.854 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:46.854 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.854 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.854 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "1209600", 2024-04-25T14:50:46.854 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.854 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.854 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.854 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T14:50:46.854 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.855 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.855 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.855 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:46.855 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:46.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:46.855 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "dashboard", 2024-04-25T14:50:46.855 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:46.855 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:46.855 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:46.856 INFO:teuthology.orchestra.run.smithi053.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T14:50:46.856 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T14:50:46.856 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:46.856 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.856 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.856 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2024-04-25T14:50:46.856 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.856 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.856 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.856 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.857 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.857 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.857 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.857 INFO:teuthology.orchestra.run.smithi053.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T14:50:46.857 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T14:50:46.857 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.857 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.857 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.858 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:46.858 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.858 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.858 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.858 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.858 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.858 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.858 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.859 INFO:teuthology.orchestra.run.smithi053.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T14:50:46.859 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T14:50:46.859 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.859 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.859 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.859 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:46.859 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.859 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.859 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.860 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.860 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.860 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.860 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.860 INFO:teuthology.orchestra.run.smithi053.stdout: "AUDIT_API_ENABLED": { 2024-04-25T14:50:46.860 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T14:50:46.860 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.860 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.861 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.861 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:46.861 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.861 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.861 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.861 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.861 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.861 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.861 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.862 INFO:teuthology.orchestra.run.smithi053.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T14:50:46.862 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T14:50:46.862 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.862 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.862 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.862 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:46.862 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.862 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.863 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.863 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.863 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.863 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.863 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.863 INFO:teuthology.orchestra.run.smithi053.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T14:50:46.863 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T14:50:46.863 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.864 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.864 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.864 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:46.864 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.864 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.864 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.864 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.864 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.865 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.865 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.865 INFO:teuthology.orchestra.run.smithi053.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T14:50:46.866 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T14:50:46.866 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.866 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.866 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.866 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:46.866 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.866 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.866 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.866 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.867 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.867 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.867 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.867 INFO:teuthology.orchestra.run.smithi053.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T14:50:46.867 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T14:50:46.867 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.867 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.868 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.868 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:46.868 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.868 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.868 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.868 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.868 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.868 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.868 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.869 INFO:teuthology.orchestra.run.smithi053.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T14:50:46.869 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T14:50:46.869 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.869 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.869 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.869 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:46.869 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.869 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.870 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.870 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.870 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.870 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.870 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.870 INFO:teuthology.orchestra.run.smithi053.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T14:50:46.870 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T14:50:46.871 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.871 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.871 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.871 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:46.871 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.871 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.871 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.871 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.871 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.872 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.872 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.872 INFO:teuthology.orchestra.run.smithi053.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T14:50:46.872 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T14:50:46.872 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.872 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.872 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.873 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:46.873 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.873 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.873 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.873 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.873 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.873 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.873 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.873 INFO:teuthology.orchestra.run.smithi053.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T14:50:46.874 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T14:50:46.874 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.874 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.874 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.874 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:46.874 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.874 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.874 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.875 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.875 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.875 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.875 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.875 INFO:teuthology.orchestra.run.smithi053.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T14:50:46.875 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T14:50:46.875 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.876 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.876 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.876 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:46.876 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.876 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.876 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.876 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.876 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.876 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.877 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.877 INFO:teuthology.orchestra.run.smithi053.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T14:50:46.877 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T14:50:46.877 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.877 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.877 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.877 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:46.877 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.878 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.878 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.878 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.878 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.878 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.878 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.878 INFO:teuthology.orchestra.run.smithi053.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T14:50:46.879 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T14:50:46.879 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.879 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.879 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.879 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "admin", 2024-04-25T14:50:46.879 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.879 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.879 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.879 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.879 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.880 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.880 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.880 INFO:teuthology.orchestra.run.smithi053.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T14:50:46.880 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T14:50:46.880 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.880 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.880 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.880 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:46.881 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.881 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.881 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.881 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.881 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.881 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.881 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.881 INFO:teuthology.orchestra.run.smithi053.stdout: "GRAFANA_API_URL": { 2024-04-25T14:50:46.882 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "GRAFANA_API_URL", 2024-04-25T14:50:46.882 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.882 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.882 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.882 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:46.882 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.882 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.882 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.882 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.883 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.883 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.883 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.883 INFO:teuthology.orchestra.run.smithi053.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T14:50:46.883 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T14:50:46.883 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.883 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.883 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.884 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "admin", 2024-04-25T14:50:46.884 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.884 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.884 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.884 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.884 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.884 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.884 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.884 INFO:teuthology.orchestra.run.smithi053.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T14:50:46.885 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T14:50:46.885 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.885 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.885 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.885 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:46.885 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.885 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.885 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.885 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.885 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.886 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.886 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.886 INFO:teuthology.orchestra.run.smithi053.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T14:50:46.886 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T14:50:46.886 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.886 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.886 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.887 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:46.887 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.887 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.887 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.887 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.887 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.887 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.887 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.888 INFO:teuthology.orchestra.run.smithi053.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T14:50:46.888 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T14:50:46.888 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.888 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.888 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.888 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:46.888 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.888 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.888 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.888 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.889 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.889 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.889 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.889 INFO:teuthology.orchestra.run.smithi053.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T14:50:46.889 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T14:50:46.889 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.889 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.890 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.890 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:46.890 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.890 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.890 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.890 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.890 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.890 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.890 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.891 INFO:teuthology.orchestra.run.smithi053.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T14:50:46.891 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T14:50:46.891 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.891 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.891 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.891 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "[]", 2024-04-25T14:50:46.891 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.891 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.892 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.892 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.892 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.892 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.892 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.892 INFO:teuthology.orchestra.run.smithi053.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T14:50:46.892 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T14:50:46.892 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.892 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.893 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.893 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "{}", 2024-04-25T14:50:46.893 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.893 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.893 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.893 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.893 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.893 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.893 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.894 INFO:teuthology.orchestra.run.smithi053.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T14:50:46.894 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T14:50:46.894 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.894 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.894 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.894 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:46.894 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.895 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.895 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.895 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.895 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.895 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.895 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.895 INFO:teuthology.orchestra.run.smithi053.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T14:50:46.895 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T14:50:46.895 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.896 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.896 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.896 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:46.896 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.896 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.896 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.896 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.896 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.896 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.897 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.897 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T14:50:46.897 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T14:50:46.897 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.897 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.897 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.897 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:46.897 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.898 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.898 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.898 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.898 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.898 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.898 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.898 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T14:50:46.899 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T14:50:46.899 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.899 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.899 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.899 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:46.899 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.899 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.899 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.899 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.900 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.900 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.900 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.900 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T14:50:46.900 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T14:50:46.900 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.900 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.900 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.901 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:46.901 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.901 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.901 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.901 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.901 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.901 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.901 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.901 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T14:50:46.902 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T14:50:46.902 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.902 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.902 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.902 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:46.902 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.902 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.902 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.903 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.903 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.903 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.903 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.903 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T14:50:46.903 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T14:50:46.903 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.903 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.903 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.904 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:46.904 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.904 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.904 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.904 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.904 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.904 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.904 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.905 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T14:50:46.905 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T14:50:46.905 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.905 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.905 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.905 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:46.905 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.905 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.905 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.905 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.906 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.906 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.906 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.906 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T14:50:46.906 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T14:50:46.906 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.906 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.906 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.907 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:46.907 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.907 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.907 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.907 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.907 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.907 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.907 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.908 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T14:50:46.908 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T14:50:46.908 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.908 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.908 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.908 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:46.908 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.908 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.908 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.908 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.909 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.909 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.909 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.909 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T14:50:46.909 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T14:50:46.909 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.909 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.909 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.909 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T14:50:46.910 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.910 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.910 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.910 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.910 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.910 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.910 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.910 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T14:50:46.911 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T14:50:46.911 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:46.911 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.911 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.911 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2024-04-25T14:50:46.911 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.911 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.911 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.911 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.912 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.912 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.912 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.912 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T14:50:46.912 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T14:50:46.912 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:46.912 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.912 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.913 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "8", 2024-04-25T14:50:46.913 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.913 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.913 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.913 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.913 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.913 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.913 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.914 INFO:teuthology.orchestra.run.smithi053.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T14:50:46.914 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T14:50:46.914 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:46.914 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.914 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.914 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "45", 2024-04-25T14:50:46.914 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.915 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.915 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.915 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.915 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.915 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.915 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.915 INFO:teuthology.orchestra.run.smithi053.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T14:50:46.915 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T14:50:46.916 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.916 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.916 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.916 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:46.916 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.916 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.916 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.916 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.916 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.917 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.917 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.917 INFO:teuthology.orchestra.run.smithi053.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T14:50:46.917 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T14:50:46.917 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.917 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.917 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.917 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "admin", 2024-04-25T14:50:46.918 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.918 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.918 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.918 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.918 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.918 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.918 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.919 INFO:teuthology.orchestra.run.smithi053.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T14:50:46.919 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T14:50:46.919 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.919 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.919 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.919 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:46.919 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.919 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.919 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.919 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.920 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.920 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.920 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.920 INFO:teuthology.orchestra.run.smithi053.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T14:50:46.920 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T14:50:46.920 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.920 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.920 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.920 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:46.921 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.921 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.921 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.921 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.921 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.921 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.921 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.921 INFO:teuthology.orchestra.run.smithi053.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T14:50:46.921 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T14:50:46.922 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.922 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.922 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.922 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:46.922 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.922 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.922 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.922 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.922 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.923 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.923 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.923 INFO:teuthology.orchestra.run.smithi053.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T14:50:46.923 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T14:50:46.923 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:46.923 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.923 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.923 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0", 2024-04-25T14:50:46.923 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.924 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.924 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.924 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.924 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.924 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.924 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.924 INFO:teuthology.orchestra.run.smithi053.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T14:50:46.924 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T14:50:46.925 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:46.925 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.925 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.925 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2024-04-25T14:50:46.925 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.925 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.925 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.925 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.925 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.925 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.926 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.926 INFO:teuthology.orchestra.run.smithi053.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T14:50:46.926 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T14:50:46.926 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:46.926 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.926 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.926 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "5", 2024-04-25T14:50:46.927 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.927 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.927 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.927 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.927 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.927 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.927 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.927 INFO:teuthology.orchestra.run.smithi053.stdout: "cross_origin_url": { 2024-04-25T14:50:46.928 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "cross_origin_url", 2024-04-25T14:50:46.928 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.928 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.928 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.928 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:46.928 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.928 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.928 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.928 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.929 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.929 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.929 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.929 INFO:teuthology.orchestra.run.smithi053.stdout: "crt_file": { 2024-04-25T14:50:46.929 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "crt_file", 2024-04-25T14:50:46.929 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.929 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.929 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.929 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:46.930 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.930 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.930 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.930 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.930 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.930 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.930 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.930 INFO:teuthology.orchestra.run.smithi053.stdout: "debug": { 2024-04-25T14:50:46.931 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "debug", 2024-04-25T14:50:46.931 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.931 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.931 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.931 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:46.931 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.931 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.931 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.931 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Enable/disable debug options", 2024-04-25T14:50:46.932 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.932 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.932 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.932 INFO:teuthology.orchestra.run.smithi053.stdout: "jwt_token_ttl": { 2024-04-25T14:50:46.932 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "jwt_token_ttl", 2024-04-25T14:50:46.932 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:46.932 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.932 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.933 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "28800", 2024-04-25T14:50:46.933 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.933 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.933 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.933 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.933 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.933 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.933 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.933 INFO:teuthology.orchestra.run.smithi053.stdout: "key_file": { 2024-04-25T14:50:46.934 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "key_file", 2024-04-25T14:50:46.934 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.934 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.934 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.934 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:46.934 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.934 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.934 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.935 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.935 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.935 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.935 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.935 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:46.935 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:46.935 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.935 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.936 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.936 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:46.936 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.936 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.936 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:46.936 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:46.936 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:46.936 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:46.936 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:46.936 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:46.937 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:46.937 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:46.937 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.937 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.937 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.937 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.937 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:46.937 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:46.937 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.938 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.938 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.938 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:46.938 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.938 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.938 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.938 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.938 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.938 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.939 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.939 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:46.939 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:46.939 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.939 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.939 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.939 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:46.939 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.939 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.940 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:46.940 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:46.940 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:46.940 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:46.940 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:46.940 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:46.940 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:46.940 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:46.940 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.940 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.941 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.941 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.941 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:46.941 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:46.941 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.941 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.941 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.941 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:46.942 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.942 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.942 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.942 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.942 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.942 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.942 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.942 INFO:teuthology.orchestra.run.smithi053.stdout: "motd": { 2024-04-25T14:50:46.943 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "motd", 2024-04-25T14:50:46.943 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.943 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.943 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.943 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:46.943 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.943 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.943 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.944 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "The message of the day", 2024-04-25T14:50:46.944 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.944 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.944 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.944 INFO:teuthology.orchestra.run.smithi053.stdout: "redirect_resolve_ip_addr": { 2024-04-25T14:50:46.944 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T14:50:46.944 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.945 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.945 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.945 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:46.945 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.945 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.945 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.945 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.945 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.945 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.946 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.946 INFO:teuthology.orchestra.run.smithi053.stdout: "server_addr": { 2024-04-25T14:50:46.946 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "server_addr", 2024-04-25T14:50:46.946 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.946 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.946 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.946 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "::", 2024-04-25T14:50:46.946 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.947 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.947 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.947 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.947 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.947 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.947 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.947 INFO:teuthology.orchestra.run.smithi053.stdout: "server_port": { 2024-04-25T14:50:46.948 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "server_port", 2024-04-25T14:50:46.948 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:46.948 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.948 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.948 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "8080", 2024-04-25T14:50:46.948 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.948 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.948 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.948 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.948 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.949 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.949 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.949 INFO:teuthology.orchestra.run.smithi053.stdout: "ssl": { 2024-04-25T14:50:46.949 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ssl", 2024-04-25T14:50:46.949 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.949 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.949 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.950 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:46.950 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.950 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.950 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.950 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.950 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.950 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.950 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.951 INFO:teuthology.orchestra.run.smithi053.stdout: "ssl_server_port": { 2024-04-25T14:50:46.951 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ssl_server_port", 2024-04-25T14:50:46.951 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:46.951 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.951 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.951 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "8443", 2024-04-25T14:50:46.951 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.951 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.951 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.952 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.952 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.952 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.952 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.952 INFO:teuthology.orchestra.run.smithi053.stdout: "standby_behaviour": { 2024-04-25T14:50:46.952 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "standby_behaviour", 2024-04-25T14:50:46.952 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.952 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.953 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.953 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "redirect", 2024-04-25T14:50:46.953 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.953 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.953 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:46.953 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:46.953 INFO:teuthology.orchestra.run.smithi053.stdout: "redirect" 2024-04-25T14:50:46.953 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:46.953 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.954 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.954 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.954 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.954 INFO:teuthology.orchestra.run.smithi053.stdout: "standby_error_status_code": { 2024-04-25T14:50:46.954 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "standby_error_status_code", 2024-04-25T14:50:46.954 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:46.954 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.954 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.955 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "500", 2024-04-25T14:50:46.955 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "400", 2024-04-25T14:50:46.955 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "599", 2024-04-25T14:50:46.955 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.955 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.955 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.955 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.955 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.956 INFO:teuthology.orchestra.run.smithi053.stdout: "url_prefix": { 2024-04-25T14:50:46.956 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "url_prefix", 2024-04-25T14:50:46.956 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.956 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.956 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.956 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:46.956 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.956 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.957 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.957 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.957 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.957 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.957 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.957 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:46.957 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:46.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:46.958 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "devicehealth", 2024-04-25T14:50:46.958 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:46.958 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:46.958 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:46.958 INFO:teuthology.orchestra.run.smithi053.stdout: "enable_monitoring": { 2024-04-25T14:50:46.958 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "enable_monitoring", 2024-04-25T14:50:46.958 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.958 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.959 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.959 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:46.959 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.959 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.959 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.959 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "monitor device health metrics", 2024-04-25T14:50:46.959 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.959 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.959 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.960 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:46.960 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:46.960 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.960 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.960 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.960 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:46.960 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.960 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.960 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:46.961 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:46.961 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:46.961 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:46.961 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:46.961 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:46.961 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:46.961 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:46.961 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.961 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.961 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.962 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.962 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:46.962 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:46.962 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.962 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.962 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.962 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:46.963 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.963 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.963 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.963 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.963 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.963 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.963 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.963 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:46.964 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:46.964 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.964 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.964 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.964 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:46.964 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.964 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.964 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:46.965 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:46.965 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:46.965 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:46.965 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:46.965 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:46.965 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:46.965 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:46.965 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.965 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.965 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.966 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.966 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:46.966 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:46.966 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.966 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.966 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.966 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:46.966 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.967 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.967 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.967 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.967 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.967 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.967 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.967 INFO:teuthology.orchestra.run.smithi053.stdout: "mark_out_threshold": { 2024-04-25T14:50:46.967 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "mark_out_threshold", 2024-04-25T14:50:46.968 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:46.968 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.968 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.968 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "2419200", 2024-04-25T14:50:46.968 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.968 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.968 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.968 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T14:50:46.968 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.968 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.969 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.969 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": { 2024-04-25T14:50:46.969 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "pool_name", 2024-04-25T14:50:46.969 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.969 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.969 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.969 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "device_health_metrics", 2024-04-25T14:50:46.969 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.970 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.970 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.970 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T14:50:46.970 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.970 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.970 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.970 INFO:teuthology.orchestra.run.smithi053.stdout: "retention_period": { 2024-04-25T14:50:46.970 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "retention_period", 2024-04-25T14:50:46.971 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:46.971 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.971 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.971 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "15552000", 2024-04-25T14:50:46.971 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.971 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.971 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.971 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how long to retain device health metrics", 2024-04-25T14:50:46.971 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.971 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.972 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.972 INFO:teuthology.orchestra.run.smithi053.stdout: "scrape_frequency": { 2024-04-25T14:50:46.972 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "scrape_frequency", 2024-04-25T14:50:46.972 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:46.972 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.972 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.972 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "86400", 2024-04-25T14:50:46.972 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.972 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.973 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.973 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T14:50:46.973 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.973 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.973 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.973 INFO:teuthology.orchestra.run.smithi053.stdout: "self_heal": { 2024-04-25T14:50:46.973 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "self_heal", 2024-04-25T14:50:46.973 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.974 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.974 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.974 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:46.974 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.974 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.974 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.974 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T14:50:46.974 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.974 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.975 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.975 INFO:teuthology.orchestra.run.smithi053.stdout: "sleep_interval": { 2024-04-25T14:50:46.975 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "sleep_interval", 2024-04-25T14:50:46.975 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:46.975 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.975 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.975 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "600", 2024-04-25T14:50:46.975 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.976 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.976 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.976 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T14:50:46.976 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.976 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.976 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.976 INFO:teuthology.orchestra.run.smithi053.stdout: "warn_threshold": { 2024-04-25T14:50:46.976 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "warn_threshold", 2024-04-25T14:50:46.976 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:46.977 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.977 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.977 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "7257600", 2024-04-25T14:50:46.977 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.977 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.977 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.977 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T14:50:46.977 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.977 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.977 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.978 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:46.978 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:46.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:46.978 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "diskprediction_local", 2024-04-25T14:50:46.978 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:46.978 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:46.978 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:46.979 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:46.979 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:46.979 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.979 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.979 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.979 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:46.979 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.979 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.979 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:46.980 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:46.980 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:46.980 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:46.980 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:46.980 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:46.980 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:46.980 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:46.980 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.980 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.980 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.981 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.981 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:46.981 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:46.981 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.981 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.981 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.981 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:46.981 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.982 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.982 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.982 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.982 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.982 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.982 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.982 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:46.982 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:46.983 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.983 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.983 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.983 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:46.983 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.983 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.983 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:46.983 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:46.984 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:46.984 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:46.984 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:46.984 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:46.984 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:46.984 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:46.984 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.984 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.984 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.985 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.985 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:46.985 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:46.985 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:46.985 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.985 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.985 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:46.985 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.986 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.986 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.986 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.986 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.986 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.986 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.986 INFO:teuthology.orchestra.run.smithi053.stdout: "predict_interval": { 2024-04-25T14:50:46.987 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "predict_interval", 2024-04-25T14:50:46.987 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.987 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.987 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.987 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "86400", 2024-04-25T14:50:46.987 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.987 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.987 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.987 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.988 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.988 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.988 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.988 INFO:teuthology.orchestra.run.smithi053.stdout: "predictor_model": { 2024-04-25T14:50:46.988 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "predictor_model", 2024-04-25T14:50:46.988 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.988 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.988 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.989 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "prophetstor", 2024-04-25T14:50:46.989 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.989 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.989 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.989 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.989 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.989 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.989 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.990 INFO:teuthology.orchestra.run.smithi053.stdout: "sleep_interval": { 2024-04-25T14:50:46.990 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "sleep_interval", 2024-04-25T14:50:46.990 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.990 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.990 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.990 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "600", 2024-04-25T14:50:46.990 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.991 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.991 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.991 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:46.991 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.991 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.991 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.991 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:46.991 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:46.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:46.992 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "influx", 2024-04-25T14:50:46.992 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": false, 2024-04-25T14:50:46.992 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "influxdb python module not found", 2024-04-25T14:50:46.992 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:46.992 INFO:teuthology.orchestra.run.smithi053.stdout: "batch_size": { 2024-04-25T14:50:46.992 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "batch_size", 2024-04-25T14:50:46.992 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:46.993 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.993 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.993 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "5000", 2024-04-25T14:50:46.993 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.993 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.993 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.993 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T14:50:46.993 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.993 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.993 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.994 INFO:teuthology.orchestra.run.smithi053.stdout: "database": { 2024-04-25T14:50:46.994 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "database", 2024-04-25T14:50:46.994 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.994 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.994 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.994 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "ceph", 2024-04-25T14:50:46.994 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.995 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.995 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.995 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T14:50:46.995 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.995 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.995 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.995 INFO:teuthology.orchestra.run.smithi053.stdout: "hostname": { 2024-04-25T14:50:46.995 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "hostname", 2024-04-25T14:50:46.996 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.996 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.996 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.996 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:46.996 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.996 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.996 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.996 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "InfluxDB server hostname", 2024-04-25T14:50:46.996 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.996 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.997 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.997 INFO:teuthology.orchestra.run.smithi053.stdout: "interval": { 2024-04-25T14:50:46.997 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "interval", 2024-04-25T14:50:46.997 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:46.997 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.997 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:46.997 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "30", 2024-04-25T14:50:46.997 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "5", 2024-04-25T14:50:46.998 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.998 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:46.998 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T14:50:46.998 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:46.998 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:46.998 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:46.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:46.998 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:46.999 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:46.999 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:46.999 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:46.999 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:46.999 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:46.999 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:46.999 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:46.999 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:46.999 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.000 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.000 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.000 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.000 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.000 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.000 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.000 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.000 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.000 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.001 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.001 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.001 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.001 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.001 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.001 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.001 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.001 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.001 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.002 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.002 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.002 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.002 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.002 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.002 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.002 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.003 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.003 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.003 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.003 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.003 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.003 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.003 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.003 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.003 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.004 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.004 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.004 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.004 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.004 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.004 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.004 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.004 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.004 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.005 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.005 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.005 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.005 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.005 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.005 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.005 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.005 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.005 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.006 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.006 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.006 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.006 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.006 INFO:teuthology.orchestra.run.smithi053.stdout: "password": { 2024-04-25T14:50:47.006 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "password", 2024-04-25T14:50:47.006 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.007 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.007 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.007 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.007 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.007 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.007 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.007 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "password of InfluxDB server user", 2024-04-25T14:50:47.007 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.007 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.007 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.008 INFO:teuthology.orchestra.run.smithi053.stdout: "port": { 2024-04-25T14:50:47.008 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "port", 2024-04-25T14:50:47.008 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.008 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.008 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.008 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "8086", 2024-04-25T14:50:47.008 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.008 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.009 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.009 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "InfluxDB server port", 2024-04-25T14:50:47.009 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.009 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.009 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.009 INFO:teuthology.orchestra.run.smithi053.stdout: "ssl": { 2024-04-25T14:50:47.009 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ssl", 2024-04-25T14:50:47.009 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.009 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.010 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.010 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "false", 2024-04-25T14:50:47.010 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.010 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.010 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.010 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T14:50:47.010 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.010 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.011 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.011 INFO:teuthology.orchestra.run.smithi053.stdout: "threads": { 2024-04-25T14:50:47.011 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "threads", 2024-04-25T14:50:47.011 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.011 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.011 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.011 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "5", 2024-04-25T14:50:47.011 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "1", 2024-04-25T14:50:47.011 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "32", 2024-04-25T14:50:47.012 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.012 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T14:50:47.012 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.012 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.012 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.012 INFO:teuthology.orchestra.run.smithi053.stdout: "username": { 2024-04-25T14:50:47.012 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "username", 2024-04-25T14:50:47.012 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.013 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.013 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.013 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.013 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.013 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.013 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.013 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "username of InfluxDB server user", 2024-04-25T14:50:47.013 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.013 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.014 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.014 INFO:teuthology.orchestra.run.smithi053.stdout: "verify_ssl": { 2024-04-25T14:50:47.014 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "verify_ssl", 2024-04-25T14:50:47.014 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.014 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.014 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.014 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "true", 2024-04-25T14:50:47.014 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.014 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.015 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.015 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T14:50:47.015 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.015 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.015 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.015 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.015 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.016 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "insights", 2024-04-25T14:50:47.016 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.016 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.016 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.016 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.016 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.016 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.016 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.016 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.017 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.017 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.017 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.017 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.017 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.017 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.017 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.017 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.017 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.017 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.018 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.018 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.018 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.018 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.018 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.018 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.018 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.018 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.019 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.019 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.019 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.019 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.019 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.019 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.019 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.019 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.019 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.019 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.020 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.020 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.020 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.020 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.020 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.020 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.020 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.020 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.020 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.021 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.021 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.021 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.021 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.021 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.021 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.021 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.021 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.021 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.021 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.022 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.022 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.022 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.022 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.022 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.022 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.022 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.022 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.023 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.023 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.023 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.023 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.023 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.023 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.023 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.023 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.024 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "iostat", 2024-04-25T14:50:47.024 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.024 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.024 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.024 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.024 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.024 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.024 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.024 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.024 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.025 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.025 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.025 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.025 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.025 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.025 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.025 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.025 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.025 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.026 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.026 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.026 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.026 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.026 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.026 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.026 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.026 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.027 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.027 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.027 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.027 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.027 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.027 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.027 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.027 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.027 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.027 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.028 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.028 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.028 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.028 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.028 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.028 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.028 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.028 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.029 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.029 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.029 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.029 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.029 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.029 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.029 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.029 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.029 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.029 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.030 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.030 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.030 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.030 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.030 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.030 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.030 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.030 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.031 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.031 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.031 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.031 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.031 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.031 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.031 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.031 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.031 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.032 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "k8sevents", 2024-04-25T14:50:47.032 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.032 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.032 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.032 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_event_retention_days": { 2024-04-25T14:50:47.032 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ceph_event_retention_days", 2024-04-25T14:50:47.032 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.033 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.033 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.033 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "7", 2024-04-25T14:50:47.033 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.033 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.033 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.033 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T14:50:47.033 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.033 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.033 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.034 INFO:teuthology.orchestra.run.smithi053.stdout: "config_check_secs": { 2024-04-25T14:50:47.034 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "config_check_secs", 2024-04-25T14:50:47.034 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.034 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.034 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.034 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2024-04-25T14:50:47.034 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "10", 2024-04-25T14:50:47.034 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.035 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.035 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T14:50:47.035 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.035 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.035 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.035 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.035 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.035 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.036 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.036 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.036 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.036 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.036 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.036 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.036 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.036 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.036 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.036 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.037 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.037 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.037 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.037 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.037 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.037 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.037 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.037 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.037 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.038 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.038 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.038 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.038 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.038 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.038 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.038 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.038 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.038 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.039 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.039 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.039 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.039 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.039 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.039 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.039 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.039 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.040 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.040 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.040 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.040 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.040 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.040 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.040 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.040 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.040 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.040 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.041 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.041 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.041 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.041 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.041 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.041 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.041 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.041 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.041 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.042 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.042 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.042 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.042 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.042 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.042 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.042 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.042 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.042 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.042 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.043 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "localpool", 2024-04-25T14:50:47.043 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.043 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.043 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.043 INFO:teuthology.orchestra.run.smithi053.stdout: "failure_domain": { 2024-04-25T14:50:47.043 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "failure_domain", 2024-04-25T14:50:47.043 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.043 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.044 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.044 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "host", 2024-04-25T14:50:47.044 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.044 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.044 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.044 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "failure domain for any created local pool", 2024-04-25T14:50:47.044 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T14:50:47.044 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.044 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.045 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.045 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.045 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.045 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.045 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.045 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.045 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.045 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.045 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.046 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.046 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.046 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.046 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.046 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.046 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.046 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.046 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.046 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.047 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.047 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.047 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.047 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.047 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.047 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.047 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.047 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.047 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.048 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.048 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.048 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.048 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.048 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.048 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.048 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.048 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.048 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.049 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.049 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.049 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.049 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.049 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.049 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.049 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.049 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.049 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.050 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.050 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.050 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.050 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.050 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.050 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.050 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.050 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.051 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.051 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.051 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.051 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.051 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.051 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.051 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.051 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.051 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.052 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.052 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.052 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.052 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.052 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": { 2024-04-25T14:50:47.052 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "min_size", 2024-04-25T14:50:47.052 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.052 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.052 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.053 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.053 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.053 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.053 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.053 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "default min_size for any created local pool", 2024-04-25T14:50:47.053 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T14:50:47.053 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.053 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.053 INFO:teuthology.orchestra.run.smithi053.stdout: "num_rep": { 2024-04-25T14:50:47.054 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "num_rep", 2024-04-25T14:50:47.054 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.054 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.054 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.054 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "3", 2024-04-25T14:50:47.054 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.054 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.054 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.054 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "default replica count for any created local pool", 2024-04-25T14:50:47.055 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.055 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.055 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.055 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": { 2024-04-25T14:50:47.055 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "pg_num", 2024-04-25T14:50:47.055 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.055 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.055 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.055 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "128", 2024-04-25T14:50:47.056 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.056 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.056 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.056 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T14:50:47.056 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.056 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.056 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.056 INFO:teuthology.orchestra.run.smithi053.stdout: "prefix": { 2024-04-25T14:50:47.057 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "prefix", 2024-04-25T14:50:47.057 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.057 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.057 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.057 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.057 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.057 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.057 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.057 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "name prefix for any created local pool", 2024-04-25T14:50:47.057 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.058 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.058 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.058 INFO:teuthology.orchestra.run.smithi053.stdout: "subtree": { 2024-04-25T14:50:47.058 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "subtree", 2024-04-25T14:50:47.058 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.058 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.058 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.059 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "rack", 2024-04-25T14:50:47.059 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.059 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.059 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.059 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T14:50:47.059 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T14:50:47.059 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.059 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.059 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.059 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.060 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "mds_autoscaler", 2024-04-25T14:50:47.060 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.060 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.060 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.060 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.060 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.060 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.060 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.061 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.061 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.061 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.061 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.061 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.061 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.061 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.061 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.061 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.061 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.062 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.062 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.062 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.062 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.062 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.062 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.062 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.062 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.063 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.063 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.063 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.063 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.063 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.063 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.063 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.063 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.063 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.064 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.064 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.064 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.064 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.064 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.064 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.064 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.064 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.064 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.065 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.065 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.065 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.065 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.065 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.065 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.065 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.065 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.065 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.065 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.066 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.066 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.066 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.066 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.066 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.066 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.066 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.066 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.067 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.067 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.067 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.067 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.067 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.067 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.067 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.067 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.067 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.067 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.068 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "mirroring", 2024-04-25T14:50:47.068 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.068 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.068 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.068 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.068 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.068 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.068 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.069 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.069 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.069 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.069 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.069 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.069 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.069 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.069 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.069 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.069 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.070 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.070 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.070 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.070 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.070 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.070 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.070 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.070 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.071 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.071 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.071 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.071 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.071 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.071 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.071 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.071 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.071 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.072 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.072 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.072 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.072 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.072 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.072 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.072 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.072 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.072 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.073 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.073 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.073 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.073 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.073 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.073 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.073 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.073 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.073 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.074 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.074 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.074 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.074 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.074 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.074 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.074 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.074 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.075 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.075 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.075 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.075 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.075 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.075 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.075 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.075 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.075 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.075 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.076 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.076 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "nfs", 2024-04-25T14:50:47.076 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.076 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.076 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.076 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.076 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.077 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.077 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.077 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.077 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.077 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.077 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.077 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.077 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.077 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.078 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.078 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.078 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.078 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.078 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.078 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.078 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.078 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.079 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.079 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.079 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.079 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.079 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.079 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.079 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.079 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.079 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.080 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.080 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.080 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.080 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.080 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.080 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.080 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.080 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.080 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.081 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.081 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.081 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.081 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.081 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.081 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.081 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.081 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.081 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.082 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.082 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.082 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.082 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.082 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.082 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.082 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.082 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.083 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.083 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.083 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.083 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.083 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.083 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.083 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.083 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.083 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.083 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.084 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.084 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.084 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.084 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.084 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "orchestrator", 2024-04-25T14:50:47.084 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.084 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.084 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.085 INFO:teuthology.orchestra.run.smithi053.stdout: "fail_fs": { 2024-04-25T14:50:47.085 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "fail_fs", 2024-04-25T14:50:47.085 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.085 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.085 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.085 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.085 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.085 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.085 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.086 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T14:50:47.086 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.086 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.086 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.086 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.086 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.086 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.086 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.086 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.087 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.087 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.087 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.087 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.087 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.087 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.087 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.087 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.087 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.088 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.088 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.088 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.088 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.088 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.088 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.088 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.088 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.088 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.089 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.089 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.089 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.089 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.089 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.089 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.089 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.089 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.089 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.089 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.090 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.090 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.090 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.090 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.090 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.090 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.090 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.090 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.091 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.091 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.091 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.091 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.091 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.091 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.091 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.091 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.092 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.092 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.092 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.092 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.092 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.092 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.092 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.092 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.092 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.093 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.093 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.093 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.093 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.093 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.093 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.093 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.093 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.093 INFO:teuthology.orchestra.run.smithi053.stdout: "orchestrator": { 2024-04-25T14:50:47.094 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "orchestrator", 2024-04-25T14:50:47.094 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.094 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.094 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.094 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.094 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.094 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.094 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.094 INFO:teuthology.orchestra.run.smithi053.stdout: "cephadm", 2024-04-25T14:50:47.095 INFO:teuthology.orchestra.run.smithi053.stdout: "rook", 2024-04-25T14:50:47.095 INFO:teuthology.orchestra.run.smithi053.stdout: "test_orchestrator" 2024-04-25T14:50:47.095 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.095 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Orchestrator backend", 2024-04-25T14:50:47.095 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.095 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.095 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.095 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.095 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.096 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "osd_perf_query", 2024-04-25T14:50:47.096 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.096 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.096 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.096 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.096 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.096 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.097 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.097 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.097 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.097 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.097 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.097 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.097 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.097 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.097 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.098 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.098 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.098 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.098 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.098 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.098 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.098 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.098 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.099 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.099 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.099 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.099 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.099 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.099 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.099 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.099 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.099 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.099 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.100 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.100 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.100 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.100 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.100 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.100 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.100 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.100 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.101 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.101 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.101 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.101 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.101 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.101 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.101 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.101 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.101 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.101 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.102 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.102 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.102 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.102 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.102 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.102 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.102 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.102 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.103 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.103 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.103 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.103 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.103 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.103 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.103 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.103 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.103 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.104 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.104 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.104 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.104 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "osd_support", 2024-04-25T14:50:47.104 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.104 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.104 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.105 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.105 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.105 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.105 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.105 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.105 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.105 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.105 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.105 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.105 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.106 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.106 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.106 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.106 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.106 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.106 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.106 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.106 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.106 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.107 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.107 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.107 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.107 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.107 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.107 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.107 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.108 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.108 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.108 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.108 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.108 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.108 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.108 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.108 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.109 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.109 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.109 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.109 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.109 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.109 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.109 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.109 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.109 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.110 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.110 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.110 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.110 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.110 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.110 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.110 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.110 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.110 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.111 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.111 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.111 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.111 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.111 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.111 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.111 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.111 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.112 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.112 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.112 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.112 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.112 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.112 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.112 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.112 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.113 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "pg_autoscaler", 2024-04-25T14:50:47.113 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.113 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.113 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.113 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.113 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.114 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.114 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.114 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.114 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.114 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.114 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.114 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.114 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.114 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.115 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.115 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.115 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.115 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.115 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.115 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.115 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.115 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.115 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.116 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.116 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.116 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.116 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.116 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.116 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.116 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.116 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.117 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.117 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.117 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.117 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.117 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.117 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.117 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.117 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.118 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.118 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.118 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.118 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.118 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.118 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.118 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.118 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.119 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.119 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.119 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.119 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.119 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.119 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.119 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.119 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.119 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.120 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.120 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.120 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.120 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.120 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.120 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.120 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.120 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.120 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.121 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.121 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.121 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.121 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.121 INFO:teuthology.orchestra.run.smithi053.stdout: "sleep_interval": { 2024-04-25T14:50:47.121 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "sleep_interval", 2024-04-25T14:50:47.121 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:47.121 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.122 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.122 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "60", 2024-04-25T14:50:47.122 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.122 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.122 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.122 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.122 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.122 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.123 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.123 INFO:teuthology.orchestra.run.smithi053.stdout: "threshold": { 2024-04-25T14:50:47.123 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "threshold", 2024-04-25T14:50:47.123 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2024-04-25T14:50:47.123 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.123 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.123 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "3.0", 2024-04-25T14:50:47.123 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "1.0", 2024-04-25T14:50:47.124 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.124 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.124 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "scaling threshold", 2024-04-25T14:50:47.124 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T14:50:47.124 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.124 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.124 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.124 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.125 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "progress", 2024-04-25T14:50:47.125 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.125 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.125 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.125 INFO:teuthology.orchestra.run.smithi053.stdout: "allow_pg_recovery_event": { 2024-04-25T14:50:47.125 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "allow_pg_recovery_event", 2024-04-25T14:50:47.125 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.125 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.126 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.126 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.126 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.126 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.126 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.126 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T14:50:47.126 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.126 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.127 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.127 INFO:teuthology.orchestra.run.smithi053.stdout: "enabled": { 2024-04-25T14:50:47.127 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "enabled", 2024-04-25T14:50:47.127 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.127 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.127 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.127 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.127 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.128 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.128 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.128 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.128 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.128 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.128 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.128 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.128 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.129 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.129 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.129 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.129 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.129 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.129 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.129 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.129 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.129 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.130 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.130 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.130 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.130 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.130 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.130 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.130 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.130 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.130 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.131 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.131 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.131 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.131 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.131 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.131 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.131 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.131 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.132 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.132 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.132 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.132 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.132 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.132 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.132 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.132 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.132 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.133 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.133 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.133 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.133 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.133 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.133 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.133 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.133 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.133 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.134 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.134 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.134 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.134 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.134 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.134 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.134 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.134 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.135 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.135 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.135 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.135 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.135 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.135 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.135 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.135 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.135 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.135 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.136 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.136 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.136 INFO:teuthology.orchestra.run.smithi053.stdout: "max_completed_events": { 2024-04-25T14:50:47.136 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "max_completed_events", 2024-04-25T14:50:47.136 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.136 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.136 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.136 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "50", 2024-04-25T14:50:47.137 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.137 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.137 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.137 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "number of past completed events to remember", 2024-04-25T14:50:47.137 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.137 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.137 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.137 INFO:teuthology.orchestra.run.smithi053.stdout: "sleep_interval": { 2024-04-25T14:50:47.137 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "sleep_interval", 2024-04-25T14:50:47.138 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:47.138 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.138 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.138 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "5", 2024-04-25T14:50:47.138 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.138 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.138 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.138 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how long the module is going to sleep", 2024-04-25T14:50:47.138 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.139 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.139 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.139 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.139 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.139 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "prometheus", 2024-04-25T14:50:47.139 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.139 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.140 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.140 INFO:teuthology.orchestra.run.smithi053.stdout: "cache": { 2024-04-25T14:50:47.140 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "cache", 2024-04-25T14:50:47.140 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.140 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.140 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.140 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.140 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.140 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.140 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.141 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.141 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.141 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.141 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.141 INFO:teuthology.orchestra.run.smithi053.stdout: "exclude_perf_counters": { 2024-04-25T14:50:47.141 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "exclude_perf_counters", 2024-04-25T14:50:47.141 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.141 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.141 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.142 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.142 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.142 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.142 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.142 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T14:50:47.142 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T14:50:47.142 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.142 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.143 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.143 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.143 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.143 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.143 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.143 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.143 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.143 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.143 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.144 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.144 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.144 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.144 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.144 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.144 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.144 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.144 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.144 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.144 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.145 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.145 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.145 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.145 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.145 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.145 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.145 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.145 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.145 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.146 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.146 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.146 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.146 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.146 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.146 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.146 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.146 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.147 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.147 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.147 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.147 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.147 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.147 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.147 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.147 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.147 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.148 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.148 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.148 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.148 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.148 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.148 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.148 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.148 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.148 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.149 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.149 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.149 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.149 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.149 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.149 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.149 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.149 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.149 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.149 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.150 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.150 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.150 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_stats_pools": { 2024-04-25T14:50:47.150 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rbd_stats_pools", 2024-04-25T14:50:47.150 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.150 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.150 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.150 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.151 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.151 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.151 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.151 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.151 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.151 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.151 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.151 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T14:50:47.151 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T14:50:47.152 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.152 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.152 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.152 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "300", 2024-04-25T14:50:47.152 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.152 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.152 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.152 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.152 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.152 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.153 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.153 INFO:teuthology.orchestra.run.smithi053.stdout: "scrape_interval": { 2024-04-25T14:50:47.153 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "scrape_interval", 2024-04-25T14:50:47.153 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2024-04-25T14:50:47.153 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.153 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.153 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "15.0", 2024-04-25T14:50:47.153 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.154 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.154 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.154 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.154 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.154 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.154 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.154 INFO:teuthology.orchestra.run.smithi053.stdout: "server_addr": { 2024-04-25T14:50:47.154 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "server_addr", 2024-04-25T14:50:47.155 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.155 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.155 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.155 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "::", 2024-04-25T14:50:47.155 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.155 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.155 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.155 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T14:50:47.155 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.156 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.156 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.156 INFO:teuthology.orchestra.run.smithi053.stdout: "server_port": { 2024-04-25T14:50:47.156 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "server_port", 2024-04-25T14:50:47.156 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.156 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.156 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.157 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "9283", 2024-04-25T14:50:47.157 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.157 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.157 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.157 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T14:50:47.157 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.157 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.157 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.158 INFO:teuthology.orchestra.run.smithi053.stdout: "stale_cache_strategy": { 2024-04-25T14:50:47.158 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "stale_cache_strategy", 2024-04-25T14:50:47.158 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.158 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.158 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.158 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "log", 2024-04-25T14:50:47.158 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.158 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.158 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.159 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.159 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.159 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.159 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.159 INFO:teuthology.orchestra.run.smithi053.stdout: "standby_behaviour": { 2024-04-25T14:50:47.159 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "standby_behaviour", 2024-04-25T14:50:47.159 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.160 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.160 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.160 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "default", 2024-04-25T14:50:47.160 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.160 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.160 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.160 INFO:teuthology.orchestra.run.smithi053.stdout: "default", 2024-04-25T14:50:47.160 INFO:teuthology.orchestra.run.smithi053.stdout: "error" 2024-04-25T14:50:47.160 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.161 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.161 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.161 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.161 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.161 INFO:teuthology.orchestra.run.smithi053.stdout: "standby_error_status_code": { 2024-04-25T14:50:47.161 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "standby_error_status_code", 2024-04-25T14:50:47.161 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.161 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.162 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.162 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "500", 2024-04-25T14:50:47.162 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "400", 2024-04-25T14:50:47.162 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "599", 2024-04-25T14:50:47.162 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.162 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.162 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.162 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.163 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.163 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.163 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.163 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rbd_support", 2024-04-25T14:50:47.163 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.163 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.163 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.164 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.164 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.164 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.164 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.164 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.164 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.164 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.164 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.164 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.164 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.165 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.165 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.165 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.165 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.165 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.165 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.165 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.165 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.165 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.165 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.166 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.166 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.166 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.166 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.166 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.166 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.166 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.166 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.167 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.167 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.167 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.167 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.167 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.167 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.167 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.167 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.167 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.168 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.168 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.168 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.168 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.168 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.168 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.168 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.168 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.168 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.168 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.169 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.169 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.169 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.169 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.169 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.169 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.169 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.169 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.170 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.170 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.170 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.170 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.170 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.170 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.170 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.170 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.170 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.170 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.171 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.171 INFO:teuthology.orchestra.run.smithi053.stdout: "max_concurrent_snap_create": { 2024-04-25T14:50:47.171 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "max_concurrent_snap_create", 2024-04-25T14:50:47.171 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.171 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.171 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.171 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2024-04-25T14:50:47.171 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.171 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.172 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.172 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.172 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.172 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.172 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.172 INFO:teuthology.orchestra.run.smithi053.stdout: "mirror_snapshot_schedule": { 2024-04-25T14:50:47.172 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T14:50:47.172 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.172 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.173 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.173 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.173 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.173 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.173 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.173 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.173 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.173 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.173 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.174 INFO:teuthology.orchestra.run.smithi053.stdout: "trash_purge_schedule": { 2024-04-25T14:50:47.174 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "trash_purge_schedule", 2024-04-25T14:50:47.174 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.174 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.174 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.174 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.174 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.175 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.175 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.175 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.175 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.175 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.175 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.175 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.175 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.176 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "restful", 2024-04-25T14:50:47.176 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.176 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.176 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.176 INFO:teuthology.orchestra.run.smithi053.stdout: "enable_auth": { 2024-04-25T14:50:47.176 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "enable_auth", 2024-04-25T14:50:47.176 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.176 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.177 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.177 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.177 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.177 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.177 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.177 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.177 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.177 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.177 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.178 INFO:teuthology.orchestra.run.smithi053.stdout: "key_file": { 2024-04-25T14:50:47.178 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "key_file", 2024-04-25T14:50:47.178 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.178 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.178 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.178 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.178 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.178 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.179 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.179 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.179 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.179 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.179 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.179 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.179 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.179 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.180 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.180 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.180 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.180 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.180 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.180 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.180 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.180 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.180 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.181 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.181 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.181 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.181 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.181 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.181 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.181 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.181 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.181 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.182 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.182 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.182 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.182 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.182 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.182 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.182 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.182 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.183 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.183 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.183 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.183 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.183 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.183 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.183 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.183 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.183 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.184 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.184 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.184 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.184 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.184 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.184 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.184 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.184 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.184 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.184 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.185 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.185 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.185 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.185 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.185 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.185 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.185 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.185 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.186 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.186 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.186 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.186 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.186 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.186 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.186 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.186 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.186 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.186 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.187 INFO:teuthology.orchestra.run.smithi053.stdout: "server_addr": { 2024-04-25T14:50:47.187 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "server_addr", 2024-04-25T14:50:47.187 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.187 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.187 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.187 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.187 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.187 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.187 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.188 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.188 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.188 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.188 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.188 INFO:teuthology.orchestra.run.smithi053.stdout: "server_port": { 2024-04-25T14:50:47.188 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "server_port", 2024-04-25T14:50:47.188 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.188 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.189 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.189 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.189 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.189 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.189 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.189 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.189 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.189 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.189 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.190 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.190 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.190 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rgw", 2024-04-25T14:50:47.190 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.190 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.190 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.190 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.191 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.191 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.191 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.191 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.191 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.191 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.191 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.191 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.191 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.192 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.192 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.192 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.192 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.192 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.192 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.192 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.192 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.192 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.192 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.193 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.193 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.193 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.193 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.193 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.193 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.193 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.193 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.193 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.194 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.194 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.194 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.194 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.194 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.194 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.194 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.195 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.195 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.195 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.195 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.195 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.195 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.195 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.195 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.195 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.196 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.196 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.196 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.196 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.196 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.196 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.196 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.196 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.197 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.197 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.197 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.197 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.197 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.197 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.197 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.197 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.197 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.197 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.198 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.198 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.198 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.198 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.198 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.198 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rook", 2024-04-25T14:50:47.198 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.199 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.199 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.199 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.199 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.199 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.199 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.199 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.199 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.199 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.200 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.200 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.200 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.200 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.200 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.200 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.200 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.200 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.200 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.200 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.201 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.201 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.201 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.201 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.201 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.201 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.201 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.201 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.201 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.202 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.202 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.202 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.202 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.202 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.202 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.202 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.203 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.203 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.203 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.203 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.203 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.203 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.203 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.203 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.203 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.203 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.204 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.204 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.204 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.204 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.204 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.204 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.204 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.204 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.204 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.205 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.205 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.205 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.205 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.205 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.205 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.205 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.205 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.205 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.206 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.206 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.206 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.206 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.206 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.206 INFO:teuthology.orchestra.run.smithi053.stdout: "storage_class": { 2024-04-25T14:50:47.206 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "storage_class", 2024-04-25T14:50:47.206 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.207 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.207 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.207 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "local", 2024-04-25T14:50:47.207 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.207 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.207 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.207 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T14:50:47.207 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.207 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.208 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.208 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.208 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.208 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "selftest", 2024-04-25T14:50:47.208 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.208 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.208 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.208 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.209 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.209 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.209 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.209 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.209 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.209 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.209 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.209 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.209 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.210 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.210 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.210 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.210 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.210 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.210 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.210 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.210 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.210 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.211 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.211 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.211 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.211 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.211 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.211 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.211 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.211 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.211 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.212 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.212 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.212 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.212 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.212 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.212 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.212 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.212 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.213 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.213 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.213 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.213 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.213 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.213 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.213 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.213 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.213 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.214 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.214 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.214 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.214 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.214 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.214 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.214 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.214 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.215 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.215 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.215 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.215 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.215 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.215 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.215 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.215 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.216 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.216 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.216 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.216 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.216 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.216 INFO:teuthology.orchestra.run.smithi053.stdout: "roption1": { 2024-04-25T14:50:47.216 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "roption1", 2024-04-25T14:50:47.216 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.217 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.217 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.217 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.217 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.217 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.217 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.217 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.217 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.218 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.218 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.218 INFO:teuthology.orchestra.run.smithi053.stdout: "roption2": { 2024-04-25T14:50:47.218 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "roption2", 2024-04-25T14:50:47.218 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.218 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.218 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.218 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "xyz", 2024-04-25T14:50:47.219 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.219 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.219 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.219 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.219 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.219 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.219 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.219 INFO:teuthology.orchestra.run.smithi053.stdout: "rwoption1": { 2024-04-25T14:50:47.220 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rwoption1", 2024-04-25T14:50:47.220 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.220 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.220 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.220 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.220 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.220 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.220 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.220 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.221 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.221 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.221 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.221 INFO:teuthology.orchestra.run.smithi053.stdout: "rwoption2": { 2024-04-25T14:50:47.221 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rwoption2", 2024-04-25T14:50:47.221 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.221 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.221 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.222 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.222 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.222 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.222 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.222 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.222 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.222 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.222 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.223 INFO:teuthology.orchestra.run.smithi053.stdout: "rwoption3": { 2024-04-25T14:50:47.223 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rwoption3", 2024-04-25T14:50:47.223 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2024-04-25T14:50:47.223 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.223 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.223 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.223 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.223 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.224 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.224 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.224 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.224 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.224 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.224 INFO:teuthology.orchestra.run.smithi053.stdout: "rwoption4": { 2024-04-25T14:50:47.224 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rwoption4", 2024-04-25T14:50:47.224 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.225 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.225 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.225 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.225 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.225 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.225 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.225 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.225 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.225 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.226 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.226 INFO:teuthology.orchestra.run.smithi053.stdout: "rwoption5": { 2024-04-25T14:50:47.226 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rwoption5", 2024-04-25T14:50:47.226 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.226 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.226 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.226 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.226 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.227 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.227 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.227 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.227 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.227 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.227 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.227 INFO:teuthology.orchestra.run.smithi053.stdout: "rwoption6": { 2024-04-25T14:50:47.227 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rwoption6", 2024-04-25T14:50:47.228 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.228 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.228 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.228 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.228 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.228 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.228 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.228 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.228 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.228 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.229 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.229 INFO:teuthology.orchestra.run.smithi053.stdout: "rwoption7": { 2024-04-25T14:50:47.229 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rwoption7", 2024-04-25T14:50:47.229 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.229 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.229 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.229 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.229 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "1", 2024-04-25T14:50:47.230 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "42", 2024-04-25T14:50:47.230 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.230 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.230 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.230 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.230 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.230 INFO:teuthology.orchestra.run.smithi053.stdout: "testkey": { 2024-04-25T14:50:47.230 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "testkey", 2024-04-25T14:50:47.231 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.231 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.231 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.231 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.231 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.231 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.231 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.231 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.231 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.232 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.232 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.232 INFO:teuthology.orchestra.run.smithi053.stdout: "testlkey": { 2024-04-25T14:50:47.232 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "testlkey", 2024-04-25T14:50:47.232 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.232 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.232 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.232 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.232 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.233 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.233 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.233 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.233 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.233 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.233 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.233 INFO:teuthology.orchestra.run.smithi053.stdout: "testnewline": { 2024-04-25T14:50:47.233 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "testnewline", 2024-04-25T14:50:47.234 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.234 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.234 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.234 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.234 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.234 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.234 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.234 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.234 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.235 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.235 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.235 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.235 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.235 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "snap_schedule", 2024-04-25T14:50:47.235 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.236 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.236 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.236 INFO:teuthology.orchestra.run.smithi053.stdout: "allow_m_granularity": { 2024-04-25T14:50:47.236 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "allow_m_granularity", 2024-04-25T14:50:47.236 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.236 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.236 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.236 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.236 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.237 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.237 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.237 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T14:50:47.237 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.237 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.237 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.237 INFO:teuthology.orchestra.run.smithi053.stdout: "dump_on_update": { 2024-04-25T14:50:47.238 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "dump_on_update", 2024-04-25T14:50:47.238 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.238 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.238 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.238 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.238 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.238 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.238 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.238 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "dump database to debug log on update", 2024-04-25T14:50:47.239 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.239 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.239 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.239 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.239 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.239 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.239 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.239 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.240 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.240 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.240 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.240 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.240 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.240 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.240 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.240 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.240 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.240 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.241 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.241 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.241 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.241 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.241 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.241 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.241 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.241 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.241 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.242 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.242 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.242 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.242 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.242 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.242 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.242 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.242 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.242 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.243 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.243 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.243 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.243 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.243 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.243 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.243 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.243 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.244 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.244 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.244 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.244 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.244 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.244 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.244 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.244 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.244 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.244 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.245 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.245 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.245 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.245 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.245 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.245 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.245 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.245 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.245 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.246 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.246 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.246 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.246 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.246 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.246 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.246 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.246 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.247 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "stats", 2024-04-25T14:50:47.247 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.247 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.247 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.247 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.247 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.247 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.247 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.248 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.248 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.248 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.248 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.248 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.248 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.248 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.248 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.248 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.248 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.249 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.249 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.249 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.249 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.249 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.249 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.249 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.249 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.249 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.250 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.250 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.250 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.250 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.250 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.250 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.250 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.250 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.250 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.251 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.251 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.251 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.251 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.251 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.251 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.251 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.251 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.252 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.252 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.252 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.252 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.252 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.252 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.252 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.252 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.252 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.253 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.253 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.253 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.253 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.253 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.253 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.253 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.253 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.254 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.254 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.254 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.254 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.254 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.254 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.254 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.254 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.254 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.254 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.255 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.255 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "status", 2024-04-25T14:50:47.255 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.255 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.255 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.255 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.255 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.256 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.256 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.256 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.256 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.256 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.256 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.256 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.256 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.256 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.256 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.257 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.257 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.257 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.257 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.257 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.257 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.257 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.257 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.257 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.258 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.258 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.258 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.258 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.258 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.258 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.258 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.258 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.258 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.259 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.259 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.259 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.259 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.259 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.259 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.259 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.260 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.260 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.260 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.260 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.260 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.260 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.260 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.260 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.260 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.260 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.261 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.261 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.261 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.261 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.261 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.261 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.261 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.261 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.262 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.262 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.262 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.262 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.262 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.262 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.262 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.262 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.262 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.262 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.263 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.263 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.263 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.263 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "telegraf", 2024-04-25T14:50:47.263 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.263 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.263 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.264 INFO:teuthology.orchestra.run.smithi053.stdout: "address": { 2024-04-25T14:50:47.264 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "address", 2024-04-25T14:50:47.264 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.264 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.264 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.264 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T14:50:47.264 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.264 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.264 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.264 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.265 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.265 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.265 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.265 INFO:teuthology.orchestra.run.smithi053.stdout: "interval": { 2024-04-25T14:50:47.265 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "interval", 2024-04-25T14:50:47.265 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:47.265 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.265 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.265 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "15", 2024-04-25T14:50:47.266 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.266 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.266 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.266 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.266 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.266 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.266 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.266 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.267 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.267 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.267 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.267 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.267 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.267 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.267 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.267 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.267 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.267 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.268 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.268 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.268 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.268 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.268 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.268 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.268 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.268 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.268 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.269 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.269 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.269 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.269 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.269 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.269 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.269 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.269 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.269 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.269 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.270 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.270 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.270 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.270 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.270 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.270 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.270 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.270 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.271 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.271 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.271 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.271 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.271 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.271 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.271 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.271 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.271 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.271 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.273 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.273 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.273 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.273 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.273 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.274 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.274 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.274 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.274 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.274 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.274 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.274 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.274 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.275 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.275 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.275 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.275 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.275 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.275 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.275 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.276 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "telemetry", 2024-04-25T14:50:47.276 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.276 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.276 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.276 INFO:teuthology.orchestra.run.smithi053.stdout: "channel_basic": { 2024-04-25T14:50:47.276 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "channel_basic", 2024-04-25T14:50:47.276 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.276 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.276 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.276 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.277 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.277 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.277 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.277 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T14:50:47.277 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.277 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.277 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.277 INFO:teuthology.orchestra.run.smithi053.stdout: "channel_crash": { 2024-04-25T14:50:47.278 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "channel_crash", 2024-04-25T14:50:47.278 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.278 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.278 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.278 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.278 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.278 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.278 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.279 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T14:50:47.279 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.279 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.279 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.279 INFO:teuthology.orchestra.run.smithi053.stdout: "channel_device": { 2024-04-25T14:50:47.279 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "channel_device", 2024-04-25T14:50:47.279 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.279 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.280 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.280 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.280 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.280 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.280 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.280 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T14:50:47.280 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.280 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.280 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.281 INFO:teuthology.orchestra.run.smithi053.stdout: "channel_ident": { 2024-04-25T14:50:47.281 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "channel_ident", 2024-04-25T14:50:47.281 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.281 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.281 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.281 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.281 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.281 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.282 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.282 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T14:50:47.282 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.282 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.282 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.282 INFO:teuthology.orchestra.run.smithi053.stdout: "channel_perf": { 2024-04-25T14:50:47.282 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "channel_perf", 2024-04-25T14:50:47.282 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.283 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.283 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.283 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.283 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.283 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.283 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.283 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T14:50:47.283 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.283 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.284 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.284 INFO:teuthology.orchestra.run.smithi053.stdout: "contact": { 2024-04-25T14:50:47.284 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "contact", 2024-04-25T14:50:47.284 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.284 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.284 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.284 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.284 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.284 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.285 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.285 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.285 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.285 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.285 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.285 INFO:teuthology.orchestra.run.smithi053.stdout: "description": { 2024-04-25T14:50:47.285 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "description", 2024-04-25T14:50:47.285 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.286 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.286 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.286 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.286 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.286 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.286 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.286 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.286 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.286 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.286 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.287 INFO:teuthology.orchestra.run.smithi053.stdout: "device_url": { 2024-04-25T14:50:47.287 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "device_url", 2024-04-25T14:50:47.287 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.287 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.287 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.287 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T14:50:47.287 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.287 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.288 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.288 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.288 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.288 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.288 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.288 INFO:teuthology.orchestra.run.smithi053.stdout: "enabled": { 2024-04-25T14:50:47.288 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "enabled", 2024-04-25T14:50:47.288 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.288 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.289 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.289 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.289 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.289 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.289 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.289 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.289 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.289 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.289 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.290 INFO:teuthology.orchestra.run.smithi053.stdout: "interval": { 2024-04-25T14:50:47.290 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "interval", 2024-04-25T14:50:47.290 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.290 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.290 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.290 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "24", 2024-04-25T14:50:47.290 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "8", 2024-04-25T14:50:47.290 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.290 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.291 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.291 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.291 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.291 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.291 INFO:teuthology.orchestra.run.smithi053.stdout: "last_opt_revision": { 2024-04-25T14:50:47.291 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "last_opt_revision", 2024-04-25T14:50:47.291 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.291 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.291 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.292 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "1", 2024-04-25T14:50:47.292 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.292 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.292 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.292 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.292 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.292 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.292 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.292 INFO:teuthology.orchestra.run.smithi053.stdout: "leaderboard": { 2024-04-25T14:50:47.293 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "leaderboard", 2024-04-25T14:50:47.293 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.293 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.293 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.293 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.293 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.293 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.293 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.293 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.294 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.294 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.294 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.294 INFO:teuthology.orchestra.run.smithi053.stdout: "leaderboard_description": { 2024-04-25T14:50:47.294 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "leaderboard_description", 2024-04-25T14:50:47.294 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.294 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.294 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.295 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.295 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.295 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.295 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.295 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.295 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.295 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.295 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.295 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.296 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.296 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.296 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.296 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.296 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.296 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.296 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.296 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.296 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.297 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.297 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.297 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.297 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.297 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.297 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.297 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.297 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.297 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.297 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.298 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.298 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.298 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.298 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.298 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.298 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.298 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.298 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.299 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.299 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.299 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.299 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.299 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.299 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.299 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.299 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.299 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.300 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.300 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.300 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.300 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.300 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.300 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.300 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.300 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.300 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.301 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.301 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.301 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.301 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.301 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.301 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.301 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.301 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.302 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.302 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.302 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.302 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.302 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.302 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.302 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.302 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.303 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.303 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.303 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.303 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.303 INFO:teuthology.orchestra.run.smithi053.stdout: "organization": { 2024-04-25T14:50:47.303 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "organization", 2024-04-25T14:50:47.303 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.303 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.303 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.304 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.304 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.304 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.304 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.304 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.304 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.304 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.304 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.305 INFO:teuthology.orchestra.run.smithi053.stdout: "proxy": { 2024-04-25T14:50:47.305 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "proxy", 2024-04-25T14:50:47.305 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.305 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.305 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.305 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.305 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.305 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.305 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.306 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.306 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.306 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.306 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.306 INFO:teuthology.orchestra.run.smithi053.stdout: "url": { 2024-04-25T14:50:47.306 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "url", 2024-04-25T14:50:47.306 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.306 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.307 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.307 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T14:50:47.307 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.307 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.307 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.307 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.307 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.307 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.307 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.307 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.308 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.308 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "test_orchestrator", 2024-04-25T14:50:47.308 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.308 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.308 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.308 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.308 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.309 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.309 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.309 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.309 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.309 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.309 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.309 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.309 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.309 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.310 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.310 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.310 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.310 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.310 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.310 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.310 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.310 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.311 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.311 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.311 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.311 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.311 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.311 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.311 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.311 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.312 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.312 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.312 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.312 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.312 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.312 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.312 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.312 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.313 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.313 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.313 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.313 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.313 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.313 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.313 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.313 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.313 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.314 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.314 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.314 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.314 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.314 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.314 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.314 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.314 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.314 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.315 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.315 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.315 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.315 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.315 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.315 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.315 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.316 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.316 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.316 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.316 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.316 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.316 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.316 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.316 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.317 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "volumes", 2024-04-25T14:50:47.317 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.317 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.317 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.317 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.317 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.317 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.317 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.317 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.317 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.318 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.318 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.318 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.318 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.318 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.318 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.318 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.318 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.318 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.319 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.319 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.319 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.319 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.319 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.319 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.319 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.319 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.319 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.320 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.320 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.320 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.320 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.320 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.320 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.320 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.320 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.320 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.321 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.321 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.321 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.321 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.321 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.321 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.321 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.321 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.321 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.322 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.322 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.322 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.322 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.322 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.322 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.322 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.322 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.322 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.323 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.323 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.323 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.323 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.323 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.323 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.323 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.323 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.324 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.324 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.324 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.324 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.324 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.324 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.324 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.324 INFO:teuthology.orchestra.run.smithi053.stdout: "max_concurrent_clones": { 2024-04-25T14:50:47.325 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "max_concurrent_clones", 2024-04-25T14:50:47.325 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.325 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.325 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.325 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "4", 2024-04-25T14:50:47.325 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.325 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.325 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.325 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T14:50:47.325 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.326 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.326 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.326 INFO:teuthology.orchestra.run.smithi053.stdout: "periodic_async_work": { 2024-04-25T14:50:47.326 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "periodic_async_work", 2024-04-25T14:50:47.326 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.326 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.326 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.326 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.327 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.327 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.327 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.327 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Periodically check for async work", 2024-04-25T14:50:47.327 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.327 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.327 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.327 INFO:teuthology.orchestra.run.smithi053.stdout: "snapshot_clone_delay": { 2024-04-25T14:50:47.328 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "snapshot_clone_delay", 2024-04-25T14:50:47.328 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.328 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.328 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.328 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0", 2024-04-25T14:50:47.328 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.328 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.328 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.328 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T14:50:47.329 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.329 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.329 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.329 INFO:teuthology.orchestra.run.smithi053.stdout: "snapshot_clone_no_wait": { 2024-04-25T14:50:47.329 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T14:50:47.329 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.329 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.329 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.330 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.330 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.330 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.330 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.330 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T14:50:47.330 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.330 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.330 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.331 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.331 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.331 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "zabbix", 2024-04-25T14:50:47.331 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.331 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.331 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.331 INFO:teuthology.orchestra.run.smithi053.stdout: "discovery_interval": { 2024-04-25T14:50:47.331 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "discovery_interval", 2024-04-25T14:50:47.332 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "uint", 2024-04-25T14:50:47.332 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.332 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.332 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "100", 2024-04-25T14:50:47.332 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.332 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.332 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.332 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.332 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.333 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.333 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.333 INFO:teuthology.orchestra.run.smithi053.stdout: "identifier": { 2024-04-25T14:50:47.333 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "identifier", 2024-04-25T14:50:47.333 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.333 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.333 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.334 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.334 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.334 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.334 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.334 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.334 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.334 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.334 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.335 INFO:teuthology.orchestra.run.smithi053.stdout: "interval": { 2024-04-25T14:50:47.335 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "interval", 2024-04-25T14:50:47.335 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:47.335 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.335 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.335 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "60", 2024-04-25T14:50:47.335 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.335 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.336 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.336 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.336 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.336 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.336 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.336 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.336 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.337 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.337 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.337 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.337 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.337 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.337 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.337 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.337 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.337 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.338 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.338 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.338 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.338 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.338 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.338 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.338 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.338 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.339 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.339 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.339 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.339 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.339 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.339 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.339 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.339 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.339 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.340 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.340 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.340 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.340 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.340 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.340 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.340 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.340 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.340 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.341 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.341 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.341 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.341 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.341 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.341 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.341 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.341 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.341 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.342 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.342 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.342 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.342 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.342 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.342 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.342 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.343 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.343 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.343 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.343 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.343 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.343 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.343 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.343 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.343 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.344 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.344 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.344 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.344 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.344 INFO:teuthology.orchestra.run.smithi053.stdout: "zabbix_host": { 2024-04-25T14:50:47.344 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "zabbix_host", 2024-04-25T14:50:47.344 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.344 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.345 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.345 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.345 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.345 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.345 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.345 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.345 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.345 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.345 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.346 INFO:teuthology.orchestra.run.smithi053.stdout: "zabbix_port": { 2024-04-25T14:50:47.346 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "zabbix_port", 2024-04-25T14:50:47.346 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.346 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.346 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.346 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10051", 2024-04-25T14:50:47.346 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.347 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.347 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.347 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.347 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.347 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.347 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.347 INFO:teuthology.orchestra.run.smithi053.stdout: "zabbix_sender": { 2024-04-25T14:50:47.347 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "zabbix_sender", 2024-04-25T14:50:47.348 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.348 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.348 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.348 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T14:50:47.348 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.348 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.348 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.348 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.348 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.349 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.349 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.349 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.349 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.349 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.349 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-04-25T14:50:47.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.349 INFO:teuthology.orchestra.run.smithi053.stdout: "gid": 25225, 2024-04-25T14:50:47.349 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "z", 2024-04-25T14:50:47.350 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_features": 4540701547738038271, 2024-04-25T14:50:47.350 INFO:teuthology.orchestra.run.smithi053.stdout: "available_modules": [ 2024-04-25T14:50:47.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.350 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "alerts", 2024-04-25T14:50:47.350 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.350 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.350 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.350 INFO:teuthology.orchestra.run.smithi053.stdout: "interval": { 2024-04-25T14:50:47.351 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "interval", 2024-04-25T14:50:47.351 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:47.351 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.351 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.351 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "60", 2024-04-25T14:50:47.351 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.351 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.351 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.351 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T14:50:47.351 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.352 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.352 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.352 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.352 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.352 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.352 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.352 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.352 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.353 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.353 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.353 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.353 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.353 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.353 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.353 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.353 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.353 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.353 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.354 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.354 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.354 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.354 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.354 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.354 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.354 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.355 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.355 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.355 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.355 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.355 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.355 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.355 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.355 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.355 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.356 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.356 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.356 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.356 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.356 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.356 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.356 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.356 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.357 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.357 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.357 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.357 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.357 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.357 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.357 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.357 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.358 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.358 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.358 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.358 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.358 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.358 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.358 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.359 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.359 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.359 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.359 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.359 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.359 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.359 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.359 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.359 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.360 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.360 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.360 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_destination": { 2024-04-25T14:50:47.360 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_destination", 2024-04-25T14:50:47.360 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.360 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.360 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.361 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.361 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.361 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.361 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.361 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Email address to send alerts to", 2024-04-25T14:50:47.361 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.361 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.361 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.362 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_from_name": { 2024-04-25T14:50:47.362 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_from_name", 2024-04-25T14:50:47.362 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.362 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.362 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.362 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "Ceph", 2024-04-25T14:50:47.362 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.362 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.363 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.363 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Email From: name", 2024-04-25T14:50:47.363 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.363 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.363 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.363 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_host": { 2024-04-25T14:50:47.363 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_host", 2024-04-25T14:50:47.363 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.364 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.364 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.364 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.364 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.364 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.364 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.364 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "SMTP server", 2024-04-25T14:50:47.364 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.364 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.365 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.365 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_password": { 2024-04-25T14:50:47.365 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_password", 2024-04-25T14:50:47.365 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.365 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.365 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.365 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.365 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.366 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.366 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.366 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Password to authenticate with", 2024-04-25T14:50:47.366 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.366 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.366 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.367 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_port": { 2024-04-25T14:50:47.367 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_port", 2024-04-25T14:50:47.367 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.367 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.367 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.367 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "465", 2024-04-25T14:50:47.367 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.367 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.367 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.368 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "SMTP port", 2024-04-25T14:50:47.368 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.368 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.368 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.368 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_sender": { 2024-04-25T14:50:47.368 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_sender", 2024-04-25T14:50:47.368 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.368 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.368 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.369 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.369 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.369 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.369 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.369 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "SMTP envelope sender", 2024-04-25T14:50:47.369 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.369 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.369 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.370 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_ssl": { 2024-04-25T14:50:47.370 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_ssl", 2024-04-25T14:50:47.370 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.370 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.370 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.370 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.370 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.370 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.370 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.371 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T14:50:47.371 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.371 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.371 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.371 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_user": { 2024-04-25T14:50:47.371 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_user", 2024-04-25T14:50:47.371 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.371 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.371 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.372 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.372 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.372 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.372 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.372 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "User to authenticate as", 2024-04-25T14:50:47.372 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.372 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.372 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.372 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.372 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.373 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "balancer", 2024-04-25T14:50:47.373 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.373 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.373 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.373 INFO:teuthology.orchestra.run.smithi053.stdout: "active": { 2024-04-25T14:50:47.373 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "active", 2024-04-25T14:50:47.373 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.374 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.374 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.374 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.374 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.374 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.374 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.374 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T14:50:47.374 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.374 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.374 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.375 INFO:teuthology.orchestra.run.smithi053.stdout: "begin_time": { 2024-04-25T14:50:47.375 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "begin_time", 2024-04-25T14:50:47.375 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.375 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.375 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.375 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0000", 2024-04-25T14:50:47.375 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.375 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.376 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.376 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T14:50:47.376 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T14:50:47.376 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.376 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.376 INFO:teuthology.orchestra.run.smithi053.stdout: "begin_weekday": { 2024-04-25T14:50:47.376 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "begin_weekday", 2024-04-25T14:50:47.376 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "uint", 2024-04-25T14:50:47.376 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.377 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.377 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0", 2024-04-25T14:50:47.377 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "0", 2024-04-25T14:50:47.377 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "6", 2024-04-25T14:50:47.377 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.377 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T14:50:47.377 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T14:50:47.377 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.377 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.378 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_compat_max_iterations": { 2024-04-25T14:50:47.378 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "crush_compat_max_iterations", 2024-04-25T14:50:47.378 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "uint", 2024-04-25T14:50:47.378 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.378 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.378 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "25", 2024-04-25T14:50:47.378 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "1", 2024-04-25T14:50:47.378 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "250", 2024-04-25T14:50:47.379 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.379 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T14:50:47.379 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.379 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.379 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.379 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_compat_metrics": { 2024-04-25T14:50:47.379 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "crush_compat_metrics", 2024-04-25T14:50:47.379 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.379 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.380 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.380 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T14:50:47.380 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.380 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.380 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.380 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T14:50:47.380 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T14:50:47.380 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.380 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.381 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_compat_step": { 2024-04-25T14:50:47.381 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "crush_compat_step", 2024-04-25T14:50:47.381 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2024-04-25T14:50:47.381 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.381 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.381 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0.5", 2024-04-25T14:50:47.381 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "0.001", 2024-04-25T14:50:47.381 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "0.999", 2024-04-25T14:50:47.381 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.382 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "aggressiveness of optimization", 2024-04-25T14:50:47.382 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T14:50:47.382 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.382 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.382 INFO:teuthology.orchestra.run.smithi053.stdout: "end_time": { 2024-04-25T14:50:47.382 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "end_time", 2024-04-25T14:50:47.382 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.382 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.383 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.383 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "2359", 2024-04-25T14:50:47.383 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.383 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.383 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.383 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T14:50:47.383 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T14:50:47.383 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.383 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.384 INFO:teuthology.orchestra.run.smithi053.stdout: "end_weekday": { 2024-04-25T14:50:47.384 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "end_weekday", 2024-04-25T14:50:47.384 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "uint", 2024-04-25T14:50:47.384 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.384 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.384 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0", 2024-04-25T14:50:47.384 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "0", 2024-04-25T14:50:47.384 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "6", 2024-04-25T14:50:47.384 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.385 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T14:50:47.385 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T14:50:47.385 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.385 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.385 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.385 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.385 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.385 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.386 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.386 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.386 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.386 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.386 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.386 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.386 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.386 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.386 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.387 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.387 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.387 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.387 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.387 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.387 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.387 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.387 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.388 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.388 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.388 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.388 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.388 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.388 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.388 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.388 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.388 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.388 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.389 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.389 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.389 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.389 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.389 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.389 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.389 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.389 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.389 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.390 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.390 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.390 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.390 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.390 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.390 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.390 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.390 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.390 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.391 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.391 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.391 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.391 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.391 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.391 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.391 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.391 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.391 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.392 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.392 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.392 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.392 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.392 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.392 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.392 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.392 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.392 INFO:teuthology.orchestra.run.smithi053.stdout: "min_score": { 2024-04-25T14:50:47.393 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "min_score", 2024-04-25T14:50:47.393 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2024-04-25T14:50:47.393 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.393 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.393 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0", 2024-04-25T14:50:47.393 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.393 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.393 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.393 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T14:50:47.393 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.394 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.394 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.394 INFO:teuthology.orchestra.run.smithi053.stdout: "mode": { 2024-04-25T14:50:47.394 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "mode", 2024-04-25T14:50:47.394 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.394 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.394 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.394 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "upmap", 2024-04-25T14:50:47.395 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.395 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.395 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.395 INFO:teuthology.orchestra.run.smithi053.stdout: "crush-compat", 2024-04-25T14:50:47.395 INFO:teuthology.orchestra.run.smithi053.stdout: "none", 2024-04-25T14:50:47.395 INFO:teuthology.orchestra.run.smithi053.stdout: "read", 2024-04-25T14:50:47.395 INFO:teuthology.orchestra.run.smithi053.stdout: "upmap", 2024-04-25T14:50:47.395 INFO:teuthology.orchestra.run.smithi053.stdout: "upmap-read" 2024-04-25T14:50:47.395 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.395 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Balancer mode", 2024-04-25T14:50:47.396 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.396 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.396 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.396 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_ids": { 2024-04-25T14:50:47.396 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "pool_ids", 2024-04-25T14:50:47.396 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.396 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.396 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.396 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.397 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.397 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.397 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.397 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T14:50:47.397 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.397 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.397 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.397 INFO:teuthology.orchestra.run.smithi053.stdout: "sleep_interval": { 2024-04-25T14:50:47.398 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "sleep_interval", 2024-04-25T14:50:47.398 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:47.398 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.398 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.398 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "60", 2024-04-25T14:50:47.398 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.398 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.398 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.398 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T14:50:47.399 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.399 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.399 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.399 INFO:teuthology.orchestra.run.smithi053.stdout: "upmap_max_deviation": { 2024-04-25T14:50:47.399 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "upmap_max_deviation", 2024-04-25T14:50:47.399 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.399 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.399 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.399 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "5", 2024-04-25T14:50:47.400 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "1", 2024-04-25T14:50:47.400 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.400 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.400 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T14:50:47.400 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T14:50:47.400 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.400 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.400 INFO:teuthology.orchestra.run.smithi053.stdout: "upmap_max_optimizations": { 2024-04-25T14:50:47.401 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "upmap_max_optimizations", 2024-04-25T14:50:47.401 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "uint", 2024-04-25T14:50:47.401 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.401 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.401 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2024-04-25T14:50:47.401 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.401 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.401 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.401 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T14:50:47.402 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.402 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.402 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.402 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.402 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.402 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "cephadm", 2024-04-25T14:50:47.402 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.403 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.403 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.403 INFO:teuthology.orchestra.run.smithi053.stdout: "agent_down_multiplier": { 2024-04-25T14:50:47.403 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "agent_down_multiplier", 2024-04-25T14:50:47.403 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2024-04-25T14:50:47.403 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.403 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.403 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "3.0", 2024-04-25T14:50:47.403 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.403 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.404 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.404 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T14:50:47.404 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.404 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.404 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.404 INFO:teuthology.orchestra.run.smithi053.stdout: "agent_refresh_rate": { 2024-04-25T14:50:47.404 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "agent_refresh_rate", 2024-04-25T14:50:47.404 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:47.405 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.405 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.405 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "20", 2024-04-25T14:50:47.405 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.405 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.405 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.405 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T14:50:47.405 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.405 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.406 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.406 INFO:teuthology.orchestra.run.smithi053.stdout: "agent_starting_port": { 2024-04-25T14:50:47.406 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "agent_starting_port", 2024-04-25T14:50:47.406 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.406 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.406 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.406 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "4721", 2024-04-25T14:50:47.406 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.406 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.407 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.407 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T14:50:47.407 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.407 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.407 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.407 INFO:teuthology.orchestra.run.smithi053.stdout: "allow_ptrace": { 2024-04-25T14:50:47.407 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "allow_ptrace", 2024-04-25T14:50:47.407 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.408 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.408 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.408 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.408 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.408 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.408 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.408 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T14:50:47.408 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T14:50:47.408 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.409 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.409 INFO:teuthology.orchestra.run.smithi053.stdout: "autotune_interval": { 2024-04-25T14:50:47.409 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "autotune_interval", 2024-04-25T14:50:47.409 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:47.409 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.409 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.409 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "600", 2024-04-25T14:50:47.409 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.409 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.410 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.410 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T14:50:47.410 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.410 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.410 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.410 INFO:teuthology.orchestra.run.smithi053.stdout: "autotune_memory_target_ratio": { 2024-04-25T14:50:47.410 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T14:50:47.410 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2024-04-25T14:50:47.411 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.411 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.411 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0.7", 2024-04-25T14:50:47.411 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.411 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.411 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.411 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T14:50:47.411 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.411 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.411 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.412 INFO:teuthology.orchestra.run.smithi053.stdout: "cephadm_log_destination": { 2024-04-25T14:50:47.412 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "cephadm_log_destination", 2024-04-25T14:50:47.412 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.412 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.412 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.412 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.412 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.412 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.413 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.413 INFO:teuthology.orchestra.run.smithi053.stdout: "file", 2024-04-25T14:50:47.413 INFO:teuthology.orchestra.run.smithi053.stdout: "file,syslog", 2024-04-25T14:50:47.413 INFO:teuthology.orchestra.run.smithi053.stdout: "syslog" 2024-04-25T14:50:47.413 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.413 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T14:50:47.413 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.413 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.413 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.414 INFO:teuthology.orchestra.run.smithi053.stdout: "cgroups_split": { 2024-04-25T14:50:47.414 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "cgroups_split", 2024-04-25T14:50:47.414 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.414 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.414 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.414 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.414 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.415 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.415 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.415 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T14:50:47.415 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.415 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.415 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.415 INFO:teuthology.orchestra.run.smithi053.stdout: "config_checks_enabled": { 2024-04-25T14:50:47.415 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "config_checks_enabled", 2024-04-25T14:50:47.416 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.416 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.416 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.416 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.416 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.416 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.416 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.416 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T14:50:47.416 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.417 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.417 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.417 INFO:teuthology.orchestra.run.smithi053.stdout: "config_dashboard": { 2024-04-25T14:50:47.417 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "config_dashboard", 2024-04-25T14:50:47.417 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.417 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.417 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.417 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.418 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.418 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.418 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.418 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T14:50:47.418 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.418 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.418 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.418 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_alertmanager": { 2024-04-25T14:50:47.419 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_alertmanager", 2024-04-25T14:50:47.419 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.419 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.419 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.419 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T14:50:47.419 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.419 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.419 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.419 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Prometheus container image", 2024-04-25T14:50:47.420 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.420 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.420 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.420 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_base": { 2024-04-25T14:50:47.420 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_base", 2024-04-25T14:50:47.420 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.420 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.420 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.421 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T14:50:47.421 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.421 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.421 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.421 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Container image name, without the tag", 2024-04-25T14:50:47.421 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.421 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.421 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.422 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_elasticsearch": { 2024-04-25T14:50:47.422 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_elasticsearch", 2024-04-25T14:50:47.422 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.422 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.422 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.422 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T14:50:47.422 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.422 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.423 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.423 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "elasticsearch container image", 2024-04-25T14:50:47.423 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.423 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.423 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.423 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_grafana": { 2024-04-25T14:50:47.423 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_grafana", 2024-04-25T14:50:47.423 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.423 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.424 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.424 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T14:50:47.424 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.424 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.424 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.424 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Prometheus container image", 2024-04-25T14:50:47.424 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.424 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.424 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.425 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_haproxy": { 2024-04-25T14:50:47.425 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_haproxy", 2024-04-25T14:50:47.425 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.425 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.425 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.425 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T14:50:47.425 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.425 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.426 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.426 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "HAproxy container image", 2024-04-25T14:50:47.426 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.426 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.426 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.426 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_jaeger_agent": { 2024-04-25T14:50:47.427 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_jaeger_agent", 2024-04-25T14:50:47.427 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.427 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.427 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.427 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T14:50:47.427 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.427 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.427 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.427 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Jaeger agent container image", 2024-04-25T14:50:47.427 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.428 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.428 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.428 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_jaeger_collector": { 2024-04-25T14:50:47.428 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_jaeger_collector", 2024-04-25T14:50:47.428 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.428 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.428 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.428 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T14:50:47.428 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.429 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.429 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.429 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Jaeger collector container image", 2024-04-25T14:50:47.429 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.429 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.429 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.429 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_jaeger_query": { 2024-04-25T14:50:47.429 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_jaeger_query", 2024-04-25T14:50:47.429 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.430 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.430 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.430 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T14:50:47.430 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.430 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.430 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.430 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Jaeger query container image", 2024-04-25T14:50:47.430 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.430 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.431 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.431 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_keepalived": { 2024-04-25T14:50:47.431 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_keepalived", 2024-04-25T14:50:47.431 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.431 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.431 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.431 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T14:50:47.431 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.432 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.432 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.432 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Keepalived container image", 2024-04-25T14:50:47.432 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.432 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.432 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.432 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_loki": { 2024-04-25T14:50:47.432 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_loki", 2024-04-25T14:50:47.432 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.433 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.433 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.433 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T14:50:47.433 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.433 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.433 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.433 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Loki container image", 2024-04-25T14:50:47.433 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.433 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.433 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.434 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_node_exporter": { 2024-04-25T14:50:47.434 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_node_exporter", 2024-04-25T14:50:47.434 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.434 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.434 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.434 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T14:50:47.434 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.434 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.435 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.435 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Prometheus container image", 2024-04-25T14:50:47.435 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.435 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.435 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.435 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_nvmeof": { 2024-04-25T14:50:47.435 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_nvmeof", 2024-04-25T14:50:47.435 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.435 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.436 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.436 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T14:50:47.436 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.436 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.436 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.436 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Nvme-of container image", 2024-04-25T14:50:47.436 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.436 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.436 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.437 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_prometheus": { 2024-04-25T14:50:47.437 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_prometheus", 2024-04-25T14:50:47.437 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.437 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.437 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.437 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T14:50:47.437 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.437 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.438 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.438 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Prometheus container image", 2024-04-25T14:50:47.438 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.438 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.438 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.438 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_promtail": { 2024-04-25T14:50:47.438 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_promtail", 2024-04-25T14:50:47.438 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.439 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.439 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.439 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T14:50:47.439 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.439 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.439 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.439 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Promtail container image", 2024-04-25T14:50:47.439 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.439 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.440 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.440 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_samba": { 2024-04-25T14:50:47.440 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_samba", 2024-04-25T14:50:47.440 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.440 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.440 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.440 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T14:50:47.440 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.440 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.441 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.441 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Samba/SMB container image", 2024-04-25T14:50:47.441 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.441 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.441 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.441 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_snmp_gateway": { 2024-04-25T14:50:47.441 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_snmp_gateway", 2024-04-25T14:50:47.441 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.442 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.442 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.442 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T14:50:47.442 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.442 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.442 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.442 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "SNMP Gateway container image", 2024-04-25T14:50:47.442 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.442 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.443 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.443 INFO:teuthology.orchestra.run.smithi053.stdout: "container_init": { 2024-04-25T14:50:47.443 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_init", 2024-04-25T14:50:47.443 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.443 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.443 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.443 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.443 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.444 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.444 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.444 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T14:50:47.444 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.444 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.444 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.444 INFO:teuthology.orchestra.run.smithi053.stdout: "daemon_cache_timeout": { 2024-04-25T14:50:47.444 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "daemon_cache_timeout", 2024-04-25T14:50:47.444 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:47.445 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.445 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.445 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "600", 2024-04-25T14:50:47.445 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.445 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.445 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.445 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T14:50:47.445 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.445 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.446 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.446 INFO:teuthology.orchestra.run.smithi053.stdout: "default_cephadm_command_timeout": { 2024-04-25T14:50:47.446 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T14:50:47.446 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.446 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.446 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.446 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "900", 2024-04-25T14:50:47.446 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.447 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.447 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.447 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T14:50:47.447 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.447 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.447 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.447 INFO:teuthology.orchestra.run.smithi053.stdout: "default_registry": { 2024-04-25T14:50:47.448 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "default_registry", 2024-04-25T14:50:47.448 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.448 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.448 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.448 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "docker.io", 2024-04-25T14:50:47.448 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.448 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.448 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.448 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T14:50:47.449 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.449 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.449 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.449 INFO:teuthology.orchestra.run.smithi053.stdout: "device_cache_timeout": { 2024-04-25T14:50:47.449 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "device_cache_timeout", 2024-04-25T14:50:47.449 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:47.449 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.450 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.450 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "1800", 2024-04-25T14:50:47.450 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.450 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.450 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.450 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "seconds to cache device inventory", 2024-04-25T14:50:47.450 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.450 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.450 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.451 INFO:teuthology.orchestra.run.smithi053.stdout: "device_enhanced_scan": { 2024-04-25T14:50:47.451 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "device_enhanced_scan", 2024-04-25T14:50:47.451 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.451 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.451 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.451 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.451 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.451 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.452 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.452 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T14:50:47.452 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.452 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.452 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.452 INFO:teuthology.orchestra.run.smithi053.stdout: "facts_cache_timeout": { 2024-04-25T14:50:47.452 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "facts_cache_timeout", 2024-04-25T14:50:47.452 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:47.453 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.453 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.453 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "60", 2024-04-25T14:50:47.453 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.453 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.453 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.453 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "seconds to cache host facts data", 2024-04-25T14:50:47.453 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.453 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.454 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.454 INFO:teuthology.orchestra.run.smithi053.stdout: "grafana_dashboards_path": { 2024-04-25T14:50:47.454 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "grafana_dashboards_path", 2024-04-25T14:50:47.454 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.454 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.454 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.454 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T14:50:47.455 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.455 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.455 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.455 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T14:50:47.455 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.455 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.455 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.455 INFO:teuthology.orchestra.run.smithi053.stdout: "host_check_interval": { 2024-04-25T14:50:47.455 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "host_check_interval", 2024-04-25T14:50:47.456 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:47.456 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.456 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.456 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "600", 2024-04-25T14:50:47.456 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.456 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.456 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.456 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how frequently to perform a host check", 2024-04-25T14:50:47.456 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.456 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.457 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.457 INFO:teuthology.orchestra.run.smithi053.stdout: "hw_monitoring": { 2024-04-25T14:50:47.457 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "hw_monitoring", 2024-04-25T14:50:47.457 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.457 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.457 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.457 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.457 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.457 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.458 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.458 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T14:50:47.458 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.458 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.458 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.458 INFO:teuthology.orchestra.run.smithi053.stdout: "inventory_list_all": { 2024-04-25T14:50:47.458 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "inventory_list_all", 2024-04-25T14:50:47.458 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.459 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.459 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.459 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.459 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.459 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.459 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.459 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T14:50:47.459 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.459 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.460 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.460 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.460 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.460 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.460 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.460 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.460 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.461 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.461 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.461 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.461 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.461 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.461 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.461 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.461 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.461 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.462 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.462 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.462 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.462 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.462 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.462 INFO:teuthology.orchestra.run.smithi053.stdout: "log_refresh_metadata": { 2024-04-25T14:50:47.462 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_refresh_metadata", 2024-04-25T14:50:47.462 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.463 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.463 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.463 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.463 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.463 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.463 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.463 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T14:50:47.463 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.463 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.463 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.464 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.464 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.464 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.464 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.464 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.464 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.464 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.464 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.465 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.465 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T14:50:47.465 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.465 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.465 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.465 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.465 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.465 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.466 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.466 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.466 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.466 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.466 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.466 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.466 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.466 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.466 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.467 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.467 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.467 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.467 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.467 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.467 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.467 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.467 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.467 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.468 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.468 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.468 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.468 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.468 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.468 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.468 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.468 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.468 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.469 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.469 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.469 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.469 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.469 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.469 INFO:teuthology.orchestra.run.smithi053.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T14:50:47.469 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T14:50:47.469 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.469 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.470 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.470 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.470 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.470 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.470 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.470 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T14:50:47.470 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.470 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.470 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.471 INFO:teuthology.orchestra.run.smithi053.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T14:50:47.471 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T14:50:47.471 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.471 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.471 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.471 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "*", 2024-04-25T14:50:47.471 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.471 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.471 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.472 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T14:50:47.472 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.472 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.472 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.472 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.472 INFO:teuthology.orchestra.run.smithi053.stdout: "max_count_per_host": { 2024-04-25T14:50:47.472 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "max_count_per_host", 2024-04-25T14:50:47.472 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.472 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.473 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.473 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2024-04-25T14:50:47.473 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.473 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.473 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.473 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "max number of daemons per service per host", 2024-04-25T14:50:47.473 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.473 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.473 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.474 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.474 INFO:teuthology.orchestra.run.smithi053.stdout: "max_osd_draining_count": { 2024-04-25T14:50:47.474 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "max_osd_draining_count", 2024-04-25T14:50:47.474 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.474 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.474 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.474 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2024-04-25T14:50:47.474 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.474 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.474 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.475 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T14:50:47.475 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.475 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.475 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.475 INFO:teuthology.orchestra.run.smithi053.stdout: "migration_current": { 2024-04-25T14:50:47.475 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "migration_current", 2024-04-25T14:50:47.475 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.475 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.476 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.476 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.476 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.476 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.476 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.476 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "internal - do not modify", 2024-04-25T14:50:47.476 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.476 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.476 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.476 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.477 INFO:teuthology.orchestra.run.smithi053.stdout: "mode": { 2024-04-25T14:50:47.477 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "mode", 2024-04-25T14:50:47.477 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.477 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.477 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.477 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "root", 2024-04-25T14:50:47.477 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.477 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.477 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.477 INFO:teuthology.orchestra.run.smithi053.stdout: "cephadm-package", 2024-04-25T14:50:47.478 INFO:teuthology.orchestra.run.smithi053.stdout: "root" 2024-04-25T14:50:47.478 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.478 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T14:50:47.478 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.478 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.478 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.478 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.478 INFO:teuthology.orchestra.run.smithi053.stdout: "oob_default_addr": { 2024-04-25T14:50:47.478 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "oob_default_addr", 2024-04-25T14:50:47.479 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.479 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.479 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.479 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "169.254.1.1", 2024-04-25T14:50:47.479 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.479 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.481 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.481 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T14:50:47.481 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.481 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.481 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.481 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.481 INFO:teuthology.orchestra.run.smithi053.stdout: "prometheus_alerts_path": { 2024-04-25T14:50:47.481 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "prometheus_alerts_path", 2024-04-25T14:50:47.481 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.482 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.482 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.482 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T14:50:47.482 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.482 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.482 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.482 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T14:50:47.482 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.482 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.482 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.483 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.483 INFO:teuthology.orchestra.run.smithi053.stdout: "registry_insecure": { 2024-04-25T14:50:47.483 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "registry_insecure", 2024-04-25T14:50:47.483 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.483 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.483 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.483 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.483 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.483 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.484 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.484 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T14:50:47.484 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.484 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.484 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.484 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.484 INFO:teuthology.orchestra.run.smithi053.stdout: "registry_password": { 2024-04-25T14:50:47.484 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "registry_password", 2024-04-25T14:50:47.484 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.484 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.485 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.485 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.485 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.485 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.485 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.485 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T14:50:47.485 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.485 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.485 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.485 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.486 INFO:teuthology.orchestra.run.smithi053.stdout: "registry_url": { 2024-04-25T14:50:47.486 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "registry_url", 2024-04-25T14:50:47.486 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.486 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.486 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.486 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.486 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.486 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.486 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.487 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T14:50:47.487 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.487 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.487 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.487 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.487 INFO:teuthology.orchestra.run.smithi053.stdout: "registry_username": { 2024-04-25T14:50:47.487 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "registry_username", 2024-04-25T14:50:47.487 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.487 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.487 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.488 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.488 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.488 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.488 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.488 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T14:50:47.488 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.488 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.488 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.488 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.488 INFO:teuthology.orchestra.run.smithi053.stdout: "secure_monitoring_stack": { 2024-04-25T14:50:47.489 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "secure_monitoring_stack", 2024-04-25T14:50:47.489 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.489 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.489 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.489 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.489 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.489 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.489 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.489 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T14:50:47.489 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.490 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.490 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.490 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.490 INFO:teuthology.orchestra.run.smithi053.stdout: "service_discovery_port": { 2024-04-25T14:50:47.490 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "service_discovery_port", 2024-04-25T14:50:47.490 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.490 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.490 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.490 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "8765", 2024-04-25T14:50:47.491 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.491 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.491 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.491 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "cephadm service discovery port", 2024-04-25T14:50:47.491 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.491 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.491 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.491 INFO:teuthology.orchestra.run.smithi053.stdout: "ssh_config_file": { 2024-04-25T14:50:47.492 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ssh_config_file", 2024-04-25T14:50:47.492 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.492 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.492 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.492 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.492 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.492 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.492 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.492 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T14:50:47.493 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.493 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.493 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.493 INFO:teuthology.orchestra.run.smithi053.stdout: "use_agent": { 2024-04-25T14:50:47.493 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "use_agent", 2024-04-25T14:50:47.493 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.493 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.493 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.493 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.494 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.494 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.494 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.494 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T14:50:47.494 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.494 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.494 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.494 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.494 INFO:teuthology.orchestra.run.smithi053.stdout: "use_repo_digest": { 2024-04-25T14:50:47.495 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "use_repo_digest", 2024-04-25T14:50:47.495 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.495 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.495 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.495 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.495 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.495 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.495 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.495 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T14:50:47.496 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.496 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.496 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.496 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.496 INFO:teuthology.orchestra.run.smithi053.stdout: "warn_on_failed_host_check": { 2024-04-25T14:50:47.496 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "warn_on_failed_host_check", 2024-04-25T14:50:47.496 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.496 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.496 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.496 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.497 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.497 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.497 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.497 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T14:50:47.497 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.497 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.497 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.497 INFO:teuthology.orchestra.run.smithi053.stdout: "warn_on_stray_daemons": { 2024-04-25T14:50:47.497 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "warn_on_stray_daemons", 2024-04-25T14:50:47.498 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.498 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.498 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.498 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.498 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.498 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.498 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.498 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T14:50:47.499 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.499 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.499 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.499 INFO:teuthology.orchestra.run.smithi053.stdout: "warn_on_stray_hosts": { 2024-04-25T14:50:47.499 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "warn_on_stray_hosts", 2024-04-25T14:50:47.499 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.499 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.499 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.499 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.500 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.500 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.500 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.500 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T14:50:47.500 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.500 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.500 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.500 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.500 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.501 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.501 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "crash", 2024-04-25T14:50:47.501 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.501 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.501 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.501 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.501 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.501 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.501 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.502 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.502 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.502 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.502 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.502 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.502 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.502 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.502 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.502 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.503 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.503 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.503 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.503 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.503 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.503 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.503 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.503 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.503 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.504 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.504 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.504 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.504 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.504 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.504 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.504 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.504 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.504 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.505 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.505 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.505 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.505 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.505 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.505 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.505 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.505 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.505 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.506 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.506 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.506 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.506 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.506 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.506 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.506 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.506 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.506 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.507 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.507 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.507 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.507 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.507 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.507 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.507 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.507 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.507 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.508 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.508 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.508 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.508 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.508 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.508 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.508 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.508 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.508 INFO:teuthology.orchestra.run.smithi053.stdout: "retain_interval": { 2024-04-25T14:50:47.509 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "retain_interval", 2024-04-25T14:50:47.509 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:47.509 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.509 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.509 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "31536000", 2024-04-25T14:50:47.509 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.509 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.509 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.509 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T14:50:47.510 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.510 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.510 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.510 INFO:teuthology.orchestra.run.smithi053.stdout: "warn_recent_interval": { 2024-04-25T14:50:47.510 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "warn_recent_interval", 2024-04-25T14:50:47.510 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:47.510 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.510 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.511 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "1209600", 2024-04-25T14:50:47.511 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.511 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.511 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.511 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T14:50:47.511 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.511 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.511 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.511 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.511 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.512 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "dashboard", 2024-04-25T14:50:47.512 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.512 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.512 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.512 INFO:teuthology.orchestra.run.smithi053.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T14:50:47.512 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T14:50:47.512 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.513 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.513 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.513 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2024-04-25T14:50:47.513 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.513 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.513 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.513 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.513 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.513 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.514 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.514 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.514 INFO:teuthology.orchestra.run.smithi053.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T14:50:47.514 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T14:50:47.514 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.514 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.514 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.514 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.514 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.515 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.515 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.515 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.515 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.515 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.515 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.515 INFO:teuthology.orchestra.run.smithi053.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T14:50:47.515 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T14:50:47.515 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.516 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.516 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.516 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.516 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.516 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.516 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.516 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.516 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.516 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.517 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.517 INFO:teuthology.orchestra.run.smithi053.stdout: "AUDIT_API_ENABLED": { 2024-04-25T14:50:47.517 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T14:50:47.517 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.517 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.517 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.517 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.517 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.518 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.518 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.518 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.518 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.518 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.518 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.518 INFO:teuthology.orchestra.run.smithi053.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T14:50:47.518 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T14:50:47.519 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.519 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.519 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.519 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.519 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.519 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.519 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.519 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.519 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.519 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.520 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.520 INFO:teuthology.orchestra.run.smithi053.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T14:50:47.520 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T14:50:47.520 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.520 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.520 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.520 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.520 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.520 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.521 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.521 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.521 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.521 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.521 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.521 INFO:teuthology.orchestra.run.smithi053.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T14:50:47.521 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T14:50:47.521 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.522 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.522 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.522 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.522 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.522 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.522 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.522 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.522 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.522 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.523 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.523 INFO:teuthology.orchestra.run.smithi053.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T14:50:47.523 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T14:50:47.523 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.523 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.523 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.523 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.523 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.524 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.524 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.524 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.524 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.524 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.524 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.524 INFO:teuthology.orchestra.run.smithi053.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T14:50:47.524 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T14:50:47.524 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.525 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.525 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.525 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.525 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.525 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.525 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.525 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.525 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.525 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.526 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.526 INFO:teuthology.orchestra.run.smithi053.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T14:50:47.526 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T14:50:47.526 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.526 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.526 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.526 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.526 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.527 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.527 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.527 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.527 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.527 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.527 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.527 INFO:teuthology.orchestra.run.smithi053.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T14:50:47.527 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T14:50:47.527 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.528 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.528 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.528 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.528 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.528 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.528 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.528 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.528 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.528 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.528 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.529 INFO:teuthology.orchestra.run.smithi053.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T14:50:47.529 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T14:50:47.529 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.529 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.529 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.529 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.529 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.529 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.530 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.530 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.530 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.530 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.530 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.530 INFO:teuthology.orchestra.run.smithi053.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T14:50:47.530 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T14:50:47.530 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.531 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.531 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.531 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.531 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.531 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.531 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.531 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.531 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.531 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.531 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.532 INFO:teuthology.orchestra.run.smithi053.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T14:50:47.532 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T14:50:47.532 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.532 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.532 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.532 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.532 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.532 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.532 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.533 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.533 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.533 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.533 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.533 INFO:teuthology.orchestra.run.smithi053.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T14:50:47.533 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T14:50:47.533 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.533 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.534 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.534 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "admin", 2024-04-25T14:50:47.534 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.534 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.534 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.534 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.534 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.534 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.534 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.535 INFO:teuthology.orchestra.run.smithi053.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T14:50:47.535 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T14:50:47.535 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.535 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.535 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.535 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.535 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.535 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.535 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.536 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.536 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.536 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.536 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.536 INFO:teuthology.orchestra.run.smithi053.stdout: "GRAFANA_API_URL": { 2024-04-25T14:50:47.536 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "GRAFANA_API_URL", 2024-04-25T14:50:47.536 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.536 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.536 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.537 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.537 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.537 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.537 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.537 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.537 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.537 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.537 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.538 INFO:teuthology.orchestra.run.smithi053.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T14:50:47.538 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T14:50:47.538 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.538 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.538 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.538 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "admin", 2024-04-25T14:50:47.538 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.538 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.538 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.539 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.539 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.539 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.539 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.539 INFO:teuthology.orchestra.run.smithi053.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T14:50:47.539 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T14:50:47.539 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.540 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.540 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.540 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.540 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.540 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.540 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.540 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.540 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.540 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.540 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.541 INFO:teuthology.orchestra.run.smithi053.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T14:50:47.541 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T14:50:47.541 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.541 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.541 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.541 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.541 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.541 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.541 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.542 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.542 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.542 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.542 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.542 INFO:teuthology.orchestra.run.smithi053.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T14:50:47.542 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T14:50:47.542 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.542 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.543 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.543 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.543 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.543 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.543 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.543 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.543 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.543 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.543 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.544 INFO:teuthology.orchestra.run.smithi053.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T14:50:47.544 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T14:50:47.544 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.544 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.544 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.544 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.544 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.544 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.544 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.545 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.545 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.545 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.545 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.545 INFO:teuthology.orchestra.run.smithi053.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T14:50:47.545 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T14:50:47.545 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.545 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.545 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.546 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "[]", 2024-04-25T14:50:47.546 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.546 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.546 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.546 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.546 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.546 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.546 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.547 INFO:teuthology.orchestra.run.smithi053.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T14:50:47.547 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T14:50:47.547 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.547 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.547 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.547 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "{}", 2024-04-25T14:50:47.547 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.547 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.547 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.548 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.548 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.548 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.548 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.548 INFO:teuthology.orchestra.run.smithi053.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T14:50:47.548 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T14:50:47.548 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.548 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.548 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.549 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.549 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.549 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.549 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.549 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.549 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.549 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.549 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.549 INFO:teuthology.orchestra.run.smithi053.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T14:50:47.550 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T14:50:47.550 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.550 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.550 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.550 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.550 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.550 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.550 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.550 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.551 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.551 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.551 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.551 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T14:50:47.551 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T14:50:47.551 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.551 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.551 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.552 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.552 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.552 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.552 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.552 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.552 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.552 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.552 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.553 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T14:50:47.553 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T14:50:47.553 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.553 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.553 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.553 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.553 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.553 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.553 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.554 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.554 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.554 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.554 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.554 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T14:50:47.554 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T14:50:47.554 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.554 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.555 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.555 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.555 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.555 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.555 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.555 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.555 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.555 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.555 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.556 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T14:50:47.556 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T14:50:47.556 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.556 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.556 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.556 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.556 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.556 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.556 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.556 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.557 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.557 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.557 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.557 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T14:50:47.557 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T14:50:47.557 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.557 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.557 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.557 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.558 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.558 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.558 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.558 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.558 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.558 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.558 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.558 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T14:50:47.559 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T14:50:47.559 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.559 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.559 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.559 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.559 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.559 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.559 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.559 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.559 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.560 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.560 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.560 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T14:50:47.560 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T14:50:47.560 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.560 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.560 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.560 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.561 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.561 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.561 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.561 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.561 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.561 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.561 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.561 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T14:50:47.562 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T14:50:47.562 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.562 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.562 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.562 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.562 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.562 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.562 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.563 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.563 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.563 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.563 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.563 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T14:50:47.563 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T14:50:47.563 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.563 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.564 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.564 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T14:50:47.564 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.564 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.564 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.564 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.564 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.564 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.564 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.565 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T14:50:47.565 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T14:50:47.565 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.565 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.565 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.565 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2024-04-25T14:50:47.565 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.565 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.566 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.566 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.566 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.566 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.566 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.566 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T14:50:47.566 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T14:50:47.567 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.567 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.567 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.567 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "8", 2024-04-25T14:50:47.567 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.567 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.567 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.567 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.567 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.568 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.568 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.568 INFO:teuthology.orchestra.run.smithi053.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T14:50:47.568 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T14:50:47.568 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.568 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.568 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.568 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "45", 2024-04-25T14:50:47.569 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.569 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.569 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.569 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.569 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.569 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.569 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.570 INFO:teuthology.orchestra.run.smithi053.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T14:50:47.570 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T14:50:47.570 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.570 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.570 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.570 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.570 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.570 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.570 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.571 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.571 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.571 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.571 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.571 INFO:teuthology.orchestra.run.smithi053.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T14:50:47.571 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T14:50:47.571 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.572 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.572 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.572 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "admin", 2024-04-25T14:50:47.572 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.572 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.572 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.572 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.572 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.572 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.573 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.573 INFO:teuthology.orchestra.run.smithi053.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T14:50:47.573 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T14:50:47.573 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.573 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.573 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.573 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.573 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.574 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.574 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.574 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.574 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.574 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.574 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.574 INFO:teuthology.orchestra.run.smithi053.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T14:50:47.575 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T14:50:47.575 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.575 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.575 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.575 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.575 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.575 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.575 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.575 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.576 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.576 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.576 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.576 INFO:teuthology.orchestra.run.smithi053.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T14:50:47.576 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T14:50:47.576 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.576 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.576 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.576 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.577 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.577 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.577 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.577 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.577 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.577 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.577 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.578 INFO:teuthology.orchestra.run.smithi053.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T14:50:47.578 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T14:50:47.578 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.578 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.578 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.578 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0", 2024-04-25T14:50:47.578 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.578 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.578 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.579 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.579 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.579 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.579 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.579 INFO:teuthology.orchestra.run.smithi053.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T14:50:47.579 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T14:50:47.579 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.579 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.580 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.580 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2024-04-25T14:50:47.580 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.580 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.580 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.580 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.580 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.580 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.580 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.581 INFO:teuthology.orchestra.run.smithi053.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T14:50:47.581 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T14:50:47.581 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.581 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.581 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.581 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "5", 2024-04-25T14:50:47.581 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.582 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.582 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.582 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.582 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.582 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.582 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.582 INFO:teuthology.orchestra.run.smithi053.stdout: "cross_origin_url": { 2024-04-25T14:50:47.582 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "cross_origin_url", 2024-04-25T14:50:47.583 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.583 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.583 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.583 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.583 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.583 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.583 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.583 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.583 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.584 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.584 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.584 INFO:teuthology.orchestra.run.smithi053.stdout: "crt_file": { 2024-04-25T14:50:47.584 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "crt_file", 2024-04-25T14:50:47.584 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.584 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.584 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.585 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.585 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.585 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.585 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.585 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.585 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.585 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.585 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.586 INFO:teuthology.orchestra.run.smithi053.stdout: "debug": { 2024-04-25T14:50:47.586 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "debug", 2024-04-25T14:50:47.586 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.586 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.586 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.586 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.586 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.586 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.587 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.587 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Enable/disable debug options", 2024-04-25T14:50:47.587 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.587 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.587 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.587 INFO:teuthology.orchestra.run.smithi053.stdout: "jwt_token_ttl": { 2024-04-25T14:50:47.587 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "jwt_token_ttl", 2024-04-25T14:50:47.587 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.587 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.588 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.588 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "28800", 2024-04-25T14:50:47.588 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.588 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.588 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.588 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.588 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.588 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.588 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.589 INFO:teuthology.orchestra.run.smithi053.stdout: "key_file": { 2024-04-25T14:50:47.589 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "key_file", 2024-04-25T14:50:47.589 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.589 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.589 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.589 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.589 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.589 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.589 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.590 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.590 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.590 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.590 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.590 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.590 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.590 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.590 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.591 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.591 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.591 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.591 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.591 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.591 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.591 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.591 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.591 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.592 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.592 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.592 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.592 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.592 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.592 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.592 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.592 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.593 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.593 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.593 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.593 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.593 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.593 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.593 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.593 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.593 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.593 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.594 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.594 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.594 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.594 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.594 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.594 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.594 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.594 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.595 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.595 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.595 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.595 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.595 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.595 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.595 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.595 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.595 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.595 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.596 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.596 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.596 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.596 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.596 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.596 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.596 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.596 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.597 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.597 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.597 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.597 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.597 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.597 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.597 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.597 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.598 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.598 INFO:teuthology.orchestra.run.smithi053.stdout: "motd": { 2024-04-25T14:50:47.598 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "motd", 2024-04-25T14:50:47.598 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.598 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.598 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.598 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.599 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.599 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.599 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.599 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "The message of the day", 2024-04-25T14:50:47.599 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.599 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.599 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.600 INFO:teuthology.orchestra.run.smithi053.stdout: "redirect_resolve_ip_addr": { 2024-04-25T14:50:47.600 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T14:50:47.600 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.600 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.600 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.600 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.600 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.600 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.600 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.601 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.601 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.601 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.601 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.601 INFO:teuthology.orchestra.run.smithi053.stdout: "server_addr": { 2024-04-25T14:50:47.601 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "server_addr", 2024-04-25T14:50:47.601 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.601 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.602 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.602 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "::", 2024-04-25T14:50:47.602 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.602 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.602 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.602 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.602 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.602 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.603 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.603 INFO:teuthology.orchestra.run.smithi053.stdout: "server_port": { 2024-04-25T14:50:47.603 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "server_port", 2024-04-25T14:50:47.603 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.603 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.603 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.603 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "8080", 2024-04-25T14:50:47.603 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.603 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.604 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.604 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.604 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.604 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.604 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.604 INFO:teuthology.orchestra.run.smithi053.stdout: "ssl": { 2024-04-25T14:50:47.604 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ssl", 2024-04-25T14:50:47.604 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.605 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.605 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.605 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.605 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.605 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.605 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.605 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.605 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.605 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.606 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.606 INFO:teuthology.orchestra.run.smithi053.stdout: "ssl_server_port": { 2024-04-25T14:50:47.606 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ssl_server_port", 2024-04-25T14:50:47.606 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.606 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.606 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.606 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "8443", 2024-04-25T14:50:47.606 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.607 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.607 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.607 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.607 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.607 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.607 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.607 INFO:teuthology.orchestra.run.smithi053.stdout: "standby_behaviour": { 2024-04-25T14:50:47.607 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "standby_behaviour", 2024-04-25T14:50:47.608 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.608 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.608 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.608 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "redirect", 2024-04-25T14:50:47.608 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.608 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.608 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.608 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.608 INFO:teuthology.orchestra.run.smithi053.stdout: "redirect" 2024-04-25T14:50:47.608 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.609 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.609 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.609 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.609 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.609 INFO:teuthology.orchestra.run.smithi053.stdout: "standby_error_status_code": { 2024-04-25T14:50:47.609 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "standby_error_status_code", 2024-04-25T14:50:47.609 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.609 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.609 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.610 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "500", 2024-04-25T14:50:47.610 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "400", 2024-04-25T14:50:47.610 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "599", 2024-04-25T14:50:47.610 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.610 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.610 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.610 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.610 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.611 INFO:teuthology.orchestra.run.smithi053.stdout: "url_prefix": { 2024-04-25T14:50:47.611 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "url_prefix", 2024-04-25T14:50:47.611 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.611 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.611 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.611 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.611 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.611 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.611 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.612 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.612 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.612 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.612 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.612 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.612 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.612 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "devicehealth", 2024-04-25T14:50:47.612 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.613 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.613 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.613 INFO:teuthology.orchestra.run.smithi053.stdout: "enable_monitoring": { 2024-04-25T14:50:47.613 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "enable_monitoring", 2024-04-25T14:50:47.613 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.613 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.613 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.613 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.613 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.614 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.614 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.614 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "monitor device health metrics", 2024-04-25T14:50:47.614 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.614 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.614 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.614 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.614 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.615 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.615 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.615 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.615 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.615 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.615 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.615 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.615 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.615 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.616 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.616 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.616 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.616 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.616 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.616 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.616 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.616 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.616 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.617 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.617 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.617 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.617 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.617 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.617 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.617 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.617 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.617 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.618 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.618 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.618 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.618 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.618 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.618 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.618 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.618 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.619 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.619 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.619 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.619 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.619 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.619 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.619 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.619 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.619 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.620 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.620 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.620 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.620 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.620 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.620 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.620 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.620 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.621 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.621 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.621 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.621 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.621 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.621 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.621 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.621 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.621 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.622 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.622 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.622 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.622 INFO:teuthology.orchestra.run.smithi053.stdout: "mark_out_threshold": { 2024-04-25T14:50:47.622 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "mark_out_threshold", 2024-04-25T14:50:47.622 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:47.622 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.623 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.623 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "2419200", 2024-04-25T14:50:47.623 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.623 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.623 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.623 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T14:50:47.623 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.623 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.623 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.624 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": { 2024-04-25T14:50:47.624 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "pool_name", 2024-04-25T14:50:47.624 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.624 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.624 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.624 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "device_health_metrics", 2024-04-25T14:50:47.624 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.624 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.625 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.625 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T14:50:47.625 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.625 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.625 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.625 INFO:teuthology.orchestra.run.smithi053.stdout: "retention_period": { 2024-04-25T14:50:47.625 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "retention_period", 2024-04-25T14:50:47.625 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:47.626 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.626 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.626 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "15552000", 2024-04-25T14:50:47.626 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.626 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.626 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.626 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how long to retain device health metrics", 2024-04-25T14:50:47.626 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.627 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.627 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.627 INFO:teuthology.orchestra.run.smithi053.stdout: "scrape_frequency": { 2024-04-25T14:50:47.627 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "scrape_frequency", 2024-04-25T14:50:47.627 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:47.627 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.627 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.627 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "86400", 2024-04-25T14:50:47.627 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.628 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.628 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.628 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T14:50:47.628 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.628 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.628 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.628 INFO:teuthology.orchestra.run.smithi053.stdout: "self_heal": { 2024-04-25T14:50:47.628 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "self_heal", 2024-04-25T14:50:47.628 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.629 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.629 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.629 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.629 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.629 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.629 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.629 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T14:50:47.629 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.629 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.630 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.630 INFO:teuthology.orchestra.run.smithi053.stdout: "sleep_interval": { 2024-04-25T14:50:47.630 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "sleep_interval", 2024-04-25T14:50:47.630 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:47.630 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.630 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.630 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "600", 2024-04-25T14:50:47.631 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.631 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.631 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.631 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T14:50:47.631 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.631 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.631 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.631 INFO:teuthology.orchestra.run.smithi053.stdout: "warn_threshold": { 2024-04-25T14:50:47.631 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "warn_threshold", 2024-04-25T14:50:47.632 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:47.632 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.632 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.632 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "7257600", 2024-04-25T14:50:47.632 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.632 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.632 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.632 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T14:50:47.632 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.633 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.633 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.633 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.633 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.633 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "diskprediction_local", 2024-04-25T14:50:47.633 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.633 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.634 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.634 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.634 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.634 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.634 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.634 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.634 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.634 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.635 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.635 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.635 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.635 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.635 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.635 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.635 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.635 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.635 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.636 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.636 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.636 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.636 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.636 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.636 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.636 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.636 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.637 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.637 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.637 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.637 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.637 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.637 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.637 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.637 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.637 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.638 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.638 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.638 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.638 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.638 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.638 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.638 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.638 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.639 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.639 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.639 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.639 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.639 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.639 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.639 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.639 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.639 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.640 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.640 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.640 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.640 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.640 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.640 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.640 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.640 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.641 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.641 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.641 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.641 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.641 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.641 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.641 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.641 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.641 INFO:teuthology.orchestra.run.smithi053.stdout: "predict_interval": { 2024-04-25T14:50:47.642 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "predict_interval", 2024-04-25T14:50:47.642 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.642 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.642 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.642 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "86400", 2024-04-25T14:50:47.642 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.642 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.642 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.643 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.643 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.643 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.643 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.643 INFO:teuthology.orchestra.run.smithi053.stdout: "predictor_model": { 2024-04-25T14:50:47.643 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "predictor_model", 2024-04-25T14:50:47.643 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.643 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.643 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.644 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "prophetstor", 2024-04-25T14:50:47.644 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.644 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.644 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.644 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.644 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.644 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.644 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.645 INFO:teuthology.orchestra.run.smithi053.stdout: "sleep_interval": { 2024-04-25T14:50:47.645 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "sleep_interval", 2024-04-25T14:50:47.645 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.645 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.645 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.645 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "600", 2024-04-25T14:50:47.645 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.645 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.645 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.646 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.646 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.646 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.646 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.646 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.646 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.646 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "influx", 2024-04-25T14:50:47.647 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": false, 2024-04-25T14:50:47.647 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "influxdb python module not found", 2024-04-25T14:50:47.647 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.647 INFO:teuthology.orchestra.run.smithi053.stdout: "batch_size": { 2024-04-25T14:50:47.647 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "batch_size", 2024-04-25T14:50:47.647 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.647 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.647 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.647 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "5000", 2024-04-25T14:50:47.648 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.648 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.648 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.648 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T14:50:47.648 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.648 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.648 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.648 INFO:teuthology.orchestra.run.smithi053.stdout: "database": { 2024-04-25T14:50:47.648 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "database", 2024-04-25T14:50:47.649 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.649 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.649 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.649 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "ceph", 2024-04-25T14:50:47.649 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.649 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.649 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.649 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T14:50:47.649 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.650 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.650 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.650 INFO:teuthology.orchestra.run.smithi053.stdout: "hostname": { 2024-04-25T14:50:47.650 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "hostname", 2024-04-25T14:50:47.650 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.650 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.650 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.650 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.651 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.651 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.651 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.651 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "InfluxDB server hostname", 2024-04-25T14:50:47.651 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.651 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.651 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.651 INFO:teuthology.orchestra.run.smithi053.stdout: "interval": { 2024-04-25T14:50:47.652 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "interval", 2024-04-25T14:50:47.652 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:47.652 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.652 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.652 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "30", 2024-04-25T14:50:47.652 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "5", 2024-04-25T14:50:47.652 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.652 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.652 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T14:50:47.652 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.653 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.653 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.653 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.653 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.653 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.653 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.653 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.653 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.653 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.654 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.654 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.654 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.654 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.654 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.654 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.654 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.654 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.654 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.655 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.655 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.655 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.655 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.655 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.655 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.655 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.655 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.655 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.656 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.656 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.656 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.656 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.656 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.656 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.656 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.656 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.657 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.657 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.657 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.657 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.657 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.657 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.657 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.657 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.657 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.657 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.658 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.658 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.658 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.658 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.658 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.658 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.658 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.658 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.658 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.659 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.659 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.659 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.659 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.659 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.659 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.659 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.659 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.660 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.660 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.660 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.660 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.660 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.660 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.660 INFO:teuthology.orchestra.run.smithi053.stdout: "password": { 2024-04-25T14:50:47.660 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "password", 2024-04-25T14:50:47.660 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.661 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.661 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.661 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.661 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.661 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.661 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.661 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "password of InfluxDB server user", 2024-04-25T14:50:47.661 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.661 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.662 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.662 INFO:teuthology.orchestra.run.smithi053.stdout: "port": { 2024-04-25T14:50:47.662 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "port", 2024-04-25T14:50:47.662 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.662 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.662 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.662 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "8086", 2024-04-25T14:50:47.662 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.663 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.663 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.663 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "InfluxDB server port", 2024-04-25T14:50:47.663 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.663 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.663 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.663 INFO:teuthology.orchestra.run.smithi053.stdout: "ssl": { 2024-04-25T14:50:47.663 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ssl", 2024-04-25T14:50:47.664 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.664 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.664 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.664 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "false", 2024-04-25T14:50:47.664 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.664 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.664 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.664 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T14:50:47.664 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.664 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.665 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.665 INFO:teuthology.orchestra.run.smithi053.stdout: "threads": { 2024-04-25T14:50:47.665 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "threads", 2024-04-25T14:50:47.665 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.665 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.665 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.665 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "5", 2024-04-25T14:50:47.665 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "1", 2024-04-25T14:50:47.666 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "32", 2024-04-25T14:50:47.666 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.666 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T14:50:47.666 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.666 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.666 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.666 INFO:teuthology.orchestra.run.smithi053.stdout: "username": { 2024-04-25T14:50:47.666 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "username", 2024-04-25T14:50:47.667 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.667 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.667 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.667 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.667 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.667 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.667 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.667 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "username of InfluxDB server user", 2024-04-25T14:50:47.667 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.668 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.668 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.668 INFO:teuthology.orchestra.run.smithi053.stdout: "verify_ssl": { 2024-04-25T14:50:47.668 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "verify_ssl", 2024-04-25T14:50:47.668 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.668 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.668 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.668 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "true", 2024-04-25T14:50:47.668 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.669 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.669 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.669 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T14:50:47.669 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.669 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.669 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.669 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.669 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.670 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "insights", 2024-04-25T14:50:47.670 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.670 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.670 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.670 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.670 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.670 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.670 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.671 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.671 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.671 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.671 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.671 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.671 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.671 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.671 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.672 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.672 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.672 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.672 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.672 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.672 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.672 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.672 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.673 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.673 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.673 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.673 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.673 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.673 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.673 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.673 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.674 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.674 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.674 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.674 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.674 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.674 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.674 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.675 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.675 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.675 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.675 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.675 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.675 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.675 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.675 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.675 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.676 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.676 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.676 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.676 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.676 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.676 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.676 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.676 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.677 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.677 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.677 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.677 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.677 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.677 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.677 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.677 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.678 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.678 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.678 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.678 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.678 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.678 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.678 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.678 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.679 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "iostat", 2024-04-25T14:50:47.679 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.679 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.679 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.679 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.679 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.679 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.680 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.680 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.680 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.680 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.680 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.680 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.680 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.680 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.680 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.681 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.681 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.681 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.681 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.681 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.681 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.681 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.681 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.681 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.682 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.682 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.682 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.682 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.682 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.682 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.682 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.682 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.683 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.683 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.683 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.683 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.683 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.683 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.683 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.683 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.684 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.684 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.684 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.684 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.684 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.684 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.684 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.684 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.684 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.685 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.685 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.685 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.685 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.685 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.685 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.685 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.685 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.685 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.686 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.686 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.686 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.686 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.686 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.686 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.686 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.686 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.687 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.687 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.687 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.687 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.687 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.687 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "k8sevents", 2024-04-25T14:50:47.689 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.689 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.689 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.689 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_event_retention_days": { 2024-04-25T14:50:47.689 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ceph_event_retention_days", 2024-04-25T14:50:47.689 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.689 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.689 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.689 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "7", 2024-04-25T14:50:47.690 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.690 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.690 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.690 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T14:50:47.690 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.690 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.690 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.690 INFO:teuthology.orchestra.run.smithi053.stdout: "config_check_secs": { 2024-04-25T14:50:47.690 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "config_check_secs", 2024-04-25T14:50:47.691 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.691 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.691 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.691 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2024-04-25T14:50:47.691 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "10", 2024-04-25T14:50:47.691 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.691 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.691 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T14:50:47.691 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.691 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.692 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.692 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.692 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.692 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.692 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.692 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.692 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.692 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.693 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.693 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.693 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.693 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.693 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.693 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.693 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.693 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.693 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.693 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.694 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.694 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.694 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.694 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.694 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.694 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.694 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.694 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.695 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.695 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.695 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.695 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.695 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.695 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.695 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.695 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.695 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.696 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.696 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.696 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.696 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.696 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.696 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.696 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.696 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.696 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.696 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.697 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.697 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.697 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.697 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.697 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.697 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.697 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.697 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.697 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.698 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.698 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.698 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.698 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.698 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.698 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.698 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.698 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.699 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.699 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.699 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.699 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.699 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.699 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.699 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.700 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "localpool", 2024-04-25T14:50:47.700 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.700 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.700 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.700 INFO:teuthology.orchestra.run.smithi053.stdout: "failure_domain": { 2024-04-25T14:50:47.700 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "failure_domain", 2024-04-25T14:50:47.700 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.700 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.700 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.700 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "host", 2024-04-25T14:50:47.701 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.701 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.701 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.701 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "failure domain for any created local pool", 2024-04-25T14:50:47.701 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T14:50:47.701 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.701 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.701 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.701 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.702 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.702 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.702 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.702 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.702 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.702 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.702 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.702 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.702 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.703 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.703 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.703 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.703 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.703 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.703 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.703 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.703 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.703 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.704 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.704 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.704 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.704 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.704 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.704 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.704 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.705 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.705 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.705 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.705 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.705 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.705 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.705 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.705 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.705 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.706 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.706 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.706 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.706 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.706 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.706 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.706 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.706 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.706 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.707 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.707 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.707 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.707 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.707 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.707 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.707 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.707 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.708 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.708 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.708 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.708 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.708 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.708 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.708 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.708 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.708 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.709 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.709 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.709 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.709 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.709 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": { 2024-04-25T14:50:47.709 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "min_size", 2024-04-25T14:50:47.709 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.709 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.709 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.710 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.710 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.710 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.710 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.710 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "default min_size for any created local pool", 2024-04-25T14:50:47.710 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T14:50:47.711 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.711 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.711 INFO:teuthology.orchestra.run.smithi053.stdout: "num_rep": { 2024-04-25T14:50:47.711 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "num_rep", 2024-04-25T14:50:47.711 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.711 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.712 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.712 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "3", 2024-04-25T14:50:47.712 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.712 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.712 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.712 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "default replica count for any created local pool", 2024-04-25T14:50:47.713 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.713 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.713 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.713 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": { 2024-04-25T14:50:47.713 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "pg_num", 2024-04-25T14:50:47.713 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.714 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.714 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.714 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "128", 2024-04-25T14:50:47.714 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.714 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.714 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.715 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T14:50:47.715 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.715 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.715 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.715 INFO:teuthology.orchestra.run.smithi053.stdout: "prefix": { 2024-04-25T14:50:47.715 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "prefix", 2024-04-25T14:50:47.715 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.716 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.716 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.716 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.716 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.716 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.716 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.716 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "name prefix for any created local pool", 2024-04-25T14:50:47.716 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.716 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.716 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.717 INFO:teuthology.orchestra.run.smithi053.stdout: "subtree": { 2024-04-25T14:50:47.717 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "subtree", 2024-04-25T14:50:47.717 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.717 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.717 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.717 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "rack", 2024-04-25T14:50:47.717 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.717 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.718 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.718 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T14:50:47.718 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T14:50:47.718 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.718 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.718 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.718 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.719 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "mds_autoscaler", 2024-04-25T14:50:47.719 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.719 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.719 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.719 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.719 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.719 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.719 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.720 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.720 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.720 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.720 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.720 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.720 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.720 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.720 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.720 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.720 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.721 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.721 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.721 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.721 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.721 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.721 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.721 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.722 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.722 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.722 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.722 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.722 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.722 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.722 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.722 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.722 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.723 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.723 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.723 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.723 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.723 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.723 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.723 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.723 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.724 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.724 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.724 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.724 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.724 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.724 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.724 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.724 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.725 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.725 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.725 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.725 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.725 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.725 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.725 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.725 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.726 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.726 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.726 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.726 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.726 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.726 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.726 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.726 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.727 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.727 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.727 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.727 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.727 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.727 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.728 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "mirroring", 2024-04-25T14:50:47.728 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.728 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.728 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.728 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.728 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.728 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.728 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.728 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.729 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.729 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.729 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.729 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.729 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.729 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.729 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.729 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.729 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.730 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.730 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.730 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.730 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.730 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.730 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.730 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.731 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.731 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.731 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.731 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.731 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.731 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.731 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.731 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.731 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.731 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.732 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.732 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.732 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.732 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.732 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.732 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.732 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.732 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.733 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.733 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.733 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.733 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.733 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.733 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.733 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.733 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.733 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.734 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.734 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.734 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.734 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.734 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.734 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.734 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.735 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.735 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.735 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.735 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.735 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.735 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.735 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.735 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.735 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.736 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.736 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.736 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.736 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.736 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "nfs", 2024-04-25T14:50:47.736 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.736 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.737 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.737 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.737 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.737 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.737 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.737 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.737 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.737 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.737 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.738 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.738 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.738 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.738 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.738 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.738 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.738 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.738 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.738 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.739 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.739 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.739 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.739 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.739 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.739 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.739 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.740 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.740 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.740 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.740 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.740 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.740 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.740 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.740 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.740 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.741 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.741 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.741 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.741 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.741 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.741 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.741 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.742 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.742 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.742 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.742 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.742 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.742 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.742 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.742 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.742 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.743 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.743 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.743 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.743 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.743 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.743 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.743 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.744 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.744 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.744 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.744 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.744 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.744 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.744 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.744 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.744 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.744 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.745 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.745 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.745 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "orchestrator", 2024-04-25T14:50:47.745 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.745 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.745 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.745 INFO:teuthology.orchestra.run.smithi053.stdout: "fail_fs": { 2024-04-25T14:50:47.746 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "fail_fs", 2024-04-25T14:50:47.746 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.746 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.746 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.746 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.746 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.746 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.746 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.746 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T14:50:47.747 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.747 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.747 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.747 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.747 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.747 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.747 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.748 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.748 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.748 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.748 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.748 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.748 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.748 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.748 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.748 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.748 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.749 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.749 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.749 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.749 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.749 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.749 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.750 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.750 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.750 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.750 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.750 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.750 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.750 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.750 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.750 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.751 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.751 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.751 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.751 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.751 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.751 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.751 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.751 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.752 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.752 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.752 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.752 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.752 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.752 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.752 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.752 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.752 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.753 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.753 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.753 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.753 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.753 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.753 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.753 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.753 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.753 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.754 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.754 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.754 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.754 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.754 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.754 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.754 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.754 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.755 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.755 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.755 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.755 INFO:teuthology.orchestra.run.smithi053.stdout: "orchestrator": { 2024-04-25T14:50:47.755 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "orchestrator", 2024-04-25T14:50:47.755 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.755 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.755 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.755 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.756 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.756 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.756 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.756 INFO:teuthology.orchestra.run.smithi053.stdout: "cephadm", 2024-04-25T14:50:47.756 INFO:teuthology.orchestra.run.smithi053.stdout: "rook", 2024-04-25T14:50:47.756 INFO:teuthology.orchestra.run.smithi053.stdout: "test_orchestrator" 2024-04-25T14:50:47.756 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.756 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Orchestrator backend", 2024-04-25T14:50:47.756 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.756 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.757 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.757 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.757 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.757 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "osd_perf_query", 2024-04-25T14:50:47.757 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.757 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.757 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.757 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.758 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.758 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.758 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.758 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.758 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.758 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.758 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.758 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.758 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.759 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.759 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.759 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.759 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.759 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.759 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.759 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.759 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.759 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.760 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.760 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.760 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.760 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.760 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.760 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.760 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.760 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.761 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.761 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.761 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.761 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.761 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.761 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.762 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.762 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.762 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.762 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.762 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.762 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.762 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.762 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.763 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.763 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.763 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.763 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.763 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.763 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.763 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.763 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.763 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.763 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.764 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.764 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.764 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.764 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.764 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.764 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.764 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.764 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.764 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.765 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.765 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.765 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.765 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.765 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.765 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.765 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.765 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.766 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "osd_support", 2024-04-25T14:50:47.766 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.766 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.766 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.766 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.766 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.766 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.766 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.767 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.767 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.767 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.767 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.767 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.767 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.767 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.767 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.767 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.768 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.768 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.768 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.768 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.768 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.768 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.768 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.768 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.768 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.769 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.769 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.769 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.769 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.769 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.769 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.769 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.769 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.769 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.769 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.770 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.770 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.770 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.770 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.770 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.770 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.770 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.771 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.771 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.771 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.771 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.771 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.771 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.771 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.771 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.771 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.771 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.772 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.772 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.772 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.772 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.772 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.772 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.772 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.772 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.773 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.773 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.773 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.773 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.773 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.773 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.773 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.773 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.774 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.774 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.774 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.774 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "pg_autoscaler", 2024-04-25T14:50:47.774 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.774 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.774 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.775 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.775 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.775 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.775 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.775 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.775 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.775 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.775 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.776 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.776 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.776 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.776 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.776 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.776 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.776 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.776 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.776 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.776 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.777 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.777 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.777 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.777 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.777 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.777 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.777 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.778 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.778 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.778 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.778 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.778 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.778 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.778 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.778 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.779 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.779 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.779 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.779 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.779 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.779 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.779 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.779 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.779 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.780 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.780 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.780 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.780 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.780 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.780 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.780 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.780 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.780 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.781 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.781 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.781 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.781 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.781 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.781 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.781 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.781 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.782 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.782 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.782 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.782 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.782 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.782 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.782 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.782 INFO:teuthology.orchestra.run.smithi053.stdout: "sleep_interval": { 2024-04-25T14:50:47.783 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "sleep_interval", 2024-04-25T14:50:47.783 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:47.783 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.783 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.783 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "60", 2024-04-25T14:50:47.783 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.783 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.783 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.783 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.784 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.784 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.784 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.784 INFO:teuthology.orchestra.run.smithi053.stdout: "threshold": { 2024-04-25T14:50:47.784 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "threshold", 2024-04-25T14:50:47.784 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2024-04-25T14:50:47.784 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.784 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.785 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "3.0", 2024-04-25T14:50:47.785 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "1.0", 2024-04-25T14:50:47.785 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.785 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.785 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "scaling threshold", 2024-04-25T14:50:47.785 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T14:50:47.785 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.785 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.786 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.786 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.786 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "progress", 2024-04-25T14:50:47.786 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.786 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.786 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.786 INFO:teuthology.orchestra.run.smithi053.stdout: "allow_pg_recovery_event": { 2024-04-25T14:50:47.787 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "allow_pg_recovery_event", 2024-04-25T14:50:47.787 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.787 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.787 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.787 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.787 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.787 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.787 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.787 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T14:50:47.788 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.788 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.788 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.788 INFO:teuthology.orchestra.run.smithi053.stdout: "enabled": { 2024-04-25T14:50:47.788 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "enabled", 2024-04-25T14:50:47.788 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.788 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.788 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.789 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.789 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.789 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.789 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.789 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.789 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.789 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.789 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.790 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.790 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.790 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.790 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.790 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.790 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.790 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.790 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.791 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.791 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.791 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.791 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.791 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.791 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.791 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.791 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.791 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.791 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.792 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.792 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.792 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.792 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.792 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.792 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.792 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.792 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.793 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.793 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.793 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.793 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.793 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.793 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.793 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.794 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.794 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.794 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.794 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.794 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.794 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.794 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.794 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.794 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.795 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.795 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.795 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.795 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.795 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.795 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.795 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.795 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.795 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.796 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.796 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.796 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.796 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.796 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.796 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.796 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.796 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.797 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.797 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.797 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.797 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.797 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.797 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.797 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.797 INFO:teuthology.orchestra.run.smithi053.stdout: "max_completed_events": { 2024-04-25T14:50:47.797 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "max_completed_events", 2024-04-25T14:50:47.798 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.798 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.798 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.798 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "50", 2024-04-25T14:50:47.798 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.798 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.798 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.798 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "number of past completed events to remember", 2024-04-25T14:50:47.799 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.799 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.799 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.799 INFO:teuthology.orchestra.run.smithi053.stdout: "sleep_interval": { 2024-04-25T14:50:47.799 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "sleep_interval", 2024-04-25T14:50:47.799 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:47.799 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.799 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.799 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "5", 2024-04-25T14:50:47.800 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.800 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.800 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.800 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how long the module is going to sleep", 2024-04-25T14:50:47.800 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.800 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.800 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.800 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.800 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.801 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "prometheus", 2024-04-25T14:50:47.801 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.801 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.801 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.801 INFO:teuthology.orchestra.run.smithi053.stdout: "cache": { 2024-04-25T14:50:47.801 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "cache", 2024-04-25T14:50:47.801 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.801 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.802 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.802 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.802 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.802 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.802 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.802 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.802 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.802 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.803 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.803 INFO:teuthology.orchestra.run.smithi053.stdout: "exclude_perf_counters": { 2024-04-25T14:50:47.803 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "exclude_perf_counters", 2024-04-25T14:50:47.803 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.803 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.803 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.803 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.803 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.803 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.804 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.804 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T14:50:47.804 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T14:50:47.804 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.804 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.804 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.804 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.804 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.804 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.805 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.805 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.805 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.805 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.805 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.805 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.805 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.805 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.805 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.805 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.806 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.806 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.806 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.806 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.806 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.806 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.806 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.807 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.807 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.807 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.807 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.807 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.807 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.807 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.807 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.807 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.807 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.808 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.808 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.808 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.808 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.808 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.808 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.808 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.808 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.808 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.809 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.809 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.809 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.809 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.809 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.809 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.809 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.809 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.809 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.809 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.810 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.810 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.810 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.810 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.810 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.810 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.810 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.810 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.811 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.811 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.811 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.811 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.811 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.811 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.811 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.811 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.811 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_stats_pools": { 2024-04-25T14:50:47.812 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rbd_stats_pools", 2024-04-25T14:50:47.812 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.812 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.812 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.812 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.812 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.812 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.812 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.813 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.813 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.813 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.813 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.813 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T14:50:47.813 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T14:50:47.813 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.813 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.814 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.814 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "300", 2024-04-25T14:50:47.814 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.814 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.814 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.814 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.814 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.814 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.815 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.815 INFO:teuthology.orchestra.run.smithi053.stdout: "scrape_interval": { 2024-04-25T14:50:47.815 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "scrape_interval", 2024-04-25T14:50:47.815 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2024-04-25T14:50:47.815 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.815 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.815 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "15.0", 2024-04-25T14:50:47.815 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.816 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.816 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.816 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.816 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.816 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.816 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.816 INFO:teuthology.orchestra.run.smithi053.stdout: "server_addr": { 2024-04-25T14:50:47.816 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "server_addr", 2024-04-25T14:50:47.817 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.817 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.817 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.817 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "::", 2024-04-25T14:50:47.817 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.817 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.817 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.817 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T14:50:47.817 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.818 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.818 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.818 INFO:teuthology.orchestra.run.smithi053.stdout: "server_port": { 2024-04-25T14:50:47.818 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "server_port", 2024-04-25T14:50:47.818 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.818 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.818 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.819 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "9283", 2024-04-25T14:50:47.819 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.819 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.819 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.819 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T14:50:47.819 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.819 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.819 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.820 INFO:teuthology.orchestra.run.smithi053.stdout: "stale_cache_strategy": { 2024-04-25T14:50:47.820 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "stale_cache_strategy", 2024-04-25T14:50:47.820 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.820 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.820 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.820 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "log", 2024-04-25T14:50:47.820 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.820 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.821 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.821 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.821 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.821 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.821 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.821 INFO:teuthology.orchestra.run.smithi053.stdout: "standby_behaviour": { 2024-04-25T14:50:47.821 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "standby_behaviour", 2024-04-25T14:50:47.822 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.822 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.822 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.822 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "default", 2024-04-25T14:50:47.822 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.822 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.822 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.822 INFO:teuthology.orchestra.run.smithi053.stdout: "default", 2024-04-25T14:50:47.822 INFO:teuthology.orchestra.run.smithi053.stdout: "error" 2024-04-25T14:50:47.823 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.823 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.823 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.823 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.823 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.823 INFO:teuthology.orchestra.run.smithi053.stdout: "standby_error_status_code": { 2024-04-25T14:50:47.823 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "standby_error_status_code", 2024-04-25T14:50:47.823 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.823 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.824 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.824 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "500", 2024-04-25T14:50:47.824 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "400", 2024-04-25T14:50:47.824 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "599", 2024-04-25T14:50:47.824 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.824 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.824 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.824 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.824 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.824 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.825 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.825 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rbd_support", 2024-04-25T14:50:47.825 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.825 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.825 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.825 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.826 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.826 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.826 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.826 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.826 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.826 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.826 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.826 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.826 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.826 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.827 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.827 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.827 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.827 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.827 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.827 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.827 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.827 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.827 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.828 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.828 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.828 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.828 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.828 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.828 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.828 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.828 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.828 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.829 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.829 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.829 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.829 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.829 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.829 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.829 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.829 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.830 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.830 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.830 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.830 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.830 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.830 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.830 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.830 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.830 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.831 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.831 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.831 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.831 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.831 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.831 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.831 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.831 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.831 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.832 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.832 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.832 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.832 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.832 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.832 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.832 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.832 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.832 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.832 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.833 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.833 INFO:teuthology.orchestra.run.smithi053.stdout: "max_concurrent_snap_create": { 2024-04-25T14:50:47.833 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "max_concurrent_snap_create", 2024-04-25T14:50:47.833 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.833 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.833 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.833 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2024-04-25T14:50:47.833 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.833 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.834 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.834 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.834 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.834 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.834 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.834 INFO:teuthology.orchestra.run.smithi053.stdout: "mirror_snapshot_schedule": { 2024-04-25T14:50:47.834 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T14:50:47.834 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.835 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.835 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.835 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.835 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.835 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.835 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.835 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.835 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.835 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.835 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.836 INFO:teuthology.orchestra.run.smithi053.stdout: "trash_purge_schedule": { 2024-04-25T14:50:47.836 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "trash_purge_schedule", 2024-04-25T14:50:47.836 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.836 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.836 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.836 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.836 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.836 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.836 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.837 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.837 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.837 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.837 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.837 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.837 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.837 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "restful", 2024-04-25T14:50:47.838 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.838 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.838 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.838 INFO:teuthology.orchestra.run.smithi053.stdout: "enable_auth": { 2024-04-25T14:50:47.838 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "enable_auth", 2024-04-25T14:50:47.838 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.838 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.838 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.838 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.838 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.839 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.839 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.839 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.839 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.839 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.839 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.839 INFO:teuthology.orchestra.run.smithi053.stdout: "key_file": { 2024-04-25T14:50:47.839 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "key_file", 2024-04-25T14:50:47.839 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.840 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.840 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.840 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.840 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.840 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.840 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.840 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.840 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.840 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.841 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.841 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.841 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.841 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.841 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.841 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.841 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.841 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.841 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.842 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.842 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.842 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.842 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.842 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.842 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.842 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.842 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.842 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.842 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.843 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.843 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.843 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.843 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.843 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.843 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.843 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.843 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.844 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.844 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.844 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.844 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.844 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.844 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.844 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.844 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.845 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.845 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.845 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.845 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.845 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.845 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.845 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.845 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.845 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.845 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.846 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.846 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.846 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.846 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.846 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.846 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.846 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.846 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.846 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.847 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.847 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.847 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.847 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.847 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.847 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.847 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.847 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.847 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.848 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.848 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.848 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.848 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.848 INFO:teuthology.orchestra.run.smithi053.stdout: "server_addr": { 2024-04-25T14:50:47.848 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "server_addr", 2024-04-25T14:50:47.848 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.848 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.849 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.849 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.849 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.849 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.849 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.849 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.849 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.849 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.849 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.850 INFO:teuthology.orchestra.run.smithi053.stdout: "server_port": { 2024-04-25T14:50:47.850 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "server_port", 2024-04-25T14:50:47.850 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.850 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.850 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.850 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.850 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.850 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.850 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.850 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.851 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.851 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.851 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.851 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.851 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.851 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rgw", 2024-04-25T14:50:47.851 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.852 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.852 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.852 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.852 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.852 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.852 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.852 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.852 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.852 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.852 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.853 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.853 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.853 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.853 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.853 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.853 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.853 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.853 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.853 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.853 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.854 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.854 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.854 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.854 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.854 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.854 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.854 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.854 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.855 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.855 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.855 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.855 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.855 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.855 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.855 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.855 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.855 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.856 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.856 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.856 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.856 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.856 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.856 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.856 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.856 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.856 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.856 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.857 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.857 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.857 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.857 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.857 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.857 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.857 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.857 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.857 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.858 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.858 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.858 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.858 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.858 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.858 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.858 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.858 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.858 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.859 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.859 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.859 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.859 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.859 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.859 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rook", 2024-04-25T14:50:47.859 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.860 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.860 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.860 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.860 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.860 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.860 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.860 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.860 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.860 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.861 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.861 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.861 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.861 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.861 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.861 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.861 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.861 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.861 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.861 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.862 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.862 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.862 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.862 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.862 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.862 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.862 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.863 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.863 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.863 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.863 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.863 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.863 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.863 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.863 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.863 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.864 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.864 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.864 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.864 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.864 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.864 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.864 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.864 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.864 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.865 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.865 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.865 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.865 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.865 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.865 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.865 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.865 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.865 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.866 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.866 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.866 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.866 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.866 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.866 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.866 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.867 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.867 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.867 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.867 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.867 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.867 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.867 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.867 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.868 INFO:teuthology.orchestra.run.smithi053.stdout: "storage_class": { 2024-04-25T14:50:47.868 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "storage_class", 2024-04-25T14:50:47.868 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.868 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.868 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.868 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "local", 2024-04-25T14:50:47.868 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.868 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.868 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.868 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T14:50:47.869 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.869 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.869 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.869 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.869 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.869 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "selftest", 2024-04-25T14:50:47.869 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.870 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.870 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.870 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.870 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.870 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.870 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.870 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.870 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.870 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.871 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.871 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.871 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.871 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.871 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.871 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.871 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.871 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.871 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.871 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.872 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.872 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.872 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.872 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.872 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.872 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.872 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.872 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.872 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.873 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.873 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.873 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.873 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.873 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.873 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.873 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.873 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.874 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.874 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.874 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.874 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.874 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.874 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.874 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.874 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.875 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.875 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.875 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.875 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.875 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.875 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.875 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.875 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.875 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.876 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.876 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.876 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.876 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.876 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.876 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.876 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.876 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.876 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.877 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.877 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.877 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.877 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.877 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.877 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.877 INFO:teuthology.orchestra.run.smithi053.stdout: "roption1": { 2024-04-25T14:50:47.877 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "roption1", 2024-04-25T14:50:47.877 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.878 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.878 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.878 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.878 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.878 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.878 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.878 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.878 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.878 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.879 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.879 INFO:teuthology.orchestra.run.smithi053.stdout: "roption2": { 2024-04-25T14:50:47.879 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "roption2", 2024-04-25T14:50:47.879 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.879 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.879 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.879 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "xyz", 2024-04-25T14:50:47.879 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.879 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.880 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.880 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.880 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.880 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.880 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.880 INFO:teuthology.orchestra.run.smithi053.stdout: "rwoption1": { 2024-04-25T14:50:47.880 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rwoption1", 2024-04-25T14:50:47.880 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.880 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.881 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.881 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.881 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.881 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.881 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.881 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.881 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.881 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.881 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.882 INFO:teuthology.orchestra.run.smithi053.stdout: "rwoption2": { 2024-04-25T14:50:47.882 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rwoption2", 2024-04-25T14:50:47.882 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.882 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.882 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.882 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.882 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.882 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.882 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.883 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.883 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.883 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.883 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.883 INFO:teuthology.orchestra.run.smithi053.stdout: "rwoption3": { 2024-04-25T14:50:47.883 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rwoption3", 2024-04-25T14:50:47.883 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2024-04-25T14:50:47.883 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.884 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.884 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.884 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.884 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.884 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.884 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.884 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.884 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.884 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.885 INFO:teuthology.orchestra.run.smithi053.stdout: "rwoption4": { 2024-04-25T14:50:47.885 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rwoption4", 2024-04-25T14:50:47.885 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.885 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.885 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.885 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.885 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.885 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.885 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.886 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.886 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.886 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.886 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.886 INFO:teuthology.orchestra.run.smithi053.stdout: "rwoption5": { 2024-04-25T14:50:47.886 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rwoption5", 2024-04-25T14:50:47.886 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.886 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.886 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.887 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.887 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.887 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.887 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.887 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.887 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.887 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.887 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.888 INFO:teuthology.orchestra.run.smithi053.stdout: "rwoption6": { 2024-04-25T14:50:47.888 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rwoption6", 2024-04-25T14:50:47.888 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.888 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.888 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.888 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.888 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.888 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.888 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.889 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.889 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.889 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.889 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.889 INFO:teuthology.orchestra.run.smithi053.stdout: "rwoption7": { 2024-04-25T14:50:47.889 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rwoption7", 2024-04-25T14:50:47.889 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.889 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.890 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.890 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.890 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "1", 2024-04-25T14:50:47.890 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "42", 2024-04-25T14:50:47.890 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.890 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.890 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.890 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.890 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.891 INFO:teuthology.orchestra.run.smithi053.stdout: "testkey": { 2024-04-25T14:50:47.891 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "testkey", 2024-04-25T14:50:47.891 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.891 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.891 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.891 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.891 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.891 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.891 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.892 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.892 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.892 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.892 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.892 INFO:teuthology.orchestra.run.smithi053.stdout: "testlkey": { 2024-04-25T14:50:47.892 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "testlkey", 2024-04-25T14:50:47.892 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.892 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.893 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.893 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.893 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.893 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.893 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.893 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.893 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.893 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.893 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.894 INFO:teuthology.orchestra.run.smithi053.stdout: "testnewline": { 2024-04-25T14:50:47.894 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "testnewline", 2024-04-25T14:50:47.894 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.894 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.894 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.894 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.894 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.895 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.895 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.895 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.895 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.895 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.895 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.895 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.895 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.896 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "snap_schedule", 2024-04-25T14:50:47.896 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.896 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.896 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.896 INFO:teuthology.orchestra.run.smithi053.stdout: "allow_m_granularity": { 2024-04-25T14:50:47.896 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "allow_m_granularity", 2024-04-25T14:50:47.896 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.897 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.897 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.897 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.897 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.897 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.897 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.897 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T14:50:47.897 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.897 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.898 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.898 INFO:teuthology.orchestra.run.smithi053.stdout: "dump_on_update": { 2024-04-25T14:50:47.898 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "dump_on_update", 2024-04-25T14:50:47.898 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.898 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.898 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.898 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.898 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.899 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.899 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.899 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "dump database to debug log on update", 2024-04-25T14:50:47.899 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.899 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.899 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.899 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.899 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.900 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.900 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.900 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.900 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.900 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.900 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.900 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.900 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.900 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.901 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.901 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.901 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.901 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.901 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.901 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.901 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.901 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.901 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.902 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.902 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.902 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.902 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.902 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.902 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.902 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.902 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.903 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.903 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.903 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.903 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.903 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.903 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.903 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.903 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.904 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.904 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.904 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.904 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.904 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.904 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.904 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.904 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.904 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.905 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.905 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.905 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.905 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.905 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.905 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.905 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.905 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.905 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.906 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.906 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.906 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.906 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.906 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.906 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.906 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.906 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.907 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.907 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.907 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.907 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.907 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.907 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.907 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "stats", 2024-04-25T14:50:47.907 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.908 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.908 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.908 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.908 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.908 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.908 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.908 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.908 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.908 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.908 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.909 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.909 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.909 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.909 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.909 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.909 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.909 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.909 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.909 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.909 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.910 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.910 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.910 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.910 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.910 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.910 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.910 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.911 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.911 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.911 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.911 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.911 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.911 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.911 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.911 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.911 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.912 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.912 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.912 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.912 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.912 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.912 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.912 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.912 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.912 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.913 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.913 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.913 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.913 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.913 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.913 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.913 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.913 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.913 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.914 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.914 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.914 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.914 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.914 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.914 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.914 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.914 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.915 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.915 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.915 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.915 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.915 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.915 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.915 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.915 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.916 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "status", 2024-04-25T14:50:47.916 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.916 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.916 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.916 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.916 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.916 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.916 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.916 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.917 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.917 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.917 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.917 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.917 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.917 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.917 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.917 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.917 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.918 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.918 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.918 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.918 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.918 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.918 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.918 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.918 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.919 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.919 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.919 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.919 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.919 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.919 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.919 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.919 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.919 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.919 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.920 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.920 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.920 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.920 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.920 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.920 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.920 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.920 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.921 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.921 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.921 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.921 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.921 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.921 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.921 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.921 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.921 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.922 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.922 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.922 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.922 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.922 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.922 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.922 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.922 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.923 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.923 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.923 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.923 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.923 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.923 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.923 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.923 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.923 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.923 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.924 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.924 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "telegraf", 2024-04-25T14:50:47.924 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.924 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.924 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.924 INFO:teuthology.orchestra.run.smithi053.stdout: "address": { 2024-04-25T14:50:47.924 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "address", 2024-04-25T14:50:47.925 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.925 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.925 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.925 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T14:50:47.925 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.925 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.925 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.925 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.925 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.926 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.926 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.926 INFO:teuthology.orchestra.run.smithi053.stdout: "interval": { 2024-04-25T14:50:47.926 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "interval", 2024-04-25T14:50:47.926 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:47.926 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.926 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.926 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "15", 2024-04-25T14:50:47.926 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.927 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.927 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.927 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.927 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.927 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.927 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.927 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.927 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.928 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.928 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.928 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.928 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.928 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.928 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.928 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.928 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.928 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.928 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.929 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.929 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.929 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.929 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.929 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.929 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.929 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.929 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.929 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.930 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.930 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.930 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.930 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.930 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.930 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.930 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.930 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.930 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.931 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.931 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.931 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.931 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.931 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.931 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.931 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.931 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.931 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.932 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.932 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.932 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.932 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.932 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.932 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.932 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.932 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.932 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.933 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.933 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.933 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.933 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.933 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.933 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.933 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.933 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.933 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.934 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.934 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.934 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.934 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.934 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.934 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.934 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.934 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.934 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.935 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.935 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.935 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "telemetry", 2024-04-25T14:50:47.935 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.935 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.935 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.935 INFO:teuthology.orchestra.run.smithi053.stdout: "channel_basic": { 2024-04-25T14:50:47.936 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "channel_basic", 2024-04-25T14:50:47.936 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.936 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.936 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.936 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.936 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.936 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.936 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.936 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T14:50:47.936 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.937 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.937 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.937 INFO:teuthology.orchestra.run.smithi053.stdout: "channel_crash": { 2024-04-25T14:50:47.937 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "channel_crash", 2024-04-25T14:50:47.937 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.937 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.937 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.937 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.938 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.938 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.938 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.938 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T14:50:47.938 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.938 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.938 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.938 INFO:teuthology.orchestra.run.smithi053.stdout: "channel_device": { 2024-04-25T14:50:47.938 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "channel_device", 2024-04-25T14:50:47.939 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.939 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.939 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.939 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.939 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.939 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.939 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.939 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T14:50:47.939 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.939 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.940 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.940 INFO:teuthology.orchestra.run.smithi053.stdout: "channel_ident": { 2024-04-25T14:50:47.940 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "channel_ident", 2024-04-25T14:50:47.940 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.940 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.940 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.940 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.940 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.941 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.941 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.941 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T14:50:47.941 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.941 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.941 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.941 INFO:teuthology.orchestra.run.smithi053.stdout: "channel_perf": { 2024-04-25T14:50:47.941 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "channel_perf", 2024-04-25T14:50:47.941 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.942 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.942 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.942 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.942 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.942 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.942 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.942 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T14:50:47.942 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.942 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.943 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.943 INFO:teuthology.orchestra.run.smithi053.stdout: "contact": { 2024-04-25T14:50:47.943 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "contact", 2024-04-25T14:50:47.943 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.943 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.943 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.943 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.943 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.943 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.944 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.944 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.944 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.944 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.944 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.944 INFO:teuthology.orchestra.run.smithi053.stdout: "description": { 2024-04-25T14:50:47.944 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "description", 2024-04-25T14:50:47.944 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.944 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.945 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.945 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.945 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.945 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.945 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.945 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.945 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.945 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.945 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.946 INFO:teuthology.orchestra.run.smithi053.stdout: "device_url": { 2024-04-25T14:50:47.946 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "device_url", 2024-04-25T14:50:47.946 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.946 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.946 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.946 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T14:50:47.946 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.946 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.946 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.947 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.947 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.947 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.947 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.947 INFO:teuthology.orchestra.run.smithi053.stdout: "enabled": { 2024-04-25T14:50:47.947 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "enabled", 2024-04-25T14:50:47.947 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.947 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.947 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.948 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.948 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.948 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.948 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.948 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.948 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.948 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.948 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.949 INFO:teuthology.orchestra.run.smithi053.stdout: "interval": { 2024-04-25T14:50:47.949 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "interval", 2024-04-25T14:50:47.949 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.949 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.949 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.949 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "24", 2024-04-25T14:50:47.949 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "8", 2024-04-25T14:50:47.949 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.949 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.950 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.950 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.950 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.950 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.950 INFO:teuthology.orchestra.run.smithi053.stdout: "last_opt_revision": { 2024-04-25T14:50:47.950 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "last_opt_revision", 2024-04-25T14:50:47.950 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.950 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.950 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.951 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "1", 2024-04-25T14:50:47.951 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.951 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.951 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.951 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.951 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.951 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.951 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.951 INFO:teuthology.orchestra.run.smithi053.stdout: "leaderboard": { 2024-04-25T14:50:47.952 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "leaderboard", 2024-04-25T14:50:47.952 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.952 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.952 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.952 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.952 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.952 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.952 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.952 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.952 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.953 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.953 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.953 INFO:teuthology.orchestra.run.smithi053.stdout: "leaderboard_description": { 2024-04-25T14:50:47.953 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "leaderboard_description", 2024-04-25T14:50:47.953 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.953 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.953 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.953 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.953 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.954 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.954 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.954 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.954 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.954 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.954 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.954 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.955 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.955 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.955 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.955 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.955 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.955 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.955 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.955 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.956 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.956 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.956 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.956 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.956 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.956 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.956 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.956 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.956 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.957 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.957 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.957 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.957 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.957 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.957 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.957 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.957 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.958 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.958 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.958 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.958 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.958 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.958 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.958 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.959 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.959 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.959 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.959 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.959 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.959 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.959 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.959 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.960 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.960 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.960 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.960 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.960 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.960 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.960 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.960 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.961 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.961 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.961 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.961 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.961 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.961 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.961 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.962 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.962 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.962 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.962 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.962 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.962 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.962 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.962 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.963 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.963 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.963 INFO:teuthology.orchestra.run.smithi053.stdout: "organization": { 2024-04-25T14:50:47.963 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "organization", 2024-04-25T14:50:47.963 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.963 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.963 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.963 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.964 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.964 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.964 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.964 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.964 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.964 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.964 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.964 INFO:teuthology.orchestra.run.smithi053.stdout: "proxy": { 2024-04-25T14:50:47.965 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "proxy", 2024-04-25T14:50:47.965 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.965 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.965 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.965 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.965 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.965 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.965 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.966 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.966 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.966 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.966 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.966 INFO:teuthology.orchestra.run.smithi053.stdout: "url": { 2024-04-25T14:50:47.966 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "url", 2024-04-25T14:50:47.966 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.967 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.967 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.967 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T14:50:47.967 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.967 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.967 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.967 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.967 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.967 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.968 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.968 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.968 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.968 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "test_orchestrator", 2024-04-25T14:50:47.968 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.968 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.969 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.969 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.969 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.969 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.969 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.969 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.969 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.969 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.970 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.970 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.970 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.970 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.970 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.970 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.970 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.970 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.971 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.971 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.971 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.971 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.971 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.971 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.971 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.971 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.972 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.972 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.972 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.972 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.972 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.972 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.972 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.972 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.972 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.973 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.973 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.973 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.973 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.973 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.973 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.973 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.973 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.973 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.974 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.974 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.974 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.974 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.974 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.974 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.974 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.974 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.974 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.975 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.975 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.975 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.975 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.975 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.975 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.975 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.976 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.976 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.976 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.976 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.976 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.976 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.976 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.976 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.976 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.977 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.977 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.977 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "volumes", 2024-04-25T14:50:47.977 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.977 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.977 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.977 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.978 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.978 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.978 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.978 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.978 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.978 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.978 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.978 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.978 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.979 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.979 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.979 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.979 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.979 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.979 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.979 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.979 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.980 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.980 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.980 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.980 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.980 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.980 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.980 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.980 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.981 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.981 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.981 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.981 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.981 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.981 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.981 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.981 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:47.981 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:47.982 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.982 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.982 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.982 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:47.982 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.982 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.982 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.982 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.982 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.983 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.983 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.983 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.983 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.983 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.983 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.983 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.983 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.983 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.984 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:47.984 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:47.984 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.984 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.984 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.984 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.984 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.985 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.985 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.985 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.985 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.985 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.985 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.985 INFO:teuthology.orchestra.run.smithi053.stdout: "max_concurrent_clones": { 2024-04-25T14:50:47.985 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "max_concurrent_clones", 2024-04-25T14:50:47.986 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.986 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.986 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.986 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "4", 2024-04-25T14:50:47.986 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.986 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.986 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.986 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T14:50:47.986 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.987 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.987 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.987 INFO:teuthology.orchestra.run.smithi053.stdout: "periodic_async_work": { 2024-04-25T14:50:47.987 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "periodic_async_work", 2024-04-25T14:50:47.987 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.987 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.987 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.987 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.987 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.988 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.988 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.988 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Periodically check for async work", 2024-04-25T14:50:47.988 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.988 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.988 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.988 INFO:teuthology.orchestra.run.smithi053.stdout: "snapshot_clone_delay": { 2024-04-25T14:50:47.988 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "snapshot_clone_delay", 2024-04-25T14:50:47.989 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:47.989 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.989 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.989 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0", 2024-04-25T14:50:47.989 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.989 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.989 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.989 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T14:50:47.989 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.990 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.990 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.990 INFO:teuthology.orchestra.run.smithi053.stdout: "snapshot_clone_no_wait": { 2024-04-25T14:50:47.990 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T14:50:47.990 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.990 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.990 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.990 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:47.990 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.991 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.991 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.991 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T14:50:47.991 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.991 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.991 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.991 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.991 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:47.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:47.992 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "zabbix", 2024-04-25T14:50:47.992 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:47.992 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:47.992 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:47.992 INFO:teuthology.orchestra.run.smithi053.stdout: "discovery_interval": { 2024-04-25T14:50:47.992 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "discovery_interval", 2024-04-25T14:50:47.992 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "uint", 2024-04-25T14:50:47.992 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.992 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.993 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "100", 2024-04-25T14:50:47.993 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.993 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.993 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.993 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.993 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.993 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.993 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.994 INFO:teuthology.orchestra.run.smithi053.stdout: "identifier": { 2024-04-25T14:50:47.994 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "identifier", 2024-04-25T14:50:47.994 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.994 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.994 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.994 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.994 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.994 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.994 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.995 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.995 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.995 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.995 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.995 INFO:teuthology.orchestra.run.smithi053.stdout: "interval": { 2024-04-25T14:50:47.995 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "interval", 2024-04-25T14:50:47.995 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:47.995 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.996 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:47.996 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "60", 2024-04-25T14:50:47.996 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.996 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.996 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:47.996 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.996 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.996 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.996 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.997 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:47.997 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:47.997 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:47.997 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.997 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.997 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:47.997 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:47.997 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:47.997 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:47.998 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:47.998 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:47.998 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:47.998 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:47.998 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:47.998 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:47.998 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:47.998 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:47.998 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:47.999 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:47.999 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:47.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:47.999 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:47.999 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:47.999 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:47.999 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:47.999 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:47.999 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:47.999 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.000 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.000 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.000 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.000 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.000 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.000 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.000 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:48.000 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:48.001 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.001 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.001 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.001 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:48.001 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.001 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.001 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.001 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.002 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.002 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.002 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.002 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.002 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.002 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.002 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.002 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.002 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.003 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.003 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:48.003 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:48.003 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.003 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.003 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.003 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.003 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.003 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.004 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.004 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.004 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.004 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.004 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.004 INFO:teuthology.orchestra.run.smithi053.stdout: "zabbix_host": { 2024-04-25T14:50:48.004 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "zabbix_host", 2024-04-25T14:50:48.004 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.005 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.005 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.005 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.005 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.005 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.005 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.005 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.005 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.006 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.006 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.006 INFO:teuthology.orchestra.run.smithi053.stdout: "zabbix_port": { 2024-04-25T14:50:48.006 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "zabbix_port", 2024-04-25T14:50:48.006 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:48.006 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.006 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.007 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10051", 2024-04-25T14:50:48.007 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.007 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.007 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.007 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.007 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.007 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.007 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.007 INFO:teuthology.orchestra.run.smithi053.stdout: "zabbix_sender": { 2024-04-25T14:50:48.008 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "zabbix_sender", 2024-04-25T14:50:48.008 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.008 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.008 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.008 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T14:50:48.008 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.008 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.008 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.008 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.009 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.009 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.009 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.009 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.009 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.009 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.009 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-04-25T14:50:48.009 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.009 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.009 INFO:teuthology.orchestra.run.smithi053.stdout: "modules": [ 2024-04-25T14:50:48.010 INFO:teuthology.orchestra.run.smithi053.stdout: "cephadm", 2024-04-25T14:50:48.010 INFO:teuthology.orchestra.run.smithi053.stdout: "dashboard", 2024-04-25T14:50:48.010 INFO:teuthology.orchestra.run.smithi053.stdout: "iostat", 2024-04-25T14:50:48.010 INFO:teuthology.orchestra.run.smithi053.stdout: "nfs", 2024-04-25T14:50:48.010 INFO:teuthology.orchestra.run.smithi053.stdout: "restful", 2024-04-25T14:50:48.010 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_schedule" 2024-04-25T14:50:48.010 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.010 INFO:teuthology.orchestra.run.smithi053.stdout: "available_modules": [ 2024-04-25T14:50:48.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.010 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "alerts", 2024-04-25T14:50:48.011 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:48.011 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:48.011 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:48.011 INFO:teuthology.orchestra.run.smithi053.stdout: "interval": { 2024-04-25T14:50:48.011 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "interval", 2024-04-25T14:50:48.011 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:48.011 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.011 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.011 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "60", 2024-04-25T14:50:48.012 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.012 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.012 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.012 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T14:50:48.012 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.012 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.012 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.012 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:48.013 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:48.013 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.013 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.013 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.013 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.013 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.013 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.013 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.013 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.014 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.014 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.014 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.014 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.014 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.014 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.014 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.014 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.015 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.015 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.015 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:48.015 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:48.015 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.015 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.015 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.015 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.016 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.016 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.016 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.016 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.016 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.016 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.016 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.017 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:48.017 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:48.017 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.017 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.017 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.017 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:48.017 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.017 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.018 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.018 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.018 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.018 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.018 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.018 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.018 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.018 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.019 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.019 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.019 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.019 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.019 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:48.019 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:48.019 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.019 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.020 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.020 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.020 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.020 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.020 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.020 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.020 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.020 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.020 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.021 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_destination": { 2024-04-25T14:50:48.021 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_destination", 2024-04-25T14:50:48.021 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.021 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.021 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.021 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.021 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.021 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.021 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.022 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Email address to send alerts to", 2024-04-25T14:50:48.022 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.022 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.022 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.022 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_from_name": { 2024-04-25T14:50:48.022 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_from_name", 2024-04-25T14:50:48.022 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.022 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.023 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.023 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "Ceph", 2024-04-25T14:50:48.023 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.023 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.023 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.023 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Email From: name", 2024-04-25T14:50:48.023 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.023 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.023 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.024 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_host": { 2024-04-25T14:50:48.024 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_host", 2024-04-25T14:50:48.024 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.024 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.024 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.024 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.024 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.024 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.024 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.025 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "SMTP server", 2024-04-25T14:50:48.025 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.025 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.025 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.025 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_password": { 2024-04-25T14:50:48.025 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_password", 2024-04-25T14:50:48.025 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.025 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.026 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.026 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.026 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.026 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.026 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.026 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Password to authenticate with", 2024-04-25T14:50:48.026 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.026 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.026 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.027 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_port": { 2024-04-25T14:50:48.027 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_port", 2024-04-25T14:50:48.027 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:48.027 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.027 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.027 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "465", 2024-04-25T14:50:48.027 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.027 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.028 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.028 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "SMTP port", 2024-04-25T14:50:48.028 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.028 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.028 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.028 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_sender": { 2024-04-25T14:50:48.028 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_sender", 2024-04-25T14:50:48.028 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.029 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.029 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.029 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.029 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.029 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.029 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.029 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "SMTP envelope sender", 2024-04-25T14:50:48.029 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.029 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.030 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.030 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_ssl": { 2024-04-25T14:50:48.030 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_ssl", 2024-04-25T14:50:48.030 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.030 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.030 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.030 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:48.031 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.031 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.031 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.031 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T14:50:48.031 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.031 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.031 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.031 INFO:teuthology.orchestra.run.smithi053.stdout: "smtp_user": { 2024-04-25T14:50:48.032 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "smtp_user", 2024-04-25T14:50:48.032 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.032 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.032 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.032 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.032 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.032 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.032 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.032 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "User to authenticate as", 2024-04-25T14:50:48.033 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.033 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.033 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.033 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.033 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.033 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "balancer", 2024-04-25T14:50:48.033 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:48.034 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:48.034 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:48.034 INFO:teuthology.orchestra.run.smithi053.stdout: "active": { 2024-04-25T14:50:48.034 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "active", 2024-04-25T14:50:48.034 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.034 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.034 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.034 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:48.035 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.035 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.035 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.035 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T14:50:48.035 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.035 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.035 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.036 INFO:teuthology.orchestra.run.smithi053.stdout: "begin_time": { 2024-04-25T14:50:48.036 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "begin_time", 2024-04-25T14:50:48.036 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.036 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.036 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.036 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0000", 2024-04-25T14:50:48.036 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.036 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.036 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.037 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T14:50:48.037 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T14:50:48.037 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.037 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.037 INFO:teuthology.orchestra.run.smithi053.stdout: "begin_weekday": { 2024-04-25T14:50:48.037 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "begin_weekday", 2024-04-25T14:50:48.037 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "uint", 2024-04-25T14:50:48.038 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.038 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.038 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0", 2024-04-25T14:50:48.038 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "0", 2024-04-25T14:50:48.038 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "6", 2024-04-25T14:50:48.038 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.038 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T14:50:48.038 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T14:50:48.038 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.039 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.039 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_compat_max_iterations": { 2024-04-25T14:50:48.039 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "crush_compat_max_iterations", 2024-04-25T14:50:48.039 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "uint", 2024-04-25T14:50:48.039 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.039 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.039 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "25", 2024-04-25T14:50:48.039 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "1", 2024-04-25T14:50:48.040 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "250", 2024-04-25T14:50:48.040 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.040 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T14:50:48.040 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.040 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.040 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.040 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_compat_metrics": { 2024-04-25T14:50:48.040 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "crush_compat_metrics", 2024-04-25T14:50:48.040 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.041 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.041 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.041 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T14:50:48.041 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.041 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.041 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.041 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T14:50:48.041 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T14:50:48.041 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.042 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.042 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_compat_step": { 2024-04-25T14:50:48.042 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "crush_compat_step", 2024-04-25T14:50:48.042 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2024-04-25T14:50:48.042 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.042 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.042 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0.5", 2024-04-25T14:50:48.042 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "0.001", 2024-04-25T14:50:48.043 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "0.999", 2024-04-25T14:50:48.043 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.043 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "aggressiveness of optimization", 2024-04-25T14:50:48.043 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T14:50:48.043 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.043 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.043 INFO:teuthology.orchestra.run.smithi053.stdout: "end_time": { 2024-04-25T14:50:48.043 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "end_time", 2024-04-25T14:50:48.043 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.044 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.044 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.044 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "2359", 2024-04-25T14:50:48.044 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.044 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.044 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.044 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T14:50:48.044 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T14:50:48.044 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.045 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.045 INFO:teuthology.orchestra.run.smithi053.stdout: "end_weekday": { 2024-04-25T14:50:48.045 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "end_weekday", 2024-04-25T14:50:48.045 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "uint", 2024-04-25T14:50:48.045 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.045 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.045 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0", 2024-04-25T14:50:48.046 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "0", 2024-04-25T14:50:48.046 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "6", 2024-04-25T14:50:48.046 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.046 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T14:50:48.046 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T14:50:48.046 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.046 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.046 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:48.046 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:48.047 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.047 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.047 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.047 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.047 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.047 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.047 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.047 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.047 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.048 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.048 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.048 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.048 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.048 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.048 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.048 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.048 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.048 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.049 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:48.049 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:48.049 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.049 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.049 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.049 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.049 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.049 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.050 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.050 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.050 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.050 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.050 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.050 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:48.050 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:48.050 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.050 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.051 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.051 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:48.051 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.051 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.051 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.051 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.051 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.051 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.051 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.052 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.052 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.052 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.052 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.052 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.052 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.052 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.052 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:48.053 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:48.053 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.053 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.053 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.053 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.053 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.053 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.053 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.053 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.054 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.054 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.054 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.054 INFO:teuthology.orchestra.run.smithi053.stdout: "min_score": { 2024-04-25T14:50:48.054 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "min_score", 2024-04-25T14:50:48.054 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2024-04-25T14:50:48.054 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.054 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.054 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0", 2024-04-25T14:50:48.055 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.055 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.055 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.055 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T14:50:48.055 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.055 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.055 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.055 INFO:teuthology.orchestra.run.smithi053.stdout: "mode": { 2024-04-25T14:50:48.056 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "mode", 2024-04-25T14:50:48.056 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.056 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.056 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.056 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "upmap", 2024-04-25T14:50:48.056 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.056 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.056 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.056 INFO:teuthology.orchestra.run.smithi053.stdout: "crush-compat", 2024-04-25T14:50:48.057 INFO:teuthology.orchestra.run.smithi053.stdout: "none", 2024-04-25T14:50:48.057 INFO:teuthology.orchestra.run.smithi053.stdout: "read", 2024-04-25T14:50:48.057 INFO:teuthology.orchestra.run.smithi053.stdout: "upmap", 2024-04-25T14:50:48.057 INFO:teuthology.orchestra.run.smithi053.stdout: "upmap-read" 2024-04-25T14:50:48.057 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.057 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Balancer mode", 2024-04-25T14:50:48.057 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.057 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.057 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.058 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_ids": { 2024-04-25T14:50:48.058 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "pool_ids", 2024-04-25T14:50:48.058 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.058 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.058 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.058 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.058 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.058 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.059 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.059 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T14:50:48.059 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.059 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.059 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.059 INFO:teuthology.orchestra.run.smithi053.stdout: "sleep_interval": { 2024-04-25T14:50:48.059 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "sleep_interval", 2024-04-25T14:50:48.059 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:48.060 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.060 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.060 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "60", 2024-04-25T14:50:48.060 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.060 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.060 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.060 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T14:50:48.060 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.060 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.061 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.061 INFO:teuthology.orchestra.run.smithi053.stdout: "upmap_max_deviation": { 2024-04-25T14:50:48.061 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "upmap_max_deviation", 2024-04-25T14:50:48.061 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:48.061 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.061 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.061 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "5", 2024-04-25T14:50:48.061 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "1", 2024-04-25T14:50:48.061 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.062 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.062 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T14:50:48.062 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T14:50:48.062 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.062 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.062 INFO:teuthology.orchestra.run.smithi053.stdout: "upmap_max_optimizations": { 2024-04-25T14:50:48.062 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "upmap_max_optimizations", 2024-04-25T14:50:48.062 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "uint", 2024-04-25T14:50:48.063 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.063 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.063 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2024-04-25T14:50:48.063 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.063 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.063 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.063 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T14:50:48.063 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.063 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.063 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.064 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.064 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.064 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "cephadm", 2024-04-25T14:50:48.064 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:48.064 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:48.064 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:48.065 INFO:teuthology.orchestra.run.smithi053.stdout: "agent_down_multiplier": { 2024-04-25T14:50:48.065 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "agent_down_multiplier", 2024-04-25T14:50:48.065 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2024-04-25T14:50:48.065 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.065 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.065 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "3.0", 2024-04-25T14:50:48.065 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.065 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.065 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.066 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T14:50:48.066 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.066 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.066 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.066 INFO:teuthology.orchestra.run.smithi053.stdout: "agent_refresh_rate": { 2024-04-25T14:50:48.066 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "agent_refresh_rate", 2024-04-25T14:50:48.066 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:48.067 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.067 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.067 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "20", 2024-04-25T14:50:48.067 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.067 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.067 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.067 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T14:50:48.067 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.067 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.067 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.068 INFO:teuthology.orchestra.run.smithi053.stdout: "agent_starting_port": { 2024-04-25T14:50:48.068 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "agent_starting_port", 2024-04-25T14:50:48.068 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:48.068 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.068 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.068 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "4721", 2024-04-25T14:50:48.068 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.068 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.069 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.069 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T14:50:48.069 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.069 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.069 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.069 INFO:teuthology.orchestra.run.smithi053.stdout: "allow_ptrace": { 2024-04-25T14:50:48.069 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "allow_ptrace", 2024-04-25T14:50:48.069 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.070 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.070 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.070 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.070 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.070 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.070 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.070 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T14:50:48.070 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T14:50:48.070 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.071 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.071 INFO:teuthology.orchestra.run.smithi053.stdout: "autotune_interval": { 2024-04-25T14:50:48.071 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "autotune_interval", 2024-04-25T14:50:48.071 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:48.071 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.071 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.071 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "600", 2024-04-25T14:50:48.071 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.072 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.072 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.072 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T14:50:48.072 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.072 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.072 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.072 INFO:teuthology.orchestra.run.smithi053.stdout: "autotune_memory_target_ratio": { 2024-04-25T14:50:48.072 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T14:50:48.073 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2024-04-25T14:50:48.073 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.073 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.073 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0.7", 2024-04-25T14:50:48.073 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.073 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.073 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.073 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T14:50:48.074 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.074 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.074 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.074 INFO:teuthology.orchestra.run.smithi053.stdout: "cephadm_log_destination": { 2024-04-25T14:50:48.074 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "cephadm_log_destination", 2024-04-25T14:50:48.074 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.074 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.074 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.075 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.075 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.075 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.075 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.075 INFO:teuthology.orchestra.run.smithi053.stdout: "file", 2024-04-25T14:50:48.075 INFO:teuthology.orchestra.run.smithi053.stdout: "file,syslog", 2024-04-25T14:50:48.075 INFO:teuthology.orchestra.run.smithi053.stdout: "syslog" 2024-04-25T14:50:48.075 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.076 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T14:50:48.076 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.076 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.076 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.076 INFO:teuthology.orchestra.run.smithi053.stdout: "cgroups_split": { 2024-04-25T14:50:48.076 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "cgroups_split", 2024-04-25T14:50:48.076 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.076 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.077 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.077 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:48.077 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.077 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.077 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.077 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T14:50:48.077 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.077 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.078 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.078 INFO:teuthology.orchestra.run.smithi053.stdout: "config_checks_enabled": { 2024-04-25T14:50:48.078 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "config_checks_enabled", 2024-04-25T14:50:48.078 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.078 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.078 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.078 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.079 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.079 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.079 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.079 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T14:50:48.079 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.079 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.079 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.079 INFO:teuthology.orchestra.run.smithi053.stdout: "config_dashboard": { 2024-04-25T14:50:48.080 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "config_dashboard", 2024-04-25T14:50:48.080 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.080 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.080 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.080 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:48.080 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.080 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.080 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.080 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T14:50:48.081 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.081 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.081 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.081 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_alertmanager": { 2024-04-25T14:50:48.081 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_alertmanager", 2024-04-25T14:50:48.081 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.081 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.081 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.081 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T14:50:48.082 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.082 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.082 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.082 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Prometheus container image", 2024-04-25T14:50:48.082 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.082 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.082 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.082 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_base": { 2024-04-25T14:50:48.083 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_base", 2024-04-25T14:50:48.083 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.083 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.083 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.083 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T14:50:48.083 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.083 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.083 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.083 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Container image name, without the tag", 2024-04-25T14:50:48.084 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.084 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.084 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.084 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_elasticsearch": { 2024-04-25T14:50:48.084 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_elasticsearch", 2024-04-25T14:50:48.084 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.084 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.085 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.085 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T14:50:48.085 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.085 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.085 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.085 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "elasticsearch container image", 2024-04-25T14:50:48.085 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.085 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.085 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.086 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_grafana": { 2024-04-25T14:50:48.086 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_grafana", 2024-04-25T14:50:48.086 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.086 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.086 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.086 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T14:50:48.086 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.086 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.087 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.087 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Prometheus container image", 2024-04-25T14:50:48.087 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.087 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.087 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.087 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_haproxy": { 2024-04-25T14:50:48.087 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_haproxy", 2024-04-25T14:50:48.087 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.088 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.088 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.088 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T14:50:48.088 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.088 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.088 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.088 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "HAproxy container image", 2024-04-25T14:50:48.088 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.089 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.089 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.089 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_jaeger_agent": { 2024-04-25T14:50:48.089 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_jaeger_agent", 2024-04-25T14:50:48.089 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.089 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.089 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.089 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T14:50:48.089 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.090 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.090 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.090 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Jaeger agent container image", 2024-04-25T14:50:48.090 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.090 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.090 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.090 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_jaeger_collector": { 2024-04-25T14:50:48.090 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_jaeger_collector", 2024-04-25T14:50:48.091 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.091 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.091 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.091 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T14:50:48.091 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.091 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.091 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.091 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Jaeger collector container image", 2024-04-25T14:50:48.091 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.092 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.092 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.092 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_jaeger_query": { 2024-04-25T14:50:48.092 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_jaeger_query", 2024-04-25T14:50:48.092 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.092 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.092 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.093 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T14:50:48.093 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.093 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.093 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.093 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Jaeger query container image", 2024-04-25T14:50:48.093 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.093 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.093 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.093 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_keepalived": { 2024-04-25T14:50:48.094 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_keepalived", 2024-04-25T14:50:48.094 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.094 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.094 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.094 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T14:50:48.094 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.094 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.094 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.095 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Keepalived container image", 2024-04-25T14:50:48.095 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.095 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.095 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.095 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_loki": { 2024-04-25T14:50:48.095 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_loki", 2024-04-25T14:50:48.095 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.095 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.095 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.096 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T14:50:48.096 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.096 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.096 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.096 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Loki container image", 2024-04-25T14:50:48.096 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.096 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.096 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.097 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_node_exporter": { 2024-04-25T14:50:48.097 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_node_exporter", 2024-04-25T14:50:48.097 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.097 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.097 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.097 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T14:50:48.097 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.097 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.097 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.097 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Prometheus container image", 2024-04-25T14:50:48.098 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.098 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.098 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.098 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_nvmeof": { 2024-04-25T14:50:48.098 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_nvmeof", 2024-04-25T14:50:48.098 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.098 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.098 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.099 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T14:50:48.099 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.099 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.099 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.099 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Nvme-of container image", 2024-04-25T14:50:48.099 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.099 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.099 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.099 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_prometheus": { 2024-04-25T14:50:48.100 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_prometheus", 2024-04-25T14:50:48.100 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.100 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.100 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.100 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T14:50:48.100 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.100 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.100 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.100 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Prometheus container image", 2024-04-25T14:50:48.100 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.101 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.101 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.101 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_promtail": { 2024-04-25T14:50:48.101 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_promtail", 2024-04-25T14:50:48.101 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.101 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.101 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.101 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T14:50:48.101 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.102 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.102 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.102 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Promtail container image", 2024-04-25T14:50:48.102 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.102 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.102 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.102 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_samba": { 2024-04-25T14:50:48.102 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_samba", 2024-04-25T14:50:48.103 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.103 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.103 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.103 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T14:50:48.103 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.103 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.103 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.103 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Samba/SMB container image", 2024-04-25T14:50:48.103 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.104 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.104 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.104 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image_snmp_gateway": { 2024-04-25T14:50:48.104 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_image_snmp_gateway", 2024-04-25T14:50:48.104 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.104 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.104 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.104 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T14:50:48.105 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.105 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.105 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.105 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "SNMP Gateway container image", 2024-04-25T14:50:48.105 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.105 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.105 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.105 INFO:teuthology.orchestra.run.smithi053.stdout: "container_init": { 2024-04-25T14:50:48.106 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "container_init", 2024-04-25T14:50:48.106 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.106 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.106 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.106 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:48.106 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.106 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.106 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.106 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T14:50:48.107 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.107 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.107 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.107 INFO:teuthology.orchestra.run.smithi053.stdout: "daemon_cache_timeout": { 2024-04-25T14:50:48.107 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "daemon_cache_timeout", 2024-04-25T14:50:48.107 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:48.107 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.107 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.107 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "600", 2024-04-25T14:50:48.108 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.108 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.108 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.108 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T14:50:48.108 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.108 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.108 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.108 INFO:teuthology.orchestra.run.smithi053.stdout: "default_cephadm_command_timeout": { 2024-04-25T14:50:48.108 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T14:50:48.109 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:48.109 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.109 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.109 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "900", 2024-04-25T14:50:48.109 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.109 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.109 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.109 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T14:50:48.109 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.110 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.110 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.110 INFO:teuthology.orchestra.run.smithi053.stdout: "default_registry": { 2024-04-25T14:50:48.110 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "default_registry", 2024-04-25T14:50:48.110 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.110 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.110 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.110 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "docker.io", 2024-04-25T14:50:48.110 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.111 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.111 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.111 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T14:50:48.111 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.111 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.111 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.111 INFO:teuthology.orchestra.run.smithi053.stdout: "device_cache_timeout": { 2024-04-25T14:50:48.111 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "device_cache_timeout", 2024-04-25T14:50:48.111 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:48.112 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.112 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.112 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "1800", 2024-04-25T14:50:48.112 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.112 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.112 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.112 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "seconds to cache device inventory", 2024-04-25T14:50:48.112 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.112 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.112 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.113 INFO:teuthology.orchestra.run.smithi053.stdout: "device_enhanced_scan": { 2024-04-25T14:50:48.113 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "device_enhanced_scan", 2024-04-25T14:50:48.113 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.113 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.113 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.113 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.113 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.113 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.113 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.114 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T14:50:48.114 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.114 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.114 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.114 INFO:teuthology.orchestra.run.smithi053.stdout: "facts_cache_timeout": { 2024-04-25T14:50:48.114 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "facts_cache_timeout", 2024-04-25T14:50:48.114 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:48.114 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.114 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.115 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "60", 2024-04-25T14:50:48.115 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.115 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.115 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.115 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "seconds to cache host facts data", 2024-04-25T14:50:48.115 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.115 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.115 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.116 INFO:teuthology.orchestra.run.smithi053.stdout: "grafana_dashboards_path": { 2024-04-25T14:50:48.116 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "grafana_dashboards_path", 2024-04-25T14:50:48.116 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.116 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.116 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.116 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T14:50:48.116 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.116 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.116 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.117 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T14:50:48.117 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.117 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.117 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.117 INFO:teuthology.orchestra.run.smithi053.stdout: "host_check_interval": { 2024-04-25T14:50:48.117 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "host_check_interval", 2024-04-25T14:50:48.117 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:48.118 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.118 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.118 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "600", 2024-04-25T14:50:48.118 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.118 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.118 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.118 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how frequently to perform a host check", 2024-04-25T14:50:48.118 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.118 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.119 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.119 INFO:teuthology.orchestra.run.smithi053.stdout: "hw_monitoring": { 2024-04-25T14:50:48.119 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "hw_monitoring", 2024-04-25T14:50:48.119 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.119 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.119 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.119 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.120 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.120 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.120 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.120 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T14:50:48.120 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.120 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.120 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.120 INFO:teuthology.orchestra.run.smithi053.stdout: "inventory_list_all": { 2024-04-25T14:50:48.121 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "inventory_list_all", 2024-04-25T14:50:48.121 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.121 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.121 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.121 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.121 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.121 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.121 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.121 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T14:50:48.122 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.122 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.122 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.122 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:48.122 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:48.122 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.122 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.123 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.123 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.123 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.123 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.123 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.123 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.123 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.123 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.123 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.124 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.124 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.124 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.124 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.124 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.124 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.124 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.125 INFO:teuthology.orchestra.run.smithi053.stdout: "log_refresh_metadata": { 2024-04-25T14:50:48.125 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_refresh_metadata", 2024-04-25T14:50:48.125 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.125 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.125 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.125 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.125 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.125 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.125 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.126 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T14:50:48.126 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.126 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.126 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.126 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:48.126 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:48.126 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.126 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.127 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.127 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:48.127 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.127 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.127 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.127 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T14:50:48.127 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.127 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.127 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.128 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:48.128 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:48.128 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.128 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.128 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.128 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:48.128 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.128 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.129 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.129 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.129 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.129 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.129 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.129 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.129 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.129 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.129 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.129 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.130 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.130 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.130 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:48.130 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:48.130 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.130 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.130 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.130 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.131 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.131 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.131 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.131 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.131 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.131 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.131 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.131 INFO:teuthology.orchestra.run.smithi053.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T14:50:48.131 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T14:50:48.132 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.132 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.132 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.132 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.132 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.132 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.132 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.132 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T14:50:48.132 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.133 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.133 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.133 INFO:teuthology.orchestra.run.smithi053.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T14:50:48.133 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T14:50:48.133 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.133 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.133 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.133 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "*", 2024-04-25T14:50:48.133 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.134 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.134 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.134 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T14:50:48.134 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.134 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.134 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.134 INFO:teuthology.orchestra.run.smithi053.stdout: "max_count_per_host": { 2024-04-25T14:50:48.134 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "max_count_per_host", 2024-04-25T14:50:48.135 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:48.135 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.135 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.135 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2024-04-25T14:50:48.135 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.135 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.135 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.135 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "max number of daemons per service per host", 2024-04-25T14:50:48.135 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.136 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.136 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.136 INFO:teuthology.orchestra.run.smithi053.stdout: "max_osd_draining_count": { 2024-04-25T14:50:48.136 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "max_osd_draining_count", 2024-04-25T14:50:48.136 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:48.136 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.136 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.136 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2024-04-25T14:50:48.137 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.137 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.137 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.137 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T14:50:48.137 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.137 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.137 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.137 INFO:teuthology.orchestra.run.smithi053.stdout: "migration_current": { 2024-04-25T14:50:48.138 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "migration_current", 2024-04-25T14:50:48.138 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:48.138 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.138 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.138 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.138 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.138 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.138 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.138 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "internal - do not modify", 2024-04-25T14:50:48.139 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.139 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.139 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.139 INFO:teuthology.orchestra.run.smithi053.stdout: "mode": { 2024-04-25T14:50:48.139 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "mode", 2024-04-25T14:50:48.139 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.139 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.139 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.140 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "root", 2024-04-25T14:50:48.140 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.140 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.140 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.140 INFO:teuthology.orchestra.run.smithi053.stdout: "cephadm-package", 2024-04-25T14:50:48.140 INFO:teuthology.orchestra.run.smithi053.stdout: "root" 2024-04-25T14:50:48.140 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.140 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T14:50:48.140 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.141 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.141 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.141 INFO:teuthology.orchestra.run.smithi053.stdout: "oob_default_addr": { 2024-04-25T14:50:48.141 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "oob_default_addr", 2024-04-25T14:50:48.141 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.141 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.141 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.142 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "169.254.1.1", 2024-04-25T14:50:48.142 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.142 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.142 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.142 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T14:50:48.142 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.142 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.142 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.143 INFO:teuthology.orchestra.run.smithi053.stdout: "prometheus_alerts_path": { 2024-04-25T14:50:48.143 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "prometheus_alerts_path", 2024-04-25T14:50:48.143 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.143 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.143 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.143 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T14:50:48.143 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.143 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.144 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.144 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T14:50:48.144 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.144 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.144 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.144 INFO:teuthology.orchestra.run.smithi053.stdout: "registry_insecure": { 2024-04-25T14:50:48.144 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "registry_insecure", 2024-04-25T14:50:48.144 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.145 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.145 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.145 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.145 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.145 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.145 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.145 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T14:50:48.145 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.146 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.146 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.146 INFO:teuthology.orchestra.run.smithi053.stdout: "registry_password": { 2024-04-25T14:50:48.146 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "registry_password", 2024-04-25T14:50:48.146 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.146 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.146 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.146 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.147 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.147 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.147 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.147 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T14:50:48.147 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.147 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.147 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.147 INFO:teuthology.orchestra.run.smithi053.stdout: "registry_url": { 2024-04-25T14:50:48.147 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "registry_url", 2024-04-25T14:50:48.148 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.148 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.148 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.148 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.148 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.148 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.148 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.148 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T14:50:48.148 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.149 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.149 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.149 INFO:teuthology.orchestra.run.smithi053.stdout: "registry_username": { 2024-04-25T14:50:48.149 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "registry_username", 2024-04-25T14:50:48.149 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.149 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.149 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.149 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.150 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.150 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.150 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.150 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T14:50:48.150 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.150 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.150 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.150 INFO:teuthology.orchestra.run.smithi053.stdout: "secure_monitoring_stack": { 2024-04-25T14:50:48.150 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "secure_monitoring_stack", 2024-04-25T14:50:48.151 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.151 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.151 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.151 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.151 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.151 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.151 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.151 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T14:50:48.151 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.151 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.152 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.152 INFO:teuthology.orchestra.run.smithi053.stdout: "service_discovery_port": { 2024-04-25T14:50:48.152 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "service_discovery_port", 2024-04-25T14:50:48.152 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:48.152 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.152 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.152 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "8765", 2024-04-25T14:50:48.152 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.152 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.153 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.153 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "cephadm service discovery port", 2024-04-25T14:50:48.153 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.153 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.153 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.153 INFO:teuthology.orchestra.run.smithi053.stdout: "ssh_config_file": { 2024-04-25T14:50:48.153 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ssh_config_file", 2024-04-25T14:50:48.153 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.153 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.154 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.154 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.154 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.154 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.154 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.154 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T14:50:48.154 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.154 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.154 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.155 INFO:teuthology.orchestra.run.smithi053.stdout: "use_agent": { 2024-04-25T14:50:48.155 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "use_agent", 2024-04-25T14:50:48.155 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.155 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.155 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.155 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.155 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.155 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.155 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.156 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T14:50:48.156 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.156 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.156 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.156 INFO:teuthology.orchestra.run.smithi053.stdout: "use_repo_digest": { 2024-04-25T14:50:48.156 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "use_repo_digest", 2024-04-25T14:50:48.156 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.156 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.156 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.157 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:48.157 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.157 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.157 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.157 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T14:50:48.157 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.157 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.157 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.157 INFO:teuthology.orchestra.run.smithi053.stdout: "warn_on_failed_host_check": { 2024-04-25T14:50:48.158 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "warn_on_failed_host_check", 2024-04-25T14:50:48.158 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.158 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.158 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.158 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:48.158 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.158 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.158 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.158 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T14:50:48.159 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.159 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.159 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.159 INFO:teuthology.orchestra.run.smithi053.stdout: "warn_on_stray_daemons": { 2024-04-25T14:50:48.159 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "warn_on_stray_daemons", 2024-04-25T14:50:48.159 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.159 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.159 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.159 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:48.160 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.160 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.160 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.160 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T14:50:48.160 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.160 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.160 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.160 INFO:teuthology.orchestra.run.smithi053.stdout: "warn_on_stray_hosts": { 2024-04-25T14:50:48.161 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "warn_on_stray_hosts", 2024-04-25T14:50:48.161 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.161 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.161 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.161 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:48.161 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.161 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.161 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.161 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T14:50:48.161 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.162 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.162 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.162 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.162 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.162 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "crash", 2024-04-25T14:50:48.162 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:48.162 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:48.163 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:48.163 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:48.163 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:48.163 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.163 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.163 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.163 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.163 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.163 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.164 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.164 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.164 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.164 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.164 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.164 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.164 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.164 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.164 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.164 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.165 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.165 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.165 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:48.165 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:48.165 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.165 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.165 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.165 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.166 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.166 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.166 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.166 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.166 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.166 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.166 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.166 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:48.167 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:48.167 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.167 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.167 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.167 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:48.167 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.167 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.167 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.167 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.168 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.168 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.168 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.168 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.168 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.168 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.168 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.168 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.168 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.168 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.169 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:48.169 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:48.169 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.169 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.169 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.169 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.169 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.169 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.169 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.170 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.170 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.170 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.170 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.170 INFO:teuthology.orchestra.run.smithi053.stdout: "retain_interval": { 2024-04-25T14:50:48.170 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "retain_interval", 2024-04-25T14:50:48.170 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:48.170 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.171 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.171 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "31536000", 2024-04-25T14:50:48.171 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.171 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.171 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.171 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T14:50:48.171 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.171 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.171 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.172 INFO:teuthology.orchestra.run.smithi053.stdout: "warn_recent_interval": { 2024-04-25T14:50:48.172 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "warn_recent_interval", 2024-04-25T14:50:48.172 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:48.172 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.172 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.172 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "1209600", 2024-04-25T14:50:48.172 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.172 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.172 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.172 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T14:50:48.173 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.173 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.173 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.173 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.173 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.173 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "dashboard", 2024-04-25T14:50:48.173 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:48.173 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:48.174 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:48.174 INFO:teuthology.orchestra.run.smithi053.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T14:50:48.174 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T14:50:48.174 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:48.174 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.174 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.174 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2024-04-25T14:50:48.174 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.174 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.175 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.175 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.175 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.175 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.175 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.175 INFO:teuthology.orchestra.run.smithi053.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T14:50:48.175 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T14:50:48.175 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.176 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.176 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.176 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.176 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.176 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.176 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.176 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.176 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.176 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.177 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.177 INFO:teuthology.orchestra.run.smithi053.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T14:50:48.177 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T14:50:48.177 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.177 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.177 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.177 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:48.177 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.178 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.178 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.178 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.178 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.178 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.178 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.178 INFO:teuthology.orchestra.run.smithi053.stdout: "AUDIT_API_ENABLED": { 2024-04-25T14:50:48.179 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T14:50:48.179 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.179 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.179 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.179 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.179 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.179 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.179 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.180 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.180 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.180 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.180 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.180 INFO:teuthology.orchestra.run.smithi053.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T14:50:48.180 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T14:50:48.180 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.180 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.181 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.181 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:48.181 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.181 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.181 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.181 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.181 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.181 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.181 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.182 INFO:teuthology.orchestra.run.smithi053.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T14:50:48.182 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T14:50:48.182 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.182 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.182 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.182 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:48.182 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.182 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.183 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.183 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.183 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.183 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.183 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.183 INFO:teuthology.orchestra.run.smithi053.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T14:50:48.183 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T14:50:48.183 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.184 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.184 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.184 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:48.184 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.184 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.184 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.184 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.184 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.185 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.185 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.185 INFO:teuthology.orchestra.run.smithi053.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T14:50:48.185 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T14:50:48.185 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.185 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.185 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.186 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:48.186 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.186 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.186 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.186 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.186 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.186 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.186 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.187 INFO:teuthology.orchestra.run.smithi053.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T14:50:48.187 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T14:50:48.187 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.187 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.187 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.187 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:48.187 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.187 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.187 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.188 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.188 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.188 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.188 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.188 INFO:teuthology.orchestra.run.smithi053.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T14:50:48.188 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T14:50:48.188 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.189 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.189 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.189 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:48.189 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.189 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.189 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.189 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.189 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.190 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.190 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.190 INFO:teuthology.orchestra.run.smithi053.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T14:50:48.190 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T14:50:48.190 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.190 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.190 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.190 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:48.191 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.191 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.191 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.191 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.191 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.191 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.191 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.191 INFO:teuthology.orchestra.run.smithi053.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T14:50:48.192 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T14:50:48.192 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.192 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.192 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.192 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:48.192 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.192 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.192 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.193 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.193 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.193 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.193 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.193 INFO:teuthology.orchestra.run.smithi053.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T14:50:48.193 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T14:50:48.193 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.193 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.194 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.194 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:48.194 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.194 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.194 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.194 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.194 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.194 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.194 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.195 INFO:teuthology.orchestra.run.smithi053.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T14:50:48.195 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T14:50:48.195 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.195 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.195 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.195 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.195 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.195 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.196 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.196 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.196 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.196 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.196 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.196 INFO:teuthology.orchestra.run.smithi053.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T14:50:48.196 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T14:50:48.196 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.196 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.197 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.197 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "admin", 2024-04-25T14:50:48.197 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.197 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.197 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.197 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.197 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.197 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.197 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.198 INFO:teuthology.orchestra.run.smithi053.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T14:50:48.198 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T14:50:48.198 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.198 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.198 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.198 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:48.198 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.198 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.198 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.199 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.199 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.199 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.199 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.199 INFO:teuthology.orchestra.run.smithi053.stdout: "GRAFANA_API_URL": { 2024-04-25T14:50:48.199 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "GRAFANA_API_URL", 2024-04-25T14:50:48.199 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.199 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.199 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.200 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.200 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.200 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.200 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.200 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.200 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.200 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.200 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.201 INFO:teuthology.orchestra.run.smithi053.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T14:50:48.201 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T14:50:48.201 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.201 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.201 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.201 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "admin", 2024-04-25T14:50:48.201 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.201 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.201 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.201 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.202 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.202 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.202 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.202 INFO:teuthology.orchestra.run.smithi053.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T14:50:48.202 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T14:50:48.202 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.202 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.202 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.203 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.203 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.203 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.203 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.203 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.203 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.203 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.203 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.203 INFO:teuthology.orchestra.run.smithi053.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T14:50:48.204 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T14:50:48.204 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.204 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.204 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.204 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.204 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.204 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.204 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.204 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.205 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.205 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.205 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.205 INFO:teuthology.orchestra.run.smithi053.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T14:50:48.205 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T14:50:48.205 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.205 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.205 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.205 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:48.206 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.206 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.206 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.206 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.206 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.206 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.206 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.206 INFO:teuthology.orchestra.run.smithi053.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T14:50:48.207 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T14:50:48.207 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.207 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.207 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.207 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.207 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.207 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.207 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.207 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.208 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.208 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.208 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.208 INFO:teuthology.orchestra.run.smithi053.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T14:50:48.208 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T14:50:48.208 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.208 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.208 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.208 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "[]", 2024-04-25T14:50:48.209 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.209 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.209 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.209 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.209 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.209 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.209 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.209 INFO:teuthology.orchestra.run.smithi053.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T14:50:48.209 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T14:50:48.210 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.210 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.210 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.210 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "{}", 2024-04-25T14:50:48.210 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.210 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.210 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.210 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.210 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.211 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.211 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.211 INFO:teuthology.orchestra.run.smithi053.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T14:50:48.211 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T14:50:48.211 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.211 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.211 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.212 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.212 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.212 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.212 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.212 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.212 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.212 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.212 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.213 INFO:teuthology.orchestra.run.smithi053.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T14:50:48.213 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T14:50:48.213 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.213 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.213 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.213 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:48.213 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.213 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.214 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.214 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.214 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.214 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.214 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.214 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T14:50:48.214 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T14:50:48.214 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.215 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.215 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.215 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.215 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.215 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.215 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.215 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.215 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.215 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.216 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.216 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T14:50:48.216 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T14:50:48.216 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.216 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.216 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.216 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.216 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.217 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.217 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.217 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.217 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.217 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.217 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.217 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T14:50:48.217 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T14:50:48.218 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.218 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.218 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.218 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:48.218 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.218 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.218 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.218 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.219 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.219 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.219 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.219 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T14:50:48.219 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T14:50:48.219 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.219 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.219 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.220 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:48.220 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.220 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.220 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.220 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.220 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.220 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.220 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.221 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T14:50:48.221 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T14:50:48.221 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.221 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.221 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.221 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.221 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.221 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.222 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.222 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.222 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.222 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.222 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.222 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T14:50:48.222 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T14:50:48.222 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.223 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.223 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.223 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.223 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.223 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.223 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.223 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.223 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.223 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.224 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.224 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T14:50:48.224 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T14:50:48.224 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.224 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.224 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.224 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.224 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.225 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.225 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.225 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.225 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.225 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.225 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.225 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T14:50:48.225 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T14:50:48.226 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.226 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.226 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.226 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:48.226 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.226 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.226 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.226 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.226 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.227 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.227 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.227 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T14:50:48.227 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T14:50:48.227 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.227 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.227 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.227 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T14:50:48.228 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.228 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.228 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.228 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.228 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.228 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.228 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.228 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T14:50:48.229 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T14:50:48.229 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:48.229 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.229 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.229 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2024-04-25T14:50:48.229 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.229 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.229 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.229 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.229 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.230 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.230 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.230 INFO:teuthology.orchestra.run.smithi053.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T14:50:48.230 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T14:50:48.230 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:48.230 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.230 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.230 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "8", 2024-04-25T14:50:48.231 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.231 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.231 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.231 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.231 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.231 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.231 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.231 INFO:teuthology.orchestra.run.smithi053.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T14:50:48.231 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T14:50:48.232 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:48.232 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.232 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.232 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "45", 2024-04-25T14:50:48.232 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.232 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.232 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.232 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.232 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.232 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.233 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.233 INFO:teuthology.orchestra.run.smithi053.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T14:50:48.233 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T14:50:48.233 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.233 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.233 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.233 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.233 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.233 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.234 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.234 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.234 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.234 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.234 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.234 INFO:teuthology.orchestra.run.smithi053.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T14:50:48.234 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T14:50:48.234 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.235 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.235 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.235 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "admin", 2024-04-25T14:50:48.235 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.235 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.235 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.235 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.235 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.235 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.235 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.236 INFO:teuthology.orchestra.run.smithi053.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T14:50:48.236 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T14:50:48.236 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.236 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.236 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.236 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.236 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.236 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.236 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.237 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.237 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.237 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.237 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.237 INFO:teuthology.orchestra.run.smithi053.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T14:50:48.237 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T14:50:48.237 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.237 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.238 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.238 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:48.238 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.238 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.238 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.238 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.238 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.238 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.238 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.239 INFO:teuthology.orchestra.run.smithi053.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T14:50:48.239 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T14:50:48.239 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.239 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.239 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.239 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.239 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.239 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.239 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.240 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.240 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.240 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.240 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.240 INFO:teuthology.orchestra.run.smithi053.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T14:50:48.240 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T14:50:48.240 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:48.240 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.240 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.241 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0", 2024-04-25T14:50:48.241 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.241 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.241 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.241 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.241 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.241 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.241 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.241 INFO:teuthology.orchestra.run.smithi053.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T14:50:48.242 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T14:50:48.242 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:48.242 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.242 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.242 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2024-04-25T14:50:48.242 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.242 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.242 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.243 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.243 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.243 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.243 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.243 INFO:teuthology.orchestra.run.smithi053.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T14:50:48.243 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T14:50:48.243 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:48.243 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.244 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.244 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "5", 2024-04-25T14:50:48.244 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.244 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.244 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.244 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.244 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.244 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.245 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.245 INFO:teuthology.orchestra.run.smithi053.stdout: "cross_origin_url": { 2024-04-25T14:50:48.245 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "cross_origin_url", 2024-04-25T14:50:48.245 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.245 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.245 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.245 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.246 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.246 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.246 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.246 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.246 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.246 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.246 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.247 INFO:teuthology.orchestra.run.smithi053.stdout: "crt_file": { 2024-04-25T14:50:48.247 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "crt_file", 2024-04-25T14:50:48.247 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.247 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.247 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.247 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.247 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.247 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.247 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.248 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.248 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.248 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.248 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.248 INFO:teuthology.orchestra.run.smithi053.stdout: "debug": { 2024-04-25T14:50:48.248 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "debug", 2024-04-25T14:50:48.248 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.248 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.249 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.249 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.249 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.249 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.249 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.249 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Enable/disable debug options", 2024-04-25T14:50:48.249 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.249 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.249 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.250 INFO:teuthology.orchestra.run.smithi053.stdout: "jwt_token_ttl": { 2024-04-25T14:50:48.250 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "jwt_token_ttl", 2024-04-25T14:50:48.250 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:48.250 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.250 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.250 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "28800", 2024-04-25T14:50:48.250 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.251 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.251 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.251 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.251 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.251 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.251 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.251 INFO:teuthology.orchestra.run.smithi053.stdout: "key_file": { 2024-04-25T14:50:48.251 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "key_file", 2024-04-25T14:50:48.252 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.252 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.252 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.252 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.252 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.252 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.252 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.252 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.252 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.252 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.253 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.253 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:48.253 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:48.253 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.253 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.253 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.253 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.253 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.253 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.254 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.254 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.254 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.254 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.254 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.254 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.254 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.254 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.254 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.255 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.255 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.255 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.255 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:48.255 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:48.255 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.255 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.255 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.255 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.256 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.256 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.256 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.256 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.256 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.256 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.256 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.256 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:48.257 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:48.257 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.257 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.257 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.257 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:48.257 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.257 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.257 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.257 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.258 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.258 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.258 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.258 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.258 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.258 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.258 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.258 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.258 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.259 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.259 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:48.259 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:48.259 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.259 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.259 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.259 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.260 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.260 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.260 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.260 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.260 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.260 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.260 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.260 INFO:teuthology.orchestra.run.smithi053.stdout: "motd": { 2024-04-25T14:50:48.260 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "motd", 2024-04-25T14:50:48.261 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.261 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.261 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.261 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.261 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.261 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.261 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.261 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "The message of the day", 2024-04-25T14:50:48.261 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.261 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.262 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.262 INFO:teuthology.orchestra.run.smithi053.stdout: "redirect_resolve_ip_addr": { 2024-04-25T14:50:48.262 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T14:50:48.262 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.262 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.262 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.262 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.262 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.263 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.263 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.263 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.263 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.263 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.263 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.263 INFO:teuthology.orchestra.run.smithi053.stdout: "server_addr": { 2024-04-25T14:50:48.263 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "server_addr", 2024-04-25T14:50:48.264 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.264 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.264 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.264 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "::", 2024-04-25T14:50:48.264 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.264 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.264 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.264 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.265 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.265 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.265 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.265 INFO:teuthology.orchestra.run.smithi053.stdout: "server_port": { 2024-04-25T14:50:48.265 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "server_port", 2024-04-25T14:50:48.265 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:48.265 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.266 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.266 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "8080", 2024-04-25T14:50:48.266 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.266 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.266 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.266 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.266 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.266 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.266 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.267 INFO:teuthology.orchestra.run.smithi053.stdout: "ssl": { 2024-04-25T14:50:48.267 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ssl", 2024-04-25T14:50:48.267 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.267 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.267 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.267 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:48.267 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.267 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.267 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.268 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.268 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.268 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.268 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.268 INFO:teuthology.orchestra.run.smithi053.stdout: "ssl_server_port": { 2024-04-25T14:50:48.268 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ssl_server_port", 2024-04-25T14:50:48.268 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:48.268 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.268 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.269 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "8443", 2024-04-25T14:50:48.269 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.269 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.269 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.269 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.269 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.269 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.269 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.269 INFO:teuthology.orchestra.run.smithi053.stdout: "standby_behaviour": { 2024-04-25T14:50:48.270 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "standby_behaviour", 2024-04-25T14:50:48.270 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.270 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.270 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.270 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "redirect", 2024-04-25T14:50:48.270 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.270 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.270 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.270 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.271 INFO:teuthology.orchestra.run.smithi053.stdout: "redirect" 2024-04-25T14:50:48.271 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.271 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.271 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.271 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.271 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.271 INFO:teuthology.orchestra.run.smithi053.stdout: "standby_error_status_code": { 2024-04-25T14:50:48.271 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "standby_error_status_code", 2024-04-25T14:50:48.271 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:48.272 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.272 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.272 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "500", 2024-04-25T14:50:48.272 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "400", 2024-04-25T14:50:48.272 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "599", 2024-04-25T14:50:48.272 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.272 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.272 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.272 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.272 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.273 INFO:teuthology.orchestra.run.smithi053.stdout: "url_prefix": { 2024-04-25T14:50:48.273 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "url_prefix", 2024-04-25T14:50:48.273 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.273 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.273 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.273 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.273 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.273 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.273 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.274 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.274 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.274 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.274 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.274 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.274 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.274 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "devicehealth", 2024-04-25T14:50:48.275 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:48.275 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:48.275 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:48.275 INFO:teuthology.orchestra.run.smithi053.stdout: "enable_monitoring": { 2024-04-25T14:50:48.275 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "enable_monitoring", 2024-04-25T14:50:48.275 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.275 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.275 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.275 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:48.276 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.276 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.276 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.276 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "monitor device health metrics", 2024-04-25T14:50:48.276 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.276 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.276 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.276 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:48.277 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:48.277 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.277 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.277 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.277 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.277 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.277 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.277 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.277 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.278 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.278 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.278 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.278 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.278 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.278 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.278 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.278 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.278 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.279 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.279 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:48.279 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:48.279 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.279 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.279 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.279 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.279 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.280 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.280 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.280 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.280 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.280 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.280 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.280 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:48.280 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:48.280 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.281 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.281 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.281 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:48.281 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.281 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.281 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.281 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.281 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.281 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.281 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.282 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.282 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.282 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.282 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.282 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.282 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.282 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.282 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:48.283 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:48.283 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.283 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.283 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.283 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.283 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.283 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.283 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.283 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.284 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.284 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.284 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.284 INFO:teuthology.orchestra.run.smithi053.stdout: "mark_out_threshold": { 2024-04-25T14:50:48.284 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "mark_out_threshold", 2024-04-25T14:50:48.284 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:48.284 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.284 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.285 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "2419200", 2024-04-25T14:50:48.285 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.285 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.285 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.285 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T14:50:48.285 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.285 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.285 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.286 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": { 2024-04-25T14:50:48.286 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "pool_name", 2024-04-25T14:50:48.286 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.286 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.286 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.286 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "device_health_metrics", 2024-04-25T14:50:48.286 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.286 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.286 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.287 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T14:50:48.287 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.287 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.287 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.287 INFO:teuthology.orchestra.run.smithi053.stdout: "retention_period": { 2024-04-25T14:50:48.287 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "retention_period", 2024-04-25T14:50:48.287 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:48.287 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.287 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.288 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "15552000", 2024-04-25T14:50:48.288 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.288 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.288 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.288 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how long to retain device health metrics", 2024-04-25T14:50:48.288 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.288 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.289 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.289 INFO:teuthology.orchestra.run.smithi053.stdout: "scrape_frequency": { 2024-04-25T14:50:48.289 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "scrape_frequency", 2024-04-25T14:50:48.289 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:48.289 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.289 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.289 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "86400", 2024-04-25T14:50:48.289 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.290 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.290 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.290 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T14:50:48.290 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.290 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.290 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.290 INFO:teuthology.orchestra.run.smithi053.stdout: "self_heal": { 2024-04-25T14:50:48.290 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "self_heal", 2024-04-25T14:50:48.291 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.291 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.291 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.291 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:48.291 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.291 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.291 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.291 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T14:50:48.291 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.292 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.292 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.292 INFO:teuthology.orchestra.run.smithi053.stdout: "sleep_interval": { 2024-04-25T14:50:48.292 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "sleep_interval", 2024-04-25T14:50:48.292 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:48.292 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.292 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.292 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "600", 2024-04-25T14:50:48.292 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.293 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.293 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.293 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T14:50:48.293 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.293 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.293 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.293 INFO:teuthology.orchestra.run.smithi053.stdout: "warn_threshold": { 2024-04-25T14:50:48.293 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "warn_threshold", 2024-04-25T14:50:48.294 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:48.294 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.294 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.294 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "7257600", 2024-04-25T14:50:48.294 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.294 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.294 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.294 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T14:50:48.295 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.295 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.295 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.295 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.295 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.295 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "diskprediction_local", 2024-04-25T14:50:48.295 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:48.296 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:48.296 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:48.296 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:48.296 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:48.296 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.296 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.296 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.296 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.297 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.297 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.297 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.297 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.297 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.297 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.298 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.298 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.298 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.301 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.301 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.301 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.301 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.301 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.302 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:48.302 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:48.302 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.302 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.302 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.303 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.303 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.303 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.303 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.303 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.303 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.304 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.304 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.304 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:48.304 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:48.304 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.304 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.304 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.305 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:48.305 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.305 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.305 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.305 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.305 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.305 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.305 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.305 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.306 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.306 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.306 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.306 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.306 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.306 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.306 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:48.306 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:48.307 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.307 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.307 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.307 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.307 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.307 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.307 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.307 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.307 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.308 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.308 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.308 INFO:teuthology.orchestra.run.smithi053.stdout: "predict_interval": { 2024-04-25T14:50:48.308 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "predict_interval", 2024-04-25T14:50:48.308 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.308 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.308 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.308 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "86400", 2024-04-25T14:50:48.308 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.309 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.309 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.309 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.309 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.309 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.309 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.309 INFO:teuthology.orchestra.run.smithi053.stdout: "predictor_model": { 2024-04-25T14:50:48.309 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "predictor_model", 2024-04-25T14:50:48.310 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.310 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.310 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.310 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "prophetstor", 2024-04-25T14:50:48.310 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.310 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.310 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.310 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.310 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.311 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.311 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.311 INFO:teuthology.orchestra.run.smithi053.stdout: "sleep_interval": { 2024-04-25T14:50:48.311 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "sleep_interval", 2024-04-25T14:50:48.311 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.311 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.311 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.311 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "600", 2024-04-25T14:50:48.311 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.312 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.312 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.312 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.312 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.312 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.312 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.312 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.312 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.313 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "influx", 2024-04-25T14:50:48.313 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": false, 2024-04-25T14:50:48.313 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "influxdb python module not found", 2024-04-25T14:50:48.313 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:48.313 INFO:teuthology.orchestra.run.smithi053.stdout: "batch_size": { 2024-04-25T14:50:48.313 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "batch_size", 2024-04-25T14:50:48.313 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:48.314 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.314 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.314 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "5000", 2024-04-25T14:50:48.314 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.314 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.314 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.314 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T14:50:48.314 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.315 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.315 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.315 INFO:teuthology.orchestra.run.smithi053.stdout: "database": { 2024-04-25T14:50:48.315 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "database", 2024-04-25T14:50:48.315 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.315 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.315 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.316 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "ceph", 2024-04-25T14:50:48.316 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.316 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.316 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.316 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T14:50:48.316 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.316 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.316 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.317 INFO:teuthology.orchestra.run.smithi053.stdout: "hostname": { 2024-04-25T14:50:48.317 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "hostname", 2024-04-25T14:50:48.317 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.317 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.317 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.317 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.317 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.317 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.317 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.318 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "InfluxDB server hostname", 2024-04-25T14:50:48.318 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.318 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.318 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.318 INFO:teuthology.orchestra.run.smithi053.stdout: "interval": { 2024-04-25T14:50:48.318 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "interval", 2024-04-25T14:50:48.318 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:48.319 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.319 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.319 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "30", 2024-04-25T14:50:48.319 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "5", 2024-04-25T14:50:48.319 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.319 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.319 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T14:50:48.319 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.319 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.319 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.320 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:48.320 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:48.320 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.320 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.320 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.320 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.320 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.320 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.320 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.321 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.321 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.321 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.321 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.321 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.321 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.321 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.321 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.321 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.321 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.322 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.322 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:48.322 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:48.322 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.322 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.322 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.322 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.322 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.323 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.323 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.323 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.323 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.323 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.323 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.323 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:48.323 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:48.324 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.324 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.324 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.324 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:48.324 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.324 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.324 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.324 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.324 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.324 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.325 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.325 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.325 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.325 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.325 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.325 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.325 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.325 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.325 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:48.326 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:48.326 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.326 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.326 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.326 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.326 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.326 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.326 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.327 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.327 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.327 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.327 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.327 INFO:teuthology.orchestra.run.smithi053.stdout: "password": { 2024-04-25T14:50:48.327 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "password", 2024-04-25T14:50:48.327 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.327 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.327 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.328 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.328 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.328 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.328 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.328 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "password of InfluxDB server user", 2024-04-25T14:50:48.328 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.328 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.328 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.329 INFO:teuthology.orchestra.run.smithi053.stdout: "port": { 2024-04-25T14:50:48.329 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "port", 2024-04-25T14:50:48.329 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:48.329 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.329 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.329 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "8086", 2024-04-25T14:50:48.329 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.329 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.329 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.329 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "InfluxDB server port", 2024-04-25T14:50:48.330 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.330 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.330 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.330 INFO:teuthology.orchestra.run.smithi053.stdout: "ssl": { 2024-04-25T14:50:48.330 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ssl", 2024-04-25T14:50:48.330 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.330 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.330 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.330 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "false", 2024-04-25T14:50:48.331 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.331 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.331 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.331 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T14:50:48.331 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.331 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.331 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.331 INFO:teuthology.orchestra.run.smithi053.stdout: "threads": { 2024-04-25T14:50:48.332 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "threads", 2024-04-25T14:50:48.332 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:48.332 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.332 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.332 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "5", 2024-04-25T14:50:48.332 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "1", 2024-04-25T14:50:48.332 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "32", 2024-04-25T14:50:48.332 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.332 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T14:50:48.332 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.333 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.333 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.333 INFO:teuthology.orchestra.run.smithi053.stdout: "username": { 2024-04-25T14:50:48.333 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "username", 2024-04-25T14:50:48.333 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.333 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.333 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.333 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.333 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.334 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.334 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.334 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "username of InfluxDB server user", 2024-04-25T14:50:48.334 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.334 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.334 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.334 INFO:teuthology.orchestra.run.smithi053.stdout: "verify_ssl": { 2024-04-25T14:50:48.334 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "verify_ssl", 2024-04-25T14:50:48.335 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.335 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.335 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.335 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "true", 2024-04-25T14:50:48.335 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.335 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.335 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.335 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T14:50:48.335 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.335 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.336 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.336 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.336 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.336 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "insights", 2024-04-25T14:50:48.336 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:48.336 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:48.336 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:48.337 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:48.337 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:48.337 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.337 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.337 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.337 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.337 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.337 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.337 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.337 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.337 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.338 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.338 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.338 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.338 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.338 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.338 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.338 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.338 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.338 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.339 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:48.339 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:48.339 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.339 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.339 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.339 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.339 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.339 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.339 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.340 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.340 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.340 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.340 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.340 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:48.340 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:48.340 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.340 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.340 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.341 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:48.341 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.341 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.341 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.341 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.341 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.341 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.341 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.341 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.342 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.342 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.342 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.342 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.342 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.342 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.342 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:48.343 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:48.343 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.343 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.343 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.343 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.343 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.343 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.343 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.343 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.344 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.344 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.344 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.344 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.344 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.344 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "iostat", 2024-04-25T14:50:48.345 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:48.345 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:48.345 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:48.345 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:48.345 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:48.345 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.345 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.345 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.346 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.346 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.346 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.346 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.346 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.346 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.346 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.346 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.346 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.347 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.347 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.347 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.347 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.347 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.347 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.347 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:48.347 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:48.348 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.348 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.348 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.348 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.348 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.348 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.348 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.348 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.349 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.349 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.349 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.349 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:48.349 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:48.349 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.349 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.349 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.350 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:48.350 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.350 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.350 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.350 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.350 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.350 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.350 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.351 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.351 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.351 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.351 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.351 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.351 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.351 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.351 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:48.351 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:48.352 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.352 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.352 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.352 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.352 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.352 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.352 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.352 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.352 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.352 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.353 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.353 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.353 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.353 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "k8sevents", 2024-04-25T14:50:48.353 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:48.353 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:48.353 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:48.353 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_event_retention_days": { 2024-04-25T14:50:48.354 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "ceph_event_retention_days", 2024-04-25T14:50:48.354 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:48.354 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.354 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.354 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "7", 2024-04-25T14:50:48.354 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.354 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.354 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.355 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T14:50:48.355 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.355 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.355 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.355 INFO:teuthology.orchestra.run.smithi053.stdout: "config_check_secs": { 2024-04-25T14:50:48.355 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "config_check_secs", 2024-04-25T14:50:48.355 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:48.355 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.355 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.356 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2024-04-25T14:50:48.356 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "10", 2024-04-25T14:50:48.356 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.356 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.356 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T14:50:48.356 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.356 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.356 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.356 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:48.357 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:48.357 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.357 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.357 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.357 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.357 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.357 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.357 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.357 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.358 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.358 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.358 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.358 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.358 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.358 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.358 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.358 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.358 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.358 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.359 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:48.359 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:48.359 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.359 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.359 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.359 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.359 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.359 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.359 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.360 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.360 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.360 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.360 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.360 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:48.360 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:48.360 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.360 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.361 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.361 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:48.361 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.361 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.361 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.361 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.361 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.361 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.361 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.362 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.362 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.362 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.362 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.362 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.362 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.362 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.363 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:48.363 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:48.363 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.363 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.363 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.363 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.363 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.363 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.364 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.364 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.364 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.364 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.364 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.364 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.364 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.365 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "localpool", 2024-04-25T14:50:48.365 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:48.365 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:48.365 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:48.365 INFO:teuthology.orchestra.run.smithi053.stdout: "failure_domain": { 2024-04-25T14:50:48.365 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "failure_domain", 2024-04-25T14:50:48.365 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.365 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.365 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.366 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "host", 2024-04-25T14:50:48.366 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.366 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.366 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.366 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "failure domain for any created local pool", 2024-04-25T14:50:48.366 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T14:50:48.366 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.367 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.367 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:48.367 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:48.367 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.367 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.367 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.367 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.367 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.368 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.368 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.368 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.368 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.368 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.368 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.368 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.368 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.368 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.369 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.369 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.369 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.369 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.369 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:48.369 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:48.369 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.369 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.370 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.370 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.370 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.370 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.370 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.370 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.370 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.370 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.370 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.371 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:48.371 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:48.371 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.371 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.371 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.371 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:48.371 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.371 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.371 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.371 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.372 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.372 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.372 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.372 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.372 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.372 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.372 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.372 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.372 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.372 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.373 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:48.373 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:48.373 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.373 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.373 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.373 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.373 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.373 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.374 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.374 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.374 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.374 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.374 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.374 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": { 2024-04-25T14:50:48.374 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "min_size", 2024-04-25T14:50:48.374 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:48.374 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.375 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.375 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.375 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.375 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.375 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.375 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "default min_size for any created local pool", 2024-04-25T14:50:48.375 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T14:50:48.375 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.375 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.376 INFO:teuthology.orchestra.run.smithi053.stdout: "num_rep": { 2024-04-25T14:50:48.376 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "num_rep", 2024-04-25T14:50:48.376 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:48.376 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.376 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.376 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "3", 2024-04-25T14:50:48.376 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.376 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.377 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.377 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "default replica count for any created local pool", 2024-04-25T14:50:48.377 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.377 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.377 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.377 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": { 2024-04-25T14:50:48.377 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "pg_num", 2024-04-25T14:50:48.377 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:48.377 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.378 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.378 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "128", 2024-04-25T14:50:48.378 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.378 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.378 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.378 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T14:50:48.378 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.378 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.378 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.379 INFO:teuthology.orchestra.run.smithi053.stdout: "prefix": { 2024-04-25T14:50:48.379 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "prefix", 2024-04-25T14:50:48.379 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.379 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.379 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.379 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.379 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.379 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.379 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.380 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "name prefix for any created local pool", 2024-04-25T14:50:48.380 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.380 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.380 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.380 INFO:teuthology.orchestra.run.smithi053.stdout: "subtree": { 2024-04-25T14:50:48.380 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "subtree", 2024-04-25T14:50:48.380 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.380 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.380 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.381 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "rack", 2024-04-25T14:50:48.381 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.381 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.381 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.381 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T14:50:48.381 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T14:50:48.381 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.381 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.381 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.381 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.382 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "mds_autoscaler", 2024-04-25T14:50:48.382 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:48.382 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:48.382 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:48.382 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:48.382 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:48.382 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.383 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.383 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.383 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.383 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.383 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.383 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.383 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.383 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.383 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.384 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.384 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.384 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.384 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.384 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.384 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.384 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.384 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.384 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:48.385 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:48.385 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.385 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.385 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.385 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.385 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.385 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.385 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.385 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.386 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.386 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.386 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.386 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:48.386 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:48.386 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.386 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.386 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.387 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:48.387 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.387 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.387 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.387 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.387 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.387 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.387 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.388 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.388 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.388 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.388 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.388 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.388 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.388 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.388 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:48.389 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:48.389 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.389 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.389 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.389 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.389 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.389 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.389 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.389 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.390 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.390 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.390 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.390 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.390 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.390 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "mirroring", 2024-04-25T14:50:48.390 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:48.391 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:48.391 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:48.391 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:48.391 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:48.391 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.391 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.391 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.391 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.391 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.392 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.392 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.392 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.392 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.392 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.392 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.392 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.392 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.392 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.393 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.393 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.393 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.393 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.393 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:48.393 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:48.393 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.393 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.393 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.394 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.394 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.394 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.394 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.394 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.394 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.394 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.394 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.395 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:48.395 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:48.395 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.395 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.395 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.395 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:48.395 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.395 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.396 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.396 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.396 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.396 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.396 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.396 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.396 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.396 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.396 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.396 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.397 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.397 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.397 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:48.397 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:48.397 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.397 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.397 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.397 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.398 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.398 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.398 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.398 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.398 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.398 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.398 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.398 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.398 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.399 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "nfs", 2024-04-25T14:50:48.399 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:48.399 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:48.399 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:48.399 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:48.399 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:48.399 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.400 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.400 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.400 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.400 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.400 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.400 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.400 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.400 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.401 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.401 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.401 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.401 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.401 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.401 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.401 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.401 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.401 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.402 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:48.402 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:48.402 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.402 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.402 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.402 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.402 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.402 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.402 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.403 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.403 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.403 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.403 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.403 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:48.403 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:48.403 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.403 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.404 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.404 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:48.404 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.404 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.404 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.404 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.404 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.404 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.404 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.404 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.405 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.405 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.405 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.405 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.405 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.405 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.405 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:48.405 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:48.406 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.406 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.406 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.406 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.406 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.406 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.406 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.406 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.406 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.407 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.407 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.407 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.407 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.407 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "orchestrator", 2024-04-25T14:50:48.407 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:48.407 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:48.408 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:48.408 INFO:teuthology.orchestra.run.smithi053.stdout: "fail_fs": { 2024-04-25T14:50:48.408 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "fail_fs", 2024-04-25T14:50:48.408 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.408 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.408 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.408 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.408 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.409 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.409 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.409 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T14:50:48.409 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.409 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.409 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.409 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:48.410 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:48.410 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.410 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.410 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.410 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.410 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.410 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.410 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.411 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.411 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.411 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.411 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.411 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.411 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.411 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.411 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.411 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.412 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.412 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.412 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:48.412 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:48.412 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.412 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.412 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.412 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.412 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.413 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.413 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.413 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.413 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.413 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.413 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.413 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:48.413 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:48.414 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.414 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.414 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.414 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:48.414 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.414 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.414 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.414 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.415 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.415 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.415 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.415 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.415 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.415 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.415 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.415 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.415 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.416 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.416 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:48.416 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:48.416 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.416 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.416 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.416 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.416 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.416 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.417 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.417 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.417 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.417 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.417 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.417 INFO:teuthology.orchestra.run.smithi053.stdout: "orchestrator": { 2024-04-25T14:50:48.417 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "orchestrator", 2024-04-25T14:50:48.417 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.418 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.418 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.418 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.418 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.418 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.418 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.418 INFO:teuthology.orchestra.run.smithi053.stdout: "cephadm", 2024-04-25T14:50:48.418 INFO:teuthology.orchestra.run.smithi053.stdout: "rook", 2024-04-25T14:50:48.419 INFO:teuthology.orchestra.run.smithi053.stdout: "test_orchestrator" 2024-04-25T14:50:48.419 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.419 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Orchestrator backend", 2024-04-25T14:50:48.419 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.419 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.419 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.419 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.419 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.420 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "osd_perf_query", 2024-04-25T14:50:48.420 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:48.420 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:48.420 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:48.420 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:48.420 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:48.420 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.420 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.420 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.421 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.421 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.421 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.421 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.421 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.421 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.421 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.421 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.422 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.422 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.422 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.422 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.422 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.422 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.422 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.423 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:48.423 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:48.423 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.423 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.423 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.423 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.423 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.423 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.423 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.424 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.424 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.424 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.424 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.424 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:48.424 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:48.424 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.425 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.425 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.425 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:48.425 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.425 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.425 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.425 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.425 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.425 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.426 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.426 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.426 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.426 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.426 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.426 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.426 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.426 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.427 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:48.427 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:48.427 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.427 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.427 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.427 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.427 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.428 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.428 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.428 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.428 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.428 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.428 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.428 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.428 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.429 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "osd_support", 2024-04-25T14:50:48.429 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:48.429 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:48.429 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:48.429 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:48.429 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:48.429 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.429 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.430 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.430 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.430 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.430 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.430 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.430 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.430 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.430 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.431 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.431 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.431 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.431 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.431 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.431 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.431 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.431 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.432 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:48.432 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:48.432 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.432 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.432 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.432 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.432 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.432 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.432 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.433 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.433 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.433 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.433 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.433 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:48.433 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:48.433 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.433 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.434 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.434 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:48.434 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.434 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.434 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.434 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.434 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.434 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.435 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.435 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.435 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.435 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.435 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.435 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.435 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.435 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.436 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:48.436 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:48.436 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.436 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.436 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.436 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.436 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.436 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.437 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.437 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.437 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.437 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.437 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.437 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.437 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.437 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "pg_autoscaler", 2024-04-25T14:50:48.438 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:48.438 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:48.438 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:48.438 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:48.438 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:48.438 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.438 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.438 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.438 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.439 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.439 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.439 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.439 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.439 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.439 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.439 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.439 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.439 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.439 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.440 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.440 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.440 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.440 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.440 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:48.440 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:48.440 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.440 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.441 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.441 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.441 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.441 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.441 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.441 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.441 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.441 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.441 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.442 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:48.442 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:48.442 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.442 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.442 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.442 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:48.442 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.442 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.442 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.443 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.443 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.443 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.443 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.443 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.443 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.443 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.443 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.443 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.443 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.444 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.444 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:48.444 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:48.444 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.444 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.444 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.444 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.444 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.445 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.445 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.445 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.445 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.445 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.445 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.445 INFO:teuthology.orchestra.run.smithi053.stdout: "sleep_interval": { 2024-04-25T14:50:48.445 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "sleep_interval", 2024-04-25T14:50:48.445 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:48.446 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.446 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.446 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "60", 2024-04-25T14:50:48.446 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.446 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.446 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.446 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.446 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.446 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.447 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.447 INFO:teuthology.orchestra.run.smithi053.stdout: "threshold": { 2024-04-25T14:50:48.447 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "threshold", 2024-04-25T14:50:48.447 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2024-04-25T14:50:48.447 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.447 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.447 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "3.0", 2024-04-25T14:50:48.447 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "1.0", 2024-04-25T14:50:48.447 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.448 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.448 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "scaling threshold", 2024-04-25T14:50:48.448 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T14:50:48.448 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.448 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.448 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.448 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.448 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "progress", 2024-04-25T14:50:48.449 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:48.449 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:48.449 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:48.449 INFO:teuthology.orchestra.run.smithi053.stdout: "allow_pg_recovery_event": { 2024-04-25T14:50:48.449 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "allow_pg_recovery_event", 2024-04-25T14:50:48.449 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.449 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.449 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.449 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.450 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.450 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.450 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.450 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T14:50:48.450 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.450 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.450 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.450 INFO:teuthology.orchestra.run.smithi053.stdout: "enabled": { 2024-04-25T14:50:48.451 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "enabled", 2024-04-25T14:50:48.451 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.451 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.451 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.451 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:48.451 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.451 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.451 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.451 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.451 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.452 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.452 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.452 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:48.452 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:48.452 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.452 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.452 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.452 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.453 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.453 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.453 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.453 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.453 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.453 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.453 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.453 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.453 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.453 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.454 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.454 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.454 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.454 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.454 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:48.454 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:48.454 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.454 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.455 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.455 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.455 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.455 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.455 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.455 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.455 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.455 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.456 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.456 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:48.456 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:48.456 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.456 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.456 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.456 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:48.457 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.457 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.457 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.457 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.457 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.457 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.457 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.457 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.457 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.458 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.458 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.458 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.458 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.458 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.458 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:48.458 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:48.458 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.459 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.459 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.459 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.459 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.459 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.459 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.459 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.459 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.460 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.460 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.460 INFO:teuthology.orchestra.run.smithi053.stdout: "max_completed_events": { 2024-04-25T14:50:48.460 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "max_completed_events", 2024-04-25T14:50:48.460 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:48.460 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.460 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.461 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "50", 2024-04-25T14:50:48.461 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.461 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.461 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.461 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "number of past completed events to remember", 2024-04-25T14:50:48.461 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.461 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.461 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.462 INFO:teuthology.orchestra.run.smithi053.stdout: "sleep_interval": { 2024-04-25T14:50:48.462 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "sleep_interval", 2024-04-25T14:50:48.462 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:48.462 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.462 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.462 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "5", 2024-04-25T14:50:48.462 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.463 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.463 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.463 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "how long the module is going to sleep", 2024-04-25T14:50:48.463 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.463 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.463 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.463 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.463 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.464 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "prometheus", 2024-04-25T14:50:48.464 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:48.464 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:48.464 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:48.464 INFO:teuthology.orchestra.run.smithi053.stdout: "cache": { 2024-04-25T14:50:48.464 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "cache", 2024-04-25T14:50:48.464 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.464 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.465 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.465 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:48.465 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.465 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.465 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.465 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.465 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.465 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.466 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.466 INFO:teuthology.orchestra.run.smithi053.stdout: "exclude_perf_counters": { 2024-04-25T14:50:48.466 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "exclude_perf_counters", 2024-04-25T14:50:48.466 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.466 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.466 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.466 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:48.466 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.467 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.467 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.467 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T14:50:48.467 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T14:50:48.467 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.467 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.467 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.467 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:48.467 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:48.467 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.468 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.468 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.468 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.468 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.468 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.468 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.468 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.468 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.468 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.469 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.469 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.469 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.469 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.469 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.469 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.469 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.469 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.469 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.469 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:48.470 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:48.470 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.470 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.470 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.470 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.470 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.470 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.470 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.470 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.471 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.471 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.471 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.471 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:48.471 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:48.471 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.471 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.471 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.471 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:48.472 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.472 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.472 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.472 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.472 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.472 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.472 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.472 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.472 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.472 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.473 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.473 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.473 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.473 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.473 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:48.473 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:48.473 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.473 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.474 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.474 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.474 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.474 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.474 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.474 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.474 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.474 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.474 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.475 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_stats_pools": { 2024-04-25T14:50:48.475 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rbd_stats_pools", 2024-04-25T14:50:48.475 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.475 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.475 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.475 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.475 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.475 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.475 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.476 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.476 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.476 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.476 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.476 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.476 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T14:50:48.476 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T14:50:48.476 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:48.476 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.477 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.477 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "300", 2024-04-25T14:50:48.477 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.477 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.477 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.477 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.477 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.477 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.477 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.477 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.478 INFO:teuthology.orchestra.run.smithi053.stdout: "scrape_interval": { 2024-04-25T14:50:48.478 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "scrape_interval", 2024-04-25T14:50:48.478 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2024-04-25T14:50:48.478 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.478 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.478 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "15.0", 2024-04-25T14:50:48.478 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.478 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.478 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.479 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.479 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.479 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.479 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.479 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.479 INFO:teuthology.orchestra.run.smithi053.stdout: "server_addr": { 2024-04-25T14:50:48.479 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "server_addr", 2024-04-25T14:50:48.479 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.479 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.479 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.480 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "::", 2024-04-25T14:50:48.480 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.480 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.480 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.480 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T14:50:48.480 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.480 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.480 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.480 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.480 INFO:teuthology.orchestra.run.smithi053.stdout: "server_port": { 2024-04-25T14:50:48.481 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "server_port", 2024-04-25T14:50:48.481 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:48.481 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.481 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.481 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "9283", 2024-04-25T14:50:48.481 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.481 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.481 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.481 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T14:50:48.481 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.482 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.482 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.482 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.482 INFO:teuthology.orchestra.run.smithi053.stdout: "stale_cache_strategy": { 2024-04-25T14:50:48.482 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "stale_cache_strategy", 2024-04-25T14:50:48.482 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.482 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.482 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.482 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "log", 2024-04-25T14:50:48.483 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.483 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.483 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.483 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.483 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.483 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.483 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.483 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.483 INFO:teuthology.orchestra.run.smithi053.stdout: "standby_behaviour": { 2024-04-25T14:50:48.483 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "standby_behaviour", 2024-04-25T14:50:48.484 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.484 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.484 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.484 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "default", 2024-04-25T14:50:48.484 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.484 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.484 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.484 INFO:teuthology.orchestra.run.smithi053.stdout: "default", 2024-04-25T14:50:48.484 INFO:teuthology.orchestra.run.smithi053.stdout: "error" 2024-04-25T14:50:48.485 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.485 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.485 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.485 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.485 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.485 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.485 INFO:teuthology.orchestra.run.smithi053.stdout: "standby_error_status_code": { 2024-04-25T14:50:48.485 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "standby_error_status_code", 2024-04-25T14:50:48.485 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:48.486 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.486 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.486 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "500", 2024-04-25T14:50:48.486 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "400", 2024-04-25T14:50:48.486 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "599", 2024-04-25T14:50:48.486 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.486 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.486 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.486 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.487 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.487 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.487 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.487 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.487 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.487 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rbd_support", 2024-04-25T14:50:48.487 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:48.487 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:48.487 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:48.487 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:48.488 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:48.488 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.488 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.488 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.488 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.488 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.488 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.488 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.488 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.488 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.489 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.489 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.489 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.489 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.489 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.489 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.489 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.489 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.489 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.490 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:48.490 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:48.490 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.490 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.490 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.490 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.490 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.490 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.490 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.491 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.491 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.491 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.491 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.491 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:48.491 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:48.491 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.491 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.492 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.492 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:48.492 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.492 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.492 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.492 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.492 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.492 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.492 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.493 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.493 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.493 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.493 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.493 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.493 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.493 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.493 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:48.494 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:48.494 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.494 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.494 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.494 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.494 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.494 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.494 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.494 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.495 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.495 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.495 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.495 INFO:teuthology.orchestra.run.smithi053.stdout: "max_concurrent_snap_create": { 2024-04-25T14:50:48.495 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "max_concurrent_snap_create", 2024-04-25T14:50:48.495 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:48.495 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.496 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.496 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10", 2024-04-25T14:50:48.496 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.496 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.496 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.496 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.496 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.496 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.496 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.496 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.497 INFO:teuthology.orchestra.run.smithi053.stdout: "mirror_snapshot_schedule": { 2024-04-25T14:50:48.497 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T14:50:48.497 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.497 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.497 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.497 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.497 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.497 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.497 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.498 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.498 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.498 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.498 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.498 INFO:teuthology.orchestra.run.smithi053.stdout: "trash_purge_schedule": { 2024-04-25T14:50:48.498 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "trash_purge_schedule", 2024-04-25T14:50:48.498 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.498 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.499 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.499 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.499 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.499 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.499 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.499 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.499 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.499 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.499 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.499 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.500 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.500 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.500 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.500 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "restful", 2024-04-25T14:50:48.500 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:48.500 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:48.500 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:48.500 INFO:teuthology.orchestra.run.smithi053.stdout: "enable_auth": { 2024-04-25T14:50:48.500 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "enable_auth", 2024-04-25T14:50:48.501 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.501 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.501 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.501 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:48.501 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.501 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.501 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.501 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.501 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.502 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.502 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.502 INFO:teuthology.orchestra.run.smithi053.stdout: "key_file": { 2024-04-25T14:50:48.502 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "key_file", 2024-04-25T14:50:48.502 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.502 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.502 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.502 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.503 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.503 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.503 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.503 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.503 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.503 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.503 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.503 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:48.503 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:48.504 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.504 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.504 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.504 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.504 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.504 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.504 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.504 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.504 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.505 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.505 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.505 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.505 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.505 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.505 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.505 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.505 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.506 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.506 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:48.506 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:48.506 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.506 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.506 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.506 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.507 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.507 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.507 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.507 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.507 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.507 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.507 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.507 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:48.507 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:48.508 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.508 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.508 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.508 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:48.508 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.508 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.508 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.508 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.508 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.509 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.509 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.509 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.509 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.509 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.509 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.509 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.509 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.509 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.510 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:48.510 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:48.510 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.510 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.510 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.510 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.510 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.510 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.511 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.511 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.511 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.511 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.511 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.511 INFO:teuthology.orchestra.run.smithi053.stdout: "server_addr": { 2024-04-25T14:50:48.511 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "server_addr", 2024-04-25T14:50:48.511 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.511 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.512 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.512 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.512 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.512 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.512 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.512 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.512 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.512 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.513 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.513 INFO:teuthology.orchestra.run.smithi053.stdout: "server_port": { 2024-04-25T14:50:48.513 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "server_port", 2024-04-25T14:50:48.513 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.513 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.513 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.513 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.514 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.514 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.514 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.514 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.514 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.514 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.514 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.514 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.514 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.515 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rgw", 2024-04-25T14:50:48.515 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:48.515 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:48.515 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:48.515 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:48.515 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:48.515 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.515 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.516 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.516 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.516 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.516 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.516 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.516 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.516 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.516 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.516 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.517 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.517 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.517 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.517 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.517 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.517 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.517 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.517 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:48.518 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:48.518 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.518 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.518 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.518 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.518 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.518 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.518 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.518 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.519 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.519 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.519 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.519 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:48.519 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:48.519 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.519 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.519 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.520 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:48.520 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.520 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.520 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.520 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.520 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.520 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.520 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.520 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.521 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.521 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.521 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.521 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.521 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.521 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.521 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:48.522 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:48.522 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.522 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.522 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.522 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.522 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.522 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.522 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.523 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.523 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.523 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.523 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.523 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.523 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.523 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rook", 2024-04-25T14:50:48.523 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:48.524 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:48.524 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:48.524 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:48.524 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:48.524 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.524 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.524 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.524 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.524 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.525 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.525 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.525 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.525 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.525 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.525 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.525 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.525 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.526 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.526 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.526 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.526 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.526 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.526 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:48.526 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:48.526 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.527 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.527 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.527 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.527 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.527 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.527 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.527 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.527 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.527 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.527 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.528 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:48.528 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:48.528 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.528 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.528 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.528 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:48.528 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.528 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.528 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.529 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.529 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.529 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.529 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.529 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.529 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.529 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.529 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.530 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.530 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.530 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.530 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:48.530 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:48.530 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.530 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.530 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.531 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.531 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.531 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.531 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.531 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.531 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.531 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.531 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.531 INFO:teuthology.orchestra.run.smithi053.stdout: "storage_class": { 2024-04-25T14:50:48.532 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "storage_class", 2024-04-25T14:50:48.532 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.532 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.532 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.532 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "local", 2024-04-25T14:50:48.532 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.532 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.532 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.532 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T14:50:48.532 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.533 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.533 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.533 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.533 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.533 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "selftest", 2024-04-25T14:50:48.533 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:48.533 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:48.533 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:48.534 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:48.534 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:48.534 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.534 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.534 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.534 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.534 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.534 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.535 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.535 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.535 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.535 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.535 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.535 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.535 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.535 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.535 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.536 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.536 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.536 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.536 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:48.536 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:48.536 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.536 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.536 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.537 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.537 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.537 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.537 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.537 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.537 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.537 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.537 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.537 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:48.538 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:48.538 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.538 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.538 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.538 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:48.538 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.538 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.538 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.538 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.539 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.539 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.539 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.539 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.539 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.539 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.539 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.539 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.539 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.540 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.540 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:48.540 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:48.540 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.540 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.540 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.540 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.540 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.541 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.541 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.541 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.541 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.541 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.541 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.541 INFO:teuthology.orchestra.run.smithi053.stdout: "roption1": { 2024-04-25T14:50:48.541 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "roption1", 2024-04-25T14:50:48.541 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.542 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.542 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.542 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.542 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.542 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.542 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.542 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.542 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.542 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.543 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.543 INFO:teuthology.orchestra.run.smithi053.stdout: "roption2": { 2024-04-25T14:50:48.543 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "roption2", 2024-04-25T14:50:48.543 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.543 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.543 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.543 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "xyz", 2024-04-25T14:50:48.543 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.544 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.544 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.544 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.544 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.544 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.544 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.544 INFO:teuthology.orchestra.run.smithi053.stdout: "rwoption1": { 2024-04-25T14:50:48.544 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rwoption1", 2024-04-25T14:50:48.544 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.545 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.545 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.545 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.545 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.545 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.545 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.545 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.545 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.545 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.546 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.546 INFO:teuthology.orchestra.run.smithi053.stdout: "rwoption2": { 2024-04-25T14:50:48.546 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rwoption2", 2024-04-25T14:50:48.546 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:48.546 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.546 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.546 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.546 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.547 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.547 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.547 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.547 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.547 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.547 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.547 INFO:teuthology.orchestra.run.smithi053.stdout: "rwoption3": { 2024-04-25T14:50:48.547 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rwoption3", 2024-04-25T14:50:48.547 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "float", 2024-04-25T14:50:48.548 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.548 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.548 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.548 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.548 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.548 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.548 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.548 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.548 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.549 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.549 INFO:teuthology.orchestra.run.smithi053.stdout: "rwoption4": { 2024-04-25T14:50:48.549 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rwoption4", 2024-04-25T14:50:48.549 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.549 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.549 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.549 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.549 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.550 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.550 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.550 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.550 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.550 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.550 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.550 INFO:teuthology.orchestra.run.smithi053.stdout: "rwoption5": { 2024-04-25T14:50:48.550 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rwoption5", 2024-04-25T14:50:48.551 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.551 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.551 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.551 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.551 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.551 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.551 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.551 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.551 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.551 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.552 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.552 INFO:teuthology.orchestra.run.smithi053.stdout: "rwoption6": { 2024-04-25T14:50:48.552 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rwoption6", 2024-04-25T14:50:48.552 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.552 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.552 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.552 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:48.552 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.553 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.553 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.553 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.553 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.553 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.553 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.553 INFO:teuthology.orchestra.run.smithi053.stdout: "rwoption7": { 2024-04-25T14:50:48.553 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rwoption7", 2024-04-25T14:50:48.553 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:48.554 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.554 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.554 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.554 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "1", 2024-04-25T14:50:48.554 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "42", 2024-04-25T14:50:48.554 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.554 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.554 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.554 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.555 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.555 INFO:teuthology.orchestra.run.smithi053.stdout: "testkey": { 2024-04-25T14:50:48.555 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "testkey", 2024-04-25T14:50:48.555 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.555 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.555 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.555 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.555 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.556 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.556 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.556 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.556 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.556 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.556 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.556 INFO:teuthology.orchestra.run.smithi053.stdout: "testlkey": { 2024-04-25T14:50:48.556 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "testlkey", 2024-04-25T14:50:48.556 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.557 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.557 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.557 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.557 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.557 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.557 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.557 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.557 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.558 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.558 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.558 INFO:teuthology.orchestra.run.smithi053.stdout: "testnewline": { 2024-04-25T14:50:48.558 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "testnewline", 2024-04-25T14:50:48.558 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.558 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.558 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.558 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.559 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.559 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.559 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.559 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.559 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.559 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.559 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.559 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.559 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.560 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "snap_schedule", 2024-04-25T14:50:48.560 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:48.560 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:48.560 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:48.560 INFO:teuthology.orchestra.run.smithi053.stdout: "allow_m_granularity": { 2024-04-25T14:50:48.560 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "allow_m_granularity", 2024-04-25T14:50:48.560 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.560 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.560 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.561 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.561 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.561 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.561 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.561 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T14:50:48.561 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.561 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.561 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.561 INFO:teuthology.orchestra.run.smithi053.stdout: "dump_on_update": { 2024-04-25T14:50:48.562 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "dump_on_update", 2024-04-25T14:50:48.562 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.562 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.562 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.562 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.562 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.562 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.562 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.562 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "dump database to debug log on update", 2024-04-25T14:50:48.563 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.563 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.563 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.563 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:48.563 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:48.563 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.563 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.564 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.564 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.564 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.564 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.564 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.564 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.564 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.564 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.564 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.565 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.565 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.565 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.565 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.565 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.565 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.565 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.565 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:48.566 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:48.566 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.566 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.566 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.566 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.566 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.566 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.566 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.567 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.567 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.567 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.567 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.567 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:48.567 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:48.567 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.568 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.568 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.568 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:48.568 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.568 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.568 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.568 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.568 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.568 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.569 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.569 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.569 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.569 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.569 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.569 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.569 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.569 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.570 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:48.570 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:48.570 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.570 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.570 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.570 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.570 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.570 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.571 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.571 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.571 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.571 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.571 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.571 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.571 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.572 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "stats", 2024-04-25T14:50:48.572 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:48.572 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:48.572 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:48.572 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:48.572 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:48.572 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.572 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.572 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.573 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.573 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.573 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.573 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.573 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.573 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.573 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.573 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.573 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.574 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.574 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.574 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.574 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.574 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.574 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.574 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:48.575 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:48.575 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.575 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.575 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.575 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.575 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.575 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.575 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.575 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.576 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.576 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.576 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.576 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:48.576 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:48.576 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.576 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.576 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.577 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:48.577 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.577 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.577 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.577 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.577 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.577 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.577 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.577 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.577 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.578 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.578 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.578 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.578 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.578 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.578 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:48.578 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:48.579 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.579 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.579 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.579 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.579 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.579 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.579 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.579 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.579 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.579 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.580 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.580 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.580 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.580 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "status", 2024-04-25T14:50:48.580 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:48.580 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:48.580 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:48.581 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:48.581 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:48.581 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.581 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.581 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.581 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.581 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.581 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.581 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.581 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.582 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.582 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.582 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.582 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.582 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.582 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.582 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.582 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.582 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.583 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.583 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:48.583 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:48.583 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.583 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.583 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.583 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.583 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.583 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.584 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.584 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.584 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.584 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.584 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.584 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:48.584 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:48.584 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.584 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.585 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.585 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:48.585 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.585 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.585 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.585 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.585 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.585 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.585 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.586 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.586 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.586 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.586 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.586 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.586 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.586 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.586 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:48.587 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:48.587 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.587 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.587 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.587 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.587 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.587 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.587 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.587 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.587 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.588 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.588 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.588 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.588 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.588 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "telegraf", 2024-04-25T14:50:48.588 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:48.588 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:48.588 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:48.589 INFO:teuthology.orchestra.run.smithi053.stdout: "address": { 2024-04-25T14:50:48.589 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "address", 2024-04-25T14:50:48.589 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.589 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.589 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.589 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T14:50:48.589 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.589 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.589 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.589 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.590 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.590 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.590 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.590 INFO:teuthology.orchestra.run.smithi053.stdout: "interval": { 2024-04-25T14:50:48.590 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "interval", 2024-04-25T14:50:48.590 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:48.590 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.590 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.591 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "15", 2024-04-25T14:50:48.591 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.591 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.591 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.591 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.591 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.591 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.591 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.592 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:48.592 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:48.592 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.592 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.592 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.592 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.592 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.592 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.592 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.592 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.593 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.593 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.593 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.593 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.593 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.593 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.593 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.593 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.593 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.593 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.594 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:48.594 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:48.594 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.594 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.594 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.594 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.594 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.595 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.595 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.595 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.595 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.595 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.595 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.595 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:48.595 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:48.596 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.596 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.596 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.596 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:48.596 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.596 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.596 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.596 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.596 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.597 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.597 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.597 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.597 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.597 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.597 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.597 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.597 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.598 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.598 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:48.598 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:48.598 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.598 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.598 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.598 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.599 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.599 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.599 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.599 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.599 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.599 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.599 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.599 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.599 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.600 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "telemetry", 2024-04-25T14:50:48.600 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:48.600 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:48.600 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:48.600 INFO:teuthology.orchestra.run.smithi053.stdout: "channel_basic": { 2024-04-25T14:50:48.600 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "channel_basic", 2024-04-25T14:50:48.600 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.600 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.601 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.601 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:48.601 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.601 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.601 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.601 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T14:50:48.601 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.601 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.601 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.602 INFO:teuthology.orchestra.run.smithi053.stdout: "channel_crash": { 2024-04-25T14:50:48.602 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "channel_crash", 2024-04-25T14:50:48.602 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.602 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.602 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.602 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:48.602 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.603 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.603 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.603 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T14:50:48.603 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.603 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.603 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.603 INFO:teuthology.orchestra.run.smithi053.stdout: "channel_device": { 2024-04-25T14:50:48.603 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "channel_device", 2024-04-25T14:50:48.604 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.604 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.604 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.604 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:48.604 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.604 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.604 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.604 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T14:50:48.605 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.605 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.605 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.605 INFO:teuthology.orchestra.run.smithi053.stdout: "channel_ident": { 2024-04-25T14:50:48.605 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "channel_ident", 2024-04-25T14:50:48.605 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.605 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.606 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.606 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.606 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.606 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.606 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.606 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T14:50:48.606 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.606 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.607 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.607 INFO:teuthology.orchestra.run.smithi053.stdout: "channel_perf": { 2024-04-25T14:50:48.607 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "channel_perf", 2024-04-25T14:50:48.607 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.607 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.607 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.607 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.607 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.608 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.608 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.608 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T14:50:48.608 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.608 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.608 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.608 INFO:teuthology.orchestra.run.smithi053.stdout: "contact": { 2024-04-25T14:50:48.608 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "contact", 2024-04-25T14:50:48.609 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.609 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.609 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.609 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.609 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.609 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.609 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.609 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.609 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.610 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.610 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.610 INFO:teuthology.orchestra.run.smithi053.stdout: "description": { 2024-04-25T14:50:48.610 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "description", 2024-04-25T14:50:48.610 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.610 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.610 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.611 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.611 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.611 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.611 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.611 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.611 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.611 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.611 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.612 INFO:teuthology.orchestra.run.smithi053.stdout: "device_url": { 2024-04-25T14:50:48.612 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "device_url", 2024-04-25T14:50:48.612 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.612 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.612 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.612 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T14:50:48.612 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.612 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.613 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.613 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.613 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.613 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.613 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.613 INFO:teuthology.orchestra.run.smithi053.stdout: "enabled": { 2024-04-25T14:50:48.613 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "enabled", 2024-04-25T14:50:48.614 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.614 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.614 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.614 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.614 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.614 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.614 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.614 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.615 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.615 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.615 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.615 INFO:teuthology.orchestra.run.smithi053.stdout: "interval": { 2024-04-25T14:50:48.615 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "interval", 2024-04-25T14:50:48.615 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:48.615 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.615 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.616 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "24", 2024-04-25T14:50:48.616 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "8", 2024-04-25T14:50:48.616 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.616 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.616 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.616 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.616 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.616 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.617 INFO:teuthology.orchestra.run.smithi053.stdout: "last_opt_revision": { 2024-04-25T14:50:48.617 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "last_opt_revision", 2024-04-25T14:50:48.617 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:48.617 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.617 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.617 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "1", 2024-04-25T14:50:48.617 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.617 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.618 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.618 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.618 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.618 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.618 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.618 INFO:teuthology.orchestra.run.smithi053.stdout: "leaderboard": { 2024-04-25T14:50:48.618 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "leaderboard", 2024-04-25T14:50:48.618 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.619 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.619 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.619 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.619 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.619 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.619 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.619 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.619 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.619 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.620 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.620 INFO:teuthology.orchestra.run.smithi053.stdout: "leaderboard_description": { 2024-04-25T14:50:48.620 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "leaderboard_description", 2024-04-25T14:50:48.620 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.620 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.620 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.620 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.620 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.621 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.621 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.621 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.621 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.621 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.621 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.621 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:48.621 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:48.622 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.622 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.622 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.622 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.622 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.622 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.622 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.622 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.623 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.623 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.623 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.623 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.623 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.623 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.623 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.623 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.623 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.624 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.624 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:48.624 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:48.624 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.624 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.624 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.624 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.624 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.625 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.625 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.625 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.625 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.625 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.625 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.625 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:48.625 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:48.626 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.626 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.626 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.626 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:48.626 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.626 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.626 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.626 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.627 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.627 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.627 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.627 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.627 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.627 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.627 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.627 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.627 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.628 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.628 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:48.628 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:48.628 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.628 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.628 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.628 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.628 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.629 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.629 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.629 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.629 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.629 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.629 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.629 INFO:teuthology.orchestra.run.smithi053.stdout: "organization": { 2024-04-25T14:50:48.629 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "organization", 2024-04-25T14:50:48.630 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.630 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.630 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.630 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.630 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.630 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.630 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.630 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.630 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.631 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.631 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.631 INFO:teuthology.orchestra.run.smithi053.stdout: "proxy": { 2024-04-25T14:50:48.631 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "proxy", 2024-04-25T14:50:48.631 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.631 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.631 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.631 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.631 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.632 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.632 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.632 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.632 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.632 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.632 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.632 INFO:teuthology.orchestra.run.smithi053.stdout: "url": { 2024-04-25T14:50:48.632 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "url", 2024-04-25T14:50:48.633 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.633 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.633 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.633 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T14:50:48.633 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.633 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.633 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.633 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.633 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.634 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.634 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.634 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.634 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.634 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "test_orchestrator", 2024-04-25T14:50:48.634 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:48.634 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:48.635 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:48.635 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:48.635 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:48.635 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.635 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.635 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.635 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.635 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.635 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.635 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.636 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.636 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.636 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.636 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.636 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.636 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.636 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.636 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.636 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.637 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.637 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.637 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:48.637 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:48.637 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.637 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.637 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.637 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.638 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.638 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.638 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.638 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.638 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.638 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.638 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.638 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:48.639 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:48.639 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.639 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.639 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.639 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:48.639 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.639 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.639 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.639 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.639 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.640 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.640 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.640 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.640 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.640 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.640 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.640 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.640 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.640 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.641 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:48.641 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:48.641 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.641 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.641 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.641 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.641 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.641 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.641 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.641 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.642 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.642 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.642 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.642 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.642 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.642 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "volumes", 2024-04-25T14:50:48.642 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:48.643 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:48.643 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:48.643 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:48.643 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:48.643 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.643 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.643 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.643 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.643 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.643 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.644 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.644 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.644 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.644 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.644 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.644 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.644 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.644 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.645 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.645 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.645 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.645 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.645 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:48.645 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:48.645 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.645 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.646 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.646 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.646 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.646 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.646 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.646 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.646 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.646 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.646 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.647 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:48.647 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:48.647 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.647 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.647 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.647 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:48.647 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.647 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.647 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.648 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.648 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.648 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.648 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.648 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.648 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.648 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.648 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.648 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.648 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.649 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.649 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:48.649 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:48.649 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.649 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.649 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.649 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.649 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.650 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.650 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.650 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.650 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.650 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.650 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.650 INFO:teuthology.orchestra.run.smithi053.stdout: "max_concurrent_clones": { 2024-04-25T14:50:48.650 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "max_concurrent_clones", 2024-04-25T14:50:48.651 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:48.651 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.651 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.651 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "4", 2024-04-25T14:50:48.651 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.651 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.651 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.651 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T14:50:48.651 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.651 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.652 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.652 INFO:teuthology.orchestra.run.smithi053.stdout: "periodic_async_work": { 2024-04-25T14:50:48.652 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "periodic_async_work", 2024-04-25T14:50:48.652 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.652 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.652 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.652 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.652 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.652 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.653 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.653 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Periodically check for async work", 2024-04-25T14:50:48.653 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.653 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.653 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.653 INFO:teuthology.orchestra.run.smithi053.stdout: "snapshot_clone_delay": { 2024-04-25T14:50:48.653 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "snapshot_clone_delay", 2024-04-25T14:50:48.653 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:48.654 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.654 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.654 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "0", 2024-04-25T14:50:48.654 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.654 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.654 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.654 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T14:50:48.654 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.655 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.655 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.655 INFO:teuthology.orchestra.run.smithi053.stdout: "snapshot_clone_no_wait": { 2024-04-25T14:50:48.655 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T14:50:48.655 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.655 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.655 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.655 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "True", 2024-04-25T14:50:48.656 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.656 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.656 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.656 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T14:50:48.656 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.656 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.656 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.656 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.657 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.657 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "zabbix", 2024-04-25T14:50:48.657 INFO:teuthology.orchestra.run.smithi053.stdout: "can_run": true, 2024-04-25T14:50:48.657 INFO:teuthology.orchestra.run.smithi053.stdout: "error_string": "", 2024-04-25T14:50:48.657 INFO:teuthology.orchestra.run.smithi053.stdout: "module_options": { 2024-04-25T14:50:48.657 INFO:teuthology.orchestra.run.smithi053.stdout: "discovery_interval": { 2024-04-25T14:50:48.657 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "discovery_interval", 2024-04-25T14:50:48.658 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "uint", 2024-04-25T14:50:48.658 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.658 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.658 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "100", 2024-04-25T14:50:48.658 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.658 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.658 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.658 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.658 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.659 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.659 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.659 INFO:teuthology.orchestra.run.smithi053.stdout: "identifier": { 2024-04-25T14:50:48.659 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "identifier", 2024-04-25T14:50:48.659 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.659 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.659 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.659 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.659 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.660 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.660 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.660 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.660 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.660 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.660 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.660 INFO:teuthology.orchestra.run.smithi053.stdout: "interval": { 2024-04-25T14:50:48.660 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "interval", 2024-04-25T14:50:48.661 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "secs", 2024-04-25T14:50:48.661 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.661 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.661 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "60", 2024-04-25T14:50:48.661 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.661 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.661 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.661 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.661 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.662 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.662 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.662 INFO:teuthology.orchestra.run.smithi053.stdout: "log_level": { 2024-04-25T14:50:48.662 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_level", 2024-04-25T14:50:48.662 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.662 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.662 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.662 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.663 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.663 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.663 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.663 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.663 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.663 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.663 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.663 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.664 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.664 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.664 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.664 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.664 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.664 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.664 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster": { 2024-04-25T14:50:48.664 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster", 2024-04-25T14:50:48.664 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.665 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.665 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.665 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.665 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.665 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.665 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.665 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.665 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.665 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.666 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.666 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_cluster_level": { 2024-04-25T14:50:48.666 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_cluster_level", 2024-04-25T14:50:48.666 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.666 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.666 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.666 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "info", 2024-04-25T14:50:48.666 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.666 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.667 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [ 2024-04-25T14:50:48.667 INFO:teuthology.orchestra.run.smithi053.stdout: "", 2024-04-25T14:50:48.667 INFO:teuthology.orchestra.run.smithi053.stdout: "critical", 2024-04-25T14:50:48.667 INFO:teuthology.orchestra.run.smithi053.stdout: "debug", 2024-04-25T14:50:48.667 INFO:teuthology.orchestra.run.smithi053.stdout: "error", 2024-04-25T14:50:48.667 INFO:teuthology.orchestra.run.smithi053.stdout: "info", 2024-04-25T14:50:48.667 INFO:teuthology.orchestra.run.smithi053.stdout: "warning" 2024-04-25T14:50:48.667 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.668 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.668 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.668 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.668 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.668 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": { 2024-04-25T14:50:48.668 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "log_to_file", 2024-04-25T14:50:48.668 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "bool", 2024-04-25T14:50:48.668 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.669 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-04-25T14:50:48.669 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "False", 2024-04-25T14:50:48.669 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.669 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.669 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.669 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.669 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.669 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.669 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.670 INFO:teuthology.orchestra.run.smithi053.stdout: "zabbix_host": { 2024-04-25T14:50:48.670 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "zabbix_host", 2024-04-25T14:50:48.670 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.670 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.670 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.670 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "", 2024-04-25T14:50:48.670 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.670 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.671 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.671 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.671 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.671 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.671 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.671 INFO:teuthology.orchestra.run.smithi053.stdout: "zabbix_port": { 2024-04-25T14:50:48.671 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "zabbix_port", 2024-04-25T14:50:48.672 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "int", 2024-04-25T14:50:48.672 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.672 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.672 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "10051", 2024-04-25T14:50:48.672 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.672 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.672 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.672 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.672 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.673 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.673 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.673 INFO:teuthology.orchestra.run.smithi053.stdout: "zabbix_sender": { 2024-04-25T14:50:48.673 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "zabbix_sender", 2024-04-25T14:50:48.673 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "str", 2024-04-25T14:50:48.673 INFO:teuthology.orchestra.run.smithi053.stdout: "level": "advanced", 2024-04-25T14:50:48.673 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 0, 2024-04-25T14:50:48.674 INFO:teuthology.orchestra.run.smithi053.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T14:50:48.674 INFO:teuthology.orchestra.run.smithi053.stdout: "min": "", 2024-04-25T14:50:48.674 INFO:teuthology.orchestra.run.smithi053.stdout: "max": "", 2024-04-25T14:50:48.674 INFO:teuthology.orchestra.run.smithi053.stdout: "enum_allowed": [], 2024-04-25T14:50:48.674 INFO:teuthology.orchestra.run.smithi053.stdout: "desc": "", 2024-04-25T14:50:48.674 INFO:teuthology.orchestra.run.smithi053.stdout: "long_desc": "", 2024-04-25T14:50:48.674 INFO:teuthology.orchestra.run.smithi053.stdout: "tags": [], 2024-04-25T14:50:48.674 INFO:teuthology.orchestra.run.smithi053.stdout: "see_also": [] 2024-04-25T14:50:48.675 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.675 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.675 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.675 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.675 INFO:teuthology.orchestra.run.smithi053.stdout: "services": { 2024-04-25T14:50:48.675 INFO:teuthology.orchestra.run.smithi053.stdout: "dashboard": "https://172.21.15.53:8443/" 2024-04-25T14:50:48.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.675 INFO:teuthology.orchestra.run.smithi053.stdout: "always_on_modules": { 2024-04-25T14:50:48.675 INFO:teuthology.orchestra.run.smithi053.stdout: "octopus": [ 2024-04-25T14:50:48.676 INFO:teuthology.orchestra.run.smithi053.stdout: "balancer", 2024-04-25T14:50:48.676 INFO:teuthology.orchestra.run.smithi053.stdout: "crash", 2024-04-25T14:50:48.676 INFO:teuthology.orchestra.run.smithi053.stdout: "devicehealth", 2024-04-25T14:50:48.676 INFO:teuthology.orchestra.run.smithi053.stdout: "orchestrator", 2024-04-25T14:50:48.676 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscaler", 2024-04-25T14:50:48.676 INFO:teuthology.orchestra.run.smithi053.stdout: "progress", 2024-04-25T14:50:48.676 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_support", 2024-04-25T14:50:48.676 INFO:teuthology.orchestra.run.smithi053.stdout: "status", 2024-04-25T14:50:48.676 INFO:teuthology.orchestra.run.smithi053.stdout: "telemetry", 2024-04-25T14:50:48.677 INFO:teuthology.orchestra.run.smithi053.stdout: "volumes" 2024-04-25T14:50:48.677 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.677 INFO:teuthology.orchestra.run.smithi053.stdout: "pacific": [ 2024-04-25T14:50:48.677 INFO:teuthology.orchestra.run.smithi053.stdout: "balancer", 2024-04-25T14:50:48.677 INFO:teuthology.orchestra.run.smithi053.stdout: "crash", 2024-04-25T14:50:48.677 INFO:teuthology.orchestra.run.smithi053.stdout: "devicehealth", 2024-04-25T14:50:48.677 INFO:teuthology.orchestra.run.smithi053.stdout: "orchestrator", 2024-04-25T14:50:48.677 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscaler", 2024-04-25T14:50:48.677 INFO:teuthology.orchestra.run.smithi053.stdout: "progress", 2024-04-25T14:50:48.677 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_support", 2024-04-25T14:50:48.678 INFO:teuthology.orchestra.run.smithi053.stdout: "status", 2024-04-25T14:50:48.678 INFO:teuthology.orchestra.run.smithi053.stdout: "telemetry", 2024-04-25T14:50:48.678 INFO:teuthology.orchestra.run.smithi053.stdout: "volumes" 2024-04-25T14:50:48.678 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.678 INFO:teuthology.orchestra.run.smithi053.stdout: "quincy": [ 2024-04-25T14:50:48.678 INFO:teuthology.orchestra.run.smithi053.stdout: "balancer", 2024-04-25T14:50:48.678 INFO:teuthology.orchestra.run.smithi053.stdout: "crash", 2024-04-25T14:50:48.678 INFO:teuthology.orchestra.run.smithi053.stdout: "devicehealth", 2024-04-25T14:50:48.678 INFO:teuthology.orchestra.run.smithi053.stdout: "orchestrator", 2024-04-25T14:50:48.679 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscaler", 2024-04-25T14:50:48.679 INFO:teuthology.orchestra.run.smithi053.stdout: "progress", 2024-04-25T14:50:48.679 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_support", 2024-04-25T14:50:48.679 INFO:teuthology.orchestra.run.smithi053.stdout: "status", 2024-04-25T14:50:48.679 INFO:teuthology.orchestra.run.smithi053.stdout: "telemetry", 2024-04-25T14:50:48.679 INFO:teuthology.orchestra.run.smithi053.stdout: "volumes" 2024-04-25T14:50:48.679 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.679 INFO:teuthology.orchestra.run.smithi053.stdout: "reef": [ 2024-04-25T14:50:48.679 INFO:teuthology.orchestra.run.smithi053.stdout: "balancer", 2024-04-25T14:50:48.679 INFO:teuthology.orchestra.run.smithi053.stdout: "crash", 2024-04-25T14:50:48.680 INFO:teuthology.orchestra.run.smithi053.stdout: "devicehealth", 2024-04-25T14:50:48.680 INFO:teuthology.orchestra.run.smithi053.stdout: "orchestrator", 2024-04-25T14:50:48.680 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscaler", 2024-04-25T14:50:48.680 INFO:teuthology.orchestra.run.smithi053.stdout: "progress", 2024-04-25T14:50:48.680 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_support", 2024-04-25T14:50:48.680 INFO:teuthology.orchestra.run.smithi053.stdout: "status", 2024-04-25T14:50:48.680 INFO:teuthology.orchestra.run.smithi053.stdout: "telemetry", 2024-04-25T14:50:48.680 INFO:teuthology.orchestra.run.smithi053.stdout: "volumes" 2024-04-25T14:50:48.680 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-04-25T14:50:48.680 INFO:teuthology.orchestra.run.smithi053.stdout: "squid": [ 2024-04-25T14:50:48.681 INFO:teuthology.orchestra.run.smithi053.stdout: "balancer", 2024-04-25T14:50:48.681 INFO:teuthology.orchestra.run.smithi053.stdout: "crash", 2024-04-25T14:50:48.681 INFO:teuthology.orchestra.run.smithi053.stdout: "devicehealth", 2024-04-25T14:50:48.681 INFO:teuthology.orchestra.run.smithi053.stdout: "orchestrator", 2024-04-25T14:50:48.681 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscaler", 2024-04-25T14:50:48.681 INFO:teuthology.orchestra.run.smithi053.stdout: "progress", 2024-04-25T14:50:48.681 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_support", 2024-04-25T14:50:48.681 INFO:teuthology.orchestra.run.smithi053.stdout: "status", 2024-04-25T14:50:48.681 INFO:teuthology.orchestra.run.smithi053.stdout: "telemetry", 2024-04-25T14:50:48.681 INFO:teuthology.orchestra.run.smithi053.stdout: "volumes" 2024-04-25T14:50:48.682 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-04-25T14:50:48.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.682 INFO:teuthology.orchestra.run.smithi053.stdout: "last_failure_osd_epoch": 93, 2024-04-25T14:50:48.682 INFO:teuthology.orchestra.run.smithi053.stdout: "active_clients": [ 2024-04-25T14:50:48.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.682 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "devicehealth", 2024-04-25T14:50:48.682 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-04-25T14:50:48.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.682 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-04-25T14:50:48.683 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:0", 2024-04-25T14:50:48.683 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2668564429 2024-04-25T14:50:48.683 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.683 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-04-25T14:50:48.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.683 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "libcephsqlite", 2024-04-25T14:50:48.683 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-04-25T14:50:48.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.683 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-04-25T14:50:48.684 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:0", 2024-04-25T14:50:48.684 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2504465022 2024-04-25T14:50:48.684 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.684 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-04-25T14:50:48.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.684 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "rbd_support", 2024-04-25T14:50:48.684 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-04-25T14:50:48.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.685 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-04-25T14:50:48.685 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:0", 2024-04-25T14:50:48.685 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 675566526 2024-04-25T14:50:48.685 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.685 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-04-25T14:50:48.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.685 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "snap_schedule", 2024-04-25T14:50:48.685 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-04-25T14:50:48.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.686 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-04-25T14:50:48.686 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:0", 2024-04-25T14:50:48.686 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3536155710 2024-04-25T14:50:48.686 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.686 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-04-25T14:50:48.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.687 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "snap_schedule", 2024-04-25T14:50:48.687 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-04-25T14:50:48.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.687 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-04-25T14:50:48.687 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:0", 2024-04-25T14:50:48.687 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1471180288 2024-04-25T14:50:48.687 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.687 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-04-25T14:50:48.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:48.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.688 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "volumes", 2024-04-25T14:50:48.688 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-04-25T14:50:48.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-04-25T14:50:48.688 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-04-25T14:50:48.688 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:0", 2024-04-25T14:50:48.688 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 780654087 2024-04-25T14:50:48.688 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.688 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-04-25T14:50:48.689 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:48.689 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-04-25T14:50:48.689 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-04-25T14:50:48.690 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:46 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/1033125475' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:50:48.691 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:46 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/2497011596' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:50:48.691 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:47 smithi150 ceph-mon[32504]: pgmap v314: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T14:50:48.691 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:47 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/2739086090' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T14:50:48.691 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:46 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/1033125475' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:50:48.691 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:46 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/2497011596' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:50:48.691 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:47 smithi183 ceph-mon[32584]: pgmap v314: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T14:50:48.691 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:47 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/2739086090' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T14:50:48.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.686+0000 7f8b46600640 1 -- 172.21.15.53:0/2739086090 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f8b280a3d20 msgr2=0x7f8b280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:50:48.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.686+0000 7f8b46600640 1 --2- 172.21.15.53:0/2739086090 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f8b280a3d20 0x7f8b280a61e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c0095f0 tx=0x7f8b2c005f90 comp rx=0 tx=0).stop 2024-04-25T14:50:48.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.686+0000 7f8b46600640 1 -- 172.21.15.53:0/2739086090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b40076480 msgr2=0x7f8b4019ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:50:48.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.686+0000 7f8b46600640 1 --2- 172.21.15.53:0/2739086090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b40076480 0x7f8b4019ecc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8b34005940 tx=0x7f8b340040c0 comp rx=0 tx=0).stop 2024-04-25T14:50:48.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.686+0000 7f8b46600640 1 -- 172.21.15.53:0/2739086090 shutdown_connections 2024-04-25T14:50:48.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.686+0000 7f8b46600640 1 --2- 172.21.15.53:0/2739086090 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f8b280a3d20 0x7f8b280a61e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:48.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.686+0000 7f8b46600640 1 --2- 172.21.15.53:0/2739086090 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b40112bf0 0x7f8b401a24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:48.693 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.686+0000 7f8b46600640 1 --2- 172.21.15.53:0/2739086090 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b40076da0 0x7f8b4019f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:48.693 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.686+0000 7f8b46600640 1 --2- 172.21.15.53:0/2739086090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b40076480 0x7f8b4019ecc0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:48.693 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.686+0000 7f8b46600640 1 -- 172.21.15.53:0/2739086090 >> 172.21.15.53:0/2739086090 conn(0x7f8b400ffa20 msgr2=0x7f8b401143c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:50:48.693 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.686+0000 7f8b46600640 1 -- 172.21.15.53:0/2739086090 shutdown_connections 2024-04-25T14:50:48.693 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:50:46.686+0000 7f8b46600640 1 -- 172.21.15.53:0/2739086090 wait complete. 2024-04-25T14:50:48.699 DEBUG:tasks.check_counter:Getting stats from a 2024-04-25T14:50:48.699 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-25T14:50:48.699 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:46 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1033125475' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:50:48.700 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:46 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/2497011596' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:50:48.700 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:47 smithi053 ceph-mon[27773]: pgmap v314: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T14:50:48.700 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:47 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/2739086090' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T14:50:49.172 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:50:49.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:49 smithi183 ceph-mon[32584]: pgmap v315: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T14:50:49.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:49 smithi150 ceph-mon[32504]: pgmap v315: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T14:50:50.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:49 smithi053 ceph-mon[27773]: pgmap v315: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T14:50:51.656 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-04-25T14:50:51.656 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:50:51.657 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 0, 2024-04-25T14:50:51.657 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 2, 2024-04-25T14:50:51.657 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 0, 2024-04-25T14:50:51.657 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 0, 2024-04-25T14:50:51.657 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 4, 2024-04-25T14:50:51.657 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 0, 2024-04-25T14:50:51.657 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.004462113, 2024-04-25T14:50:51.657 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T14:50:51.657 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.000907476, 2024-04-25T14:50:51.658 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T14:50:51.658 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:50:51.658 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:51.658 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:51.658 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:51.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:51.658 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:50:51.658 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:51.659 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:51.659 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:51.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:51.659 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T14:50:51.659 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T14:50:51.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:51.659 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:50:51.659 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 2, 2024-04-25T14:50:51.659 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 158, 2024-04-25T14:50:51.660 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 320, 2024-04-25T14:50:51.660 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 183888, 2024-04-25T14:50:51.660 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 5, 2024-04-25T14:50:51.660 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 3, 2024-04-25T14:50:51.660 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.034188454, 2024-04-25T14:50:51.660 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.024510918, 2024-04-25T14:50:51.660 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.003798665, 2024-04-25T14:50:51.660 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T14:50:51.660 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:50:51.660 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 155, 2024-04-25T14:50:51.661 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.027612608, 2024-04-25T14:50:51.661 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000178145 2024-04-25T14:50:51.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:51.661 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:50:51.661 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:51.661 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:51.661 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:51.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:51.661 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T14:50:51.662 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 183888 2024-04-25T14:50:51.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:51.662 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:50:51.662 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 223, 2024-04-25T14:50:51.662 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 209, 2024-04-25T14:50:51.662 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 1730400, 2024-04-25T14:50:51.662 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 127712, 2024-04-25T14:50:51.662 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 2, 2024-04-25T14:50:51.663 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 1, 2024-04-25T14:50:51.663 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.101200916, 2024-04-25T14:50:51.663 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.032282943, 2024-04-25T14:50:51.663 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.084862972, 2024-04-25T14:50:51.663 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.009654721, 2024-04-25T14:50:51.663 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:50:51.663 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 209, 2024-04-25T14:50:51.663 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.022258763, 2024-04-25T14:50:51.663 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000106501 2024-04-25T14:50:51.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:51.664 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:50:51.664 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:51.664 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:51.664 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:51.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:51.664 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 1730400, 2024-04-25T14:50:51.664 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 127712 2024-04-25T14:50:51.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:51.665 INFO:teuthology.orchestra.run.smithi053.stdout: "cct": { 2024-04-25T14:50:51.665 INFO:teuthology.orchestra.run.smithi053.stdout: "total_workers": 0, 2024-04-25T14:50:51.665 INFO:teuthology.orchestra.run.smithi053.stdout: "unhealthy_workers": 0 2024-04-25T14:50:51.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:51.665 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool": { 2024-04-25T14:50:51.665 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:50:51.665 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_items": 0, 2024-04-25T14:50:51.665 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:50:51.665 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:50:51.665 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:50:51.666 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:50:51.666 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:50:51.666 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:50:51.666 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:50:51.666 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:50:51.666 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:50:51.666 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:50:51.666 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:50:51.666 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:50:51.667 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:50:51.667 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_items": 0, 2024-04-25T14:50:51.667 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:50:51.667 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_items": 0, 2024-04-25T14:50:51.667 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:50:51.667 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:50:51.667 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:50:51.667 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:50:51.667 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:50:51.668 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:50:51.668 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:50:51.668 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_items": 0, 2024-04-25T14:50:51.668 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:50:51.668 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:50:51.668 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:50:51.668 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_items": 0, 2024-04-25T14:50:51.668 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_bytes": 0, 2024-04-25T14:50:51.668 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_items": 0, 2024-04-25T14:50:51.669 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:50:51.669 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:50:51.669 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:50:51.669 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:50:51.669 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_bytes": 148977, 2024-04-25T14:50:51.669 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_items": 30, 2024-04-25T14:50:51.669 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_bytes": 0, 2024-04-25T14:50:51.669 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_items": 0, 2024-04-25T14:50:51.669 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bytes": 0, 2024-04-25T14:50:51.670 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_items": 0, 2024-04-25T14:50:51.670 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:50:51.670 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_items": 0, 2024-04-25T14:50:51.670 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:50:51.670 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_items": 0, 2024-04-25T14:50:51.670 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_bytes": 0, 2024-04-25T14:50:51.670 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_items": 0, 2024-04-25T14:50:51.670 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:50:51.671 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:50:51.671 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_bytes": 0, 2024-04-25T14:50:51.671 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_items": 0, 2024-04-25T14:50:51.671 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_bytes": 928, 2024-04-25T14:50:51.671 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_items": 2, 2024-04-25T14:50:51.671 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_bytes": 0, 2024-04-25T14:50:51.671 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_items": 0, 2024-04-25T14:50:51.671 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_bytes": 0, 2024-04-25T14:50:51.671 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_items": 0 2024-04-25T14:50:51.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:51.672 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:50:51.672 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-04-25T14:50:51.672 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-04-25T14:50:51.672 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-04-25T14:50:51.672 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 221, 2024-04-25T14:50:51.672 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 1696546, 2024-04-25T14:50:51.672 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-04-25T14:50:51.672 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 221, 2024-04-25T14:50:51.672 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-04-25T14:50:51.673 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-04-25T14:50:51.673 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 221, 2024-04-25T14:50:51.673 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 1696546, 2024-04-25T14:50:51.673 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-04-25T14:50:51.673 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:51.673 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:51.673 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:51.673 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:51.673 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:51.674 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-04-25T14:50:51.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:51 smithi183 ceph-mon[32584]: pgmap v316: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T14:50:51.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:51 smithi150 ceph-mon[32504]: pgmap v316: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T14:50:52.014 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:51 smithi053 ceph-mon[27773]: pgmap v316: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T14:50:52.571 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-04-25T14:50:52.571 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-04-25T14:50:52.571 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-04-25T14:50:52.572 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-04-25T14:50:52.572 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-04-25T14:50:52.572 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-04-25T14:50:52.572 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-04-25T14:50:52.572 DEBUG:tasks.check_counter:Getting stats from d 2024-04-25T14:50:52.572 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-25T14:50:52.997 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:50:53.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:53 smithi053 ceph-mon[27773]: pgmap v317: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T14:50:53.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:53 smithi183 ceph-mon[32584]: pgmap v317: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T14:50:53.987 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:53 smithi150 ceph-mon[32504]: pgmap v317: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T14:50:55.348 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-04-25T14:50:55.349 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:50:55.349 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 225, 2024-04-25T14:50:55.349 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 213, 2024-04-25T14:50:55.349 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 1729040, 2024-04-25T14:50:55.349 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 131104, 2024-04-25T14:50:55.349 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 4, 2024-04-25T14:50:55.349 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 0, 2024-04-25T14:50:55.349 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.111414907, 2024-04-25T14:50:55.350 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.024280456, 2024-04-25T14:50:55.350 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.105883535, 2024-04-25T14:50:55.350 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.010714831, 2024-04-25T14:50:55.350 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:50:55.350 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 212, 2024-04-25T14:50:55.350 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.020945008, 2024-04-25T14:50:55.350 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000098797 2024-04-25T14:50:55.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:55.350 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:50:55.351 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:55.351 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:55.351 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:55.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:55.351 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 1729040, 2024-04-25T14:50:55.351 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 131104 2024-04-25T14:50:55.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:55.351 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:50:55.352 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 2, 2024-04-25T14:50:55.352 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 163, 2024-04-25T14:50:55.352 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 320, 2024-04-25T14:50:55.352 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 184768, 2024-04-25T14:50:55.352 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 5, 2024-04-25T14:50:55.352 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 3, 2024-04-25T14:50:55.352 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.036470081, 2024-04-25T14:50:55.352 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.025921820, 2024-04-25T14:50:55.352 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.004177420, 2024-04-25T14:50:55.353 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T14:50:55.353 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:50:55.353 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 160, 2024-04-25T14:50:55.353 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.026316999, 2024-04-25T14:50:55.353 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000164481 2024-04-25T14:50:55.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:55.353 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:50:55.353 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:55.353 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:55.354 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:55.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:55.354 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T14:50:55.354 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 184768 2024-04-25T14:50:55.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:55.354 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:50:55.354 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 4, 2024-04-25T14:50:55.354 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 4, 2024-04-25T14:50:55.354 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 2992, 2024-04-25T14:50:55.355 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 448, 2024-04-25T14:50:55.355 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 2, 2024-04-25T14:50:55.355 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 1, 2024-04-25T14:50:55.355 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.001682905, 2024-04-25T14:50:55.355 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.000059091, 2024-04-25T14:50:55.355 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.001499237, 2024-04-25T14:50:55.355 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T14:50:55.355 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:50:55.356 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 3, 2024-04-25T14:50:55.356 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.001699957, 2024-04-25T14:50:55.356 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000566652 2024-04-25T14:50:55.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:55.356 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:50:55.356 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:55.356 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:55.356 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:55.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:55.356 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-25T14:50:55.357 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T14:50:55.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:55.357 INFO:teuthology.orchestra.run.smithi053.stdout: "cct": { 2024-04-25T14:50:55.357 INFO:teuthology.orchestra.run.smithi053.stdout: "total_workers": 0, 2024-04-25T14:50:55.357 INFO:teuthology.orchestra.run.smithi053.stdout: "unhealthy_workers": 0 2024-04-25T14:50:55.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:55.357 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool": { 2024-04-25T14:50:55.357 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:50:55.357 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_items": 0, 2024-04-25T14:50:55.358 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:50:55.358 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:50:55.358 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:50:55.358 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:50:55.358 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:50:55.358 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:50:55.358 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:50:55.358 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:50:55.359 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:50:55.359 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:50:55.359 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:50:55.359 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:50:55.359 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:50:55.359 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_items": 0, 2024-04-25T14:50:55.359 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:50:55.359 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_items": 0, 2024-04-25T14:50:55.359 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:50:55.360 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:50:55.360 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:50:55.360 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:50:55.360 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:50:55.360 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:50:55.360 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:50:55.360 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_items": 0, 2024-04-25T14:50:55.360 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:50:55.360 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:50:55.361 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:50:55.361 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_items": 0, 2024-04-25T14:50:55.361 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_bytes": 0, 2024-04-25T14:50:55.361 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_items": 0, 2024-04-25T14:50:55.361 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:50:55.361 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:50:55.361 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:50:55.361 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:50:55.361 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_bytes": 148714, 2024-04-25T14:50:55.362 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_items": 30, 2024-04-25T14:50:55.362 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_bytes": 0, 2024-04-25T14:50:55.362 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_items": 0, 2024-04-25T14:50:55.362 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bytes": 0, 2024-04-25T14:50:55.362 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_items": 0, 2024-04-25T14:50:55.362 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:50:55.362 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_items": 0, 2024-04-25T14:50:55.362 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:50:55.362 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_items": 0, 2024-04-25T14:50:55.363 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_bytes": 0, 2024-04-25T14:50:55.363 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_items": 0, 2024-04-25T14:50:55.363 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:50:55.363 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:50:55.363 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_bytes": 0, 2024-04-25T14:50:55.363 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_items": 0, 2024-04-25T14:50:55.363 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_bytes": 928, 2024-04-25T14:50:55.363 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_items": 2, 2024-04-25T14:50:55.364 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_bytes": 0, 2024-04-25T14:50:55.364 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_items": 0, 2024-04-25T14:50:55.364 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_bytes": 0, 2024-04-25T14:50:55.364 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_items": 0 2024-04-25T14:50:55.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:55.364 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:50:55.364 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-04-25T14:50:55.364 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-04-25T14:50:55.364 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-04-25T14:50:55.365 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 227, 2024-04-25T14:50:55.365 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 1697326, 2024-04-25T14:50:55.365 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-04-25T14:50:55.365 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 227, 2024-04-25T14:50:55.365 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-04-25T14:50:55.365 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-04-25T14:50:55.365 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 227, 2024-04-25T14:50:55.365 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 1697326, 2024-04-25T14:50:55.365 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-04-25T14:50:55.366 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:55.366 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:55.366 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:55.366 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:55.366 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:55.366 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-04-25T14:50:55.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:55 smithi183 ceph-mon[32584]: pgmap v318: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T14:50:55.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:55 smithi150 ceph-mon[32504]: pgmap v318: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T14:50:56.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:55 smithi053 ceph-mon[27773]: pgmap v318: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T14:50:56.211 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-25T14:50:56.211 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-25T14:50:56.211 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-04-25T14:50:56.211 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-04-25T14:50:56.211 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-25T14:50:56.211 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-25T14:50:56.212 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-25T14:50:56.212 DEBUG:tasks.check_counter:Getting stats from g 2024-04-25T14:50:56.212 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-25T14:50:56.693 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:50:57.100 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:56 smithi053 ceph-mon[27773]: pgmap v319: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T14:50:57.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:56 smithi183 ceph-mon[32584]: pgmap v319: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T14:50:57.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:56 smithi150 ceph-mon[32504]: pgmap v319: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T14:50:58.956 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-04-25T14:50:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:50:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 353, 2024-04-25T14:50:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 560, 2024-04-25T14:50:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 96928, 2024-04-25T14:50:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 330020, 2024-04-25T14:50:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 10, 2024-04-25T14:50:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 3, 2024-04-25T14:50:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.127949374, 2024-04-25T14:50:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.057480314, 2024-04-25T14:50:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.054188178, 2024-04-25T14:50:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.007900951, 2024-04-25T14:50:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:50:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 559, 2024-04-25T14:50:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.089003660, 2024-04-25T14:50:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000159219 2024-04-25T14:50:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:50:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 849, 2024-04-25T14:50:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000430989, 2024-04-25T14:50:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000507 2024-04-25T14:50:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 6480, 2024-04-25T14:50:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 11504 2024-04-25T14:50:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:50:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 286, 2024-04-25T14:50:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 396, 2024-04-25T14:50:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 1765046, 2024-04-25T14:50:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 459561, 2024-04-25T14:50:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 6, 2024-04-25T14:50:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 5, 2024-04-25T14:50:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.133075677, 2024-04-25T14:50:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.047254768, 2024-04-25T14:50:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.102734266, 2024-04-25T14:50:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.011376494, 2024-04-25T14:50:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:50:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 395, 2024-04-25T14:50:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.074145375, 2024-04-25T14:50:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000187709 2024-04-25T14:50:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:50:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 1760672, 2024-04-25T14:50:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 446592 2024-04-25T14:50:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.963 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:50:58.963 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 81, 2024-04-25T14:50:58.963 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 133, 2024-04-25T14:50:58.963 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 26644, 2024-04-25T14:50:58.963 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 213473, 2024-04-25T14:50:58.963 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 15, 2024-04-25T14:50:58.963 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 4, 2024-04-25T14:50:58.963 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.058534909, 2024-04-25T14:50:58.963 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.030076999, 2024-04-25T14:50:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.025884914, 2024-04-25T14:50:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.000156733, 2024-04-25T14:50:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:50:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 130, 2024-04-25T14:50:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.020699149, 2024-04-25T14:50:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000159224 2024-04-25T14:50:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:50:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 159, 2024-04-25T14:50:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000067538, 2024-04-25T14:50:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000424 2024-04-25T14:50:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 3920, 2024-04-25T14:50:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 141792 2024-04-25T14:50:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: "cct": { 2024-04-25T14:50:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: "total_workers": 1, 2024-04-25T14:50:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: "unhealthy_workers": 0 2024-04-25T14:50:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-MDSRank": { 2024-04-25T14:50:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-04-25T14:50:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-04-25T14:50:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 54, 2024-04-25T14:50:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.003119610, 2024-04-25T14:50:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000057770 2024-04-25T14:50:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-PurgeQueue": { 2024-04-25T14:50:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-04-25T14:50:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-04-25T14:50:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 2, 2024-04-25T14:50:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000024627, 2024-04-25T14:50:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000012313 2024-04-25T14:50:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": { 2024-04-25T14:50:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: "request": 13, 2024-04-25T14:50:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: "reply": 7, 2024-04-25T14:50:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: "reply_latency": { 2024-04-25T14:50:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 7, 2024-04-25T14:50:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.003581048, 2024-04-25T14:50:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000511578 2024-04-25T14:50:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_reply": 0, 2024-04-25T14:50:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: "forward": 6, 2024-04-25T14:50:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_complete": 0, 2024-04-25T14:50:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_keys": 0, 2024-04-25T14:50:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_commit": 11, 2024-04-25T14:50:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_split": 0, 2024-04-25T14:50:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_merge": 0, 2024-04-25T14:50:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes": 14, 2024-04-25T14:50:58.970 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_top": 1, 2024-04-25T14:50:58.970 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_bottom": 13, 2024-04-25T14:50:58.970 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pin_tail": 0, 2024-04-25T14:50:58.970 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pinned": 12, 2024-04-25T14:50:58.970 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_expired": 1, 2024-04-25T14:50:58.970 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_with_caps": 1, 2024-04-25T14:50:58.970 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": 1, 2024-04-25T14:50:58.970 INFO:teuthology.orchestra.run.smithi053.stdout: "subtrees": 2, 2024-04-25T14:50:58.971 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse": 24, 2024-04-25T14:50:58.971 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_hit": 14, 2024-04-25T14:50:58.971 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_forward": 2, 2024-04-25T14:50:58.971 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_discover": 4, 2024-04-25T14:50:58.971 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_dir_fetch": 0, 2024-04-25T14:50:58.971 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_remote_ino": 0, 2024-04-25T14:50:58.971 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_lock": 0, 2024-04-25T14:50:58.971 INFO:teuthology.orchestra.run.smithi053.stdout: "load_cent": 0, 2024-04-25T14:50:58.971 INFO:teuthology.orchestra.run.smithi053.stdout: "q": 0, 2024-04-25T14:50:58.972 INFO:teuthology.orchestra.run.smithi053.stdout: "exported": 0, 2024-04-25T14:50:58.972 INFO:teuthology.orchestra.run.smithi053.stdout: "exported_inodes": 0, 2024-04-25T14:50:58.972 INFO:teuthology.orchestra.run.smithi053.stdout: "imported": 0, 2024-04-25T14:50:58.972 INFO:teuthology.orchestra.run.smithi053.stdout: "imported_inodes": 0, 2024-04-25T14:50:58.972 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_dir_fetch": 0, 2024-04-25T14:50:58.972 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_backtrace_fetch": 0, 2024-04-25T14:50:58.972 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_peer_discover": 0, 2024-04-25T14:50:58.972 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rfiles": 4, 2024-04-25T14:50:58.973 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rbytes": 234, 2024-04-25T14:50:58.973 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rsnaps": 0, 2024-04-25T14:50:58.973 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T14:50:58.973 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_set_tag": 0, 2024-04-25T14:50:58.973 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T14:50:58.973 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_inotable_repaired": 0, 2024-04-25T14:50:58.973 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_inodes": 0, 2024-04-25T14:50:58.973 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T14:50:58.973 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T14:50:58.974 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_file_inodes": 0, 2024-04-25T14:50:58.974 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_inode_file_caps": 0, 2024-04-25T14:50:58.974 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T14:50:58.974 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_grant": 0, 2024-04-25T14:50:58.974 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T14:50:58.974 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T14:50:58.974 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T14:50:58.974 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps": 0, 2024-04-25T14:50:58.975 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps_dirty": 0, 2024-04-25T14:50:58.975 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_cap_release": 0, 2024-04-25T14:50:58.975 INFO:teuthology.orchestra.run.smithi053.stdout: "process_request_cap_release": 1 2024-04-25T14:50:58.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.975 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache": { 2024-04-25T14:50:58.975 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update": 0, 2024-04-25T14:50:58.975 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update_receipt": 6, 2024-04-25T14:50:58.975 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_try_discover": 4, 2024-04-25T14:50:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_send_discover": 7, 2024-04-25T14:50:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_handle_discover": 0, 2024-04-25T14:50:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays": 0, 2024-04-25T14:50:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_delayed": 0, 2024-04-25T14:50:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_enqueuing": 0, 2024-04-25T14:50:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_created": 0, 2024-04-25T14:50:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_enqueued": 0, 2024-04-25T14:50:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_reintegrated": 0, 2024-04-25T14:50:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_migrated": 0, 2024-04-25T14:50:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_processing": 0, 2024-04-25T14:50:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_enqueued": 0, 2024-04-25T14:50:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_prioritized": 0, 2024-04-25T14:50:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_started": 0, 2024-04-25T14:50:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_completed": 0, 2024-04-25T14:50:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_quiesce_path": 0, 2024-04-25T14:50:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_quiesce_inode": 0, 2024-04-25T14:50:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T14:50:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_exportdir": 0, 2024-04-25T14:50:58.978 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_flush": 0, 2024-04-25T14:50:58.978 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragmentdir": 0, 2024-04-25T14:50:58.978 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragstats": 0, 2024-04-25T14:50:58.978 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_inodestats": 0 2024-04-25T14:50:58.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.978 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log": { 2024-04-25T14:50:58.978 INFO:teuthology.orchestra.run.smithi053.stdout: "evlrg": 0, 2024-04-25T14:50:58.978 INFO:teuthology.orchestra.run.smithi053.stdout: "evadd": 8, 2024-04-25T14:50:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: "evex": 0, 2024-04-25T14:50:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: "evtrm": 0, 2024-04-25T14:50:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: "ev": 8, 2024-04-25T14:50:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: "evexg": 0, 2024-04-25T14:50:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: "evexd": 0, 2024-04-25T14:50:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: "segadd": 2, 2024-04-25T14:50:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: "segex": 0, 2024-04-25T14:50:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: "segtrm": 0, 2024-04-25T14:50:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: "seg": 2, 2024-04-25T14:50:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: "segmjr": 2, 2024-04-25T14:50:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: "segexg": 0, 2024-04-25T14:50:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: "segexd": 0, 2024-04-25T14:50:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: "expos": 4194304, 2024-04-25T14:50:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: "wrpos": 4196657, 2024-04-25T14:50:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: "rdpos": 0, 2024-04-25T14:50:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: "jlat": { 2024-04-25T14:50:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 7, 2024-04-25T14:50:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.025669243, 2024-04-25T14:50:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.003667034 2024-04-25T14:50:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: "replayed": 0 2024-04-25T14:50:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_mem": { 2024-04-25T14:50:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: "ino": 17, 2024-04-25T14:50:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: "ino+": 17, 2024-04-25T14:50:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: "ino-": 0, 2024-04-25T14:50:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: "dir": 16, 2024-04-25T14:50:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: "dir+": 17, 2024-04-25T14:50:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: "dir-": 1, 2024-04-25T14:50:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: "dn": 14, 2024-04-25T14:50:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: "dn+": 15, 2024-04-25T14:50:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: "dn-": 1, 2024-04-25T14:50:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: "cap": 1, 2024-04-25T14:50:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: "cap+": 4, 2024-04-25T14:50:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: "cap-": 3, 2024-04-25T14:50:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: "rss": 51072, 2024-04-25T14:50:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: "heap": 223608 2024-04-25T14:50:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_server": { 2024-04-25T14:50:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_client_request": 18, 2024-04-25T14:50:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_server_request": 0, 2024-04-25T14:50:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_request": 13, 2024-04-25T14:50:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_session": 25, 2024-04-25T14:50:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_peer_request": 1, 2024-04-25T14:50:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: "req_create_latency": { 2024-04-25T14:50:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getattr_latency": { 2024-04-25T14:50:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 5, 2024-04-25T14:50:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.002831461, 2024-04-25T14:50:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000566292 2024-04-25T14:50:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getfilelock_latency": { 2024-04-25T14:50:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: "req_link_latency": { 2024-04-25T14:50:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:58.986 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:58.986 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:58.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.986 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookup_latency": { 2024-04-25T14:50:58.986 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 2, 2024-04-25T14:50:58.986 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000749587, 2024-04-25T14:50:58.986 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000374793 2024-04-25T14:50:58.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookuphash_latency": { 2024-04-25T14:50:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupino_latency": { 2024-04-25T14:50:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupname_latency": { 2024-04-25T14:50:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupparent_latency": { 2024-04-25T14:50:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupsnap_latency": { 2024-04-25T14:50:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lssnap_latency": { 2024-04-25T14:50:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mkdir_latency": { 2024-04-25T14:50:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mknod_latency": { 2024-04-25T14:50:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mksnap_latency": { 2024-04-25T14:50:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: "req_open_latency": { 2024-04-25T14:50:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: "req_readdir_latency": { 2024-04-25T14:50:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rename_latency": { 2024-04-25T14:50:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: "req_renamesnap_latency": { 2024-04-25T14:50:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: "req_snapdiff_latency": { 2024-04-25T14:50:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmdir_latency": { 2024-04-25T14:50:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmsnap_latency": { 2024-04-25T14:50:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmxattr_latency": { 2024-04-25T14:50:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setattr_latency": { 2024-04-25T14:50:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setdirlayout_latency": { 2024-04-25T14:50:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setfilelock_latency": { 2024-04-25T14:50:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setlayout_latency": { 2024-04-25T14:50:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setxattr_latency": { 2024-04-25T14:50:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "req_symlink_latency": { 2024-04-25T14:50:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "req_unlink_latency": { 2024-04-25T14:50:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_revoke_eviction": 0, 2024-04-25T14:50:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_acquisition_throttle": 0, 2024-04-25T14:50:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getvxattr_latency": { 2024-04-25T14:50:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_sessions": { 2024-04-25T14:50:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: "session_count": 2, 2024-04-25T14:50:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: "session_add": 4, 2024-04-25T14:50:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: "session_remove": 2, 2024-04-25T14:50:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_open": 2, 2024-04-25T14:50:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_stale": 0, 2024-04-25T14:50:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: "total_load": 5, 2024-04-25T14:50:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "average_load": 1, 2024-04-25T14:50:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "avg_session_uptime": 608, 2024-04-25T14:50:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "mdthresh_evicted": 0 2024-04-25T14:50:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool": { 2024-04-25T14:50:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:50:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_items": 0, 2024-04-25T14:50:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:50:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:50:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:50:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:50:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:50:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:50:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:50:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:50:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:50:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:50:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:50:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:50:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:50:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_items": 0, 2024-04-25T14:50:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:50:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_items": 0, 2024-04-25T14:50:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:50:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:50:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:50:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:50:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:50:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:50:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:50:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_items": 0, 2024-04-25T14:50:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:50:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:50:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:50:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_items": 0, 2024-04-25T14:50:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_bytes": 0, 2024-04-25T14:50:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_items": 0, 2024-04-25T14:50:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:50:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:50:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:50:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:50:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_bytes": 257099, 2024-04-25T14:50:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_items": 100, 2024-04-25T14:50:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_bytes": 176, 2024-04-25T14:50:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_items": 2, 2024-04-25T14:50:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bytes": 0, 2024-04-25T14:50:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_items": 0, 2024-04-25T14:50:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:50:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_items": 0, 2024-04-25T14:50:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:50:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_items": 0, 2024-04-25T14:50:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_bytes": 8240, 2024-04-25T14:50:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_items": 173, 2024-04-25T14:50:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:50:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:50:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_bytes": 0, 2024-04-25T14:50:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_items": 0, 2024-04-25T14:50:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_bytes": 69496, 2024-04-25T14:50:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_items": 266, 2024-04-25T14:50:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_bytes": 0, 2024-04-25T14:50:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_items": 0, 2024-04-25T14:50:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_bytes": 0, 2024-04-25T14:50:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_items": 0 2024-04-25T14:50:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter": { 2024-04-25T14:50:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: "op_active": 0, 2024-04-25T14:50:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: "op_laggy": 0, 2024-04-25T14:50:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send": 47, 2024-04-25T14:50:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send_bytes": 12980, 2024-04-25T14:50:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: "op_resend": 30, 2024-04-25T14:50:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: "op_reply": 47, 2024-04-25T14:50:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: "op_latency": { 2024-04-25T14:50:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 47, 2024-04-25T14:50:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.383992356, 2024-04-25T14:50:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.008170050 2024-04-25T14:50:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: "op_inflight": 0, 2024-04-25T14:50:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: "oplen_avg": { 2024-04-25T14:50:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 47, 2024-04-25T14:50:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 71 2024-04-25T14:50:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: "op": 47, 2024-04-25T14:50:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: "op_r": 0, 2024-04-25T14:50:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: "op_w": 47, 2024-04-25T14:50:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: "op_rmw": 0, 2024-04-25T14:50:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: "op_pg": 0, 2024-04-25T14:50:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_stat": 0, 2024-04-25T14:50:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_create": 10, 2024-04-25T14:50:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_read": 0, 2024-04-25T14:50:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_write": 7, 2024-04-25T14:50:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writefull": 9, 2024-04-25T14:50:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writesame": 0, 2024-04-25T14:50:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_append": 0, 2024-04-25T14:50:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_zero": 0, 2024-04-25T14:50:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_truncate": 0, 2024-04-25T14:50:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_delete": 6, 2024-04-25T14:50:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_mapext": 0, 2024-04-25T14:50:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_sparse_read": 0, 2024-04-25T14:50:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_clonerange": 0, 2024-04-25T14:50:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_getxattr": 0, 2024-04-25T14:50:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_setxattr": 20, 2024-04-25T14:50:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_cmpxattr": 0, 2024-04-25T14:50:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_rmxattr": 0, 2024-04-25T14:50:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_resetxattrs": 0, 2024-04-25T14:50:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_call": 0, 2024-04-25T14:50:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_watch": 0, 2024-04-25T14:50:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_notify": 0, 2024-04-25T14:50:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T14:50:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls": 0, 2024-04-25T14:50:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls_filter": 0, 2024-04-25T14:50:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_other": 0, 2024-04-25T14:50:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_active": 0, 2024-04-25T14:50:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_send": 0, 2024-04-25T14:50:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_resend": 0, 2024-04-25T14:50:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_ping": 0, 2024-04-25T14:50:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_active": 0, 2024-04-25T14:50:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_send": 0, 2024-04-25T14:50:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_resend": 0, 2024-04-25T14:50:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_active": 0, 2024-04-25T14:50:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_send": 0, 2024-04-25T14:50:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_resend": 0, 2024-04-25T14:50:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_active": 0, 2024-04-25T14:50:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_send": 0, 2024-04-25T14:50:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_resend": 0, 2024-04-25T14:50:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "command_active": 0, 2024-04-25T14:50:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "command_send": 0, 2024-04-25T14:50:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "command_resend": 0, 2024-04-25T14:50:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "map_epoch": 101, 2024-04-25T14:50:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "map_full": 0, 2024-04-25T14:50:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "map_inc": 13, 2024-04-25T14:50:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_sessions": 8, 2024-04-25T14:50:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_open": 8, 2024-04-25T14:50:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_close": 0, 2024-04-25T14:50:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_laggy": 0, 2024-04-25T14:50:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_wr": 17, 2024-04-25T14:50:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_rd": 0, 2024-04-25T14:50:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_del": 2 2024-04-25T14:50:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "oft": { 2024-04-25T14:50:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_objs": 1, 2024-04-25T14:50:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_kv_pairs": 0, 2024-04-25T14:50:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_updates": 0, 2024-04-25T14:50:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_removes": 0 2024-04-25T14:50:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "osd-slow-ops": { 2024-04-25T14:50:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_ops_count": 0 2024-04-25T14:50:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "purge_queue": { 2024-04-25T14:50:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops": 0, 2024-04-25T14:50:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T14:50:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing": 0, 2024-04-25T14:50:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_high_water": 0, 2024-04-25T14:50:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed_ops": 0, 2024-04-25T14:50:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed": 0, 2024-04-25T14:50:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_item_in_journal": 0 2024-04-25T14:50:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:50:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-04-25T14:50:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-04-25T14:50:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-04-25T14:50:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 716, 2024-04-25T14:50:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 1806324, 2024-04-25T14:50:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-04-25T14:50:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 716, 2024-04-25T14:50:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-04-25T14:50:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-04-25T14:50:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 716, 2024-04-25T14:50:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 1806324, 2024-04-25T14:50:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-04-25T14:50:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_bytes": { 2024-04-25T14:50:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-04-25T14:50:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-04-25T14:50:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-04-25T14:50:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-04-25T14:50:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-04-25T14:50:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-04-25T14:50:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-04-25T14:50:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 47, 2024-04-25T14:50:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 12980, 2024-04-25T14:50:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 47, 2024-04-25T14:50:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 12980, 2024-04-25T14:50:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-04-25T14:50:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_ops": { 2024-04-25T14:50:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-04-25T14:50:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 1024, 2024-04-25T14:50:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-04-25T14:50:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-04-25T14:50:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-04-25T14:50:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-04-25T14:50:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-04-25T14:50:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 47, 2024-04-25T14:50:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 47, 2024-04-25T14:50:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 47, 2024-04-25T14:50:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 47, 2024-04-25T14:50:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-04-25T14:50:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle": { 2024-04-25T14:50:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-04-25T14:50:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-04-25T14:50:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-04-25T14:50:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-04-25T14:50:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-04-25T14:50:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-04-25T14:50:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-04-25T14:50:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-04-25T14:50:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-04-25T14:50:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 0, 2024-04-25T14:50:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 0, 2024-04-25T14:50:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-04-25T14:50:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:50:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle-0x55a18ea2a500": { 2024-04-25T14:50:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-04-25T14:50:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-04-25T14:50:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-04-25T14:50:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 8, 2024-04-25T14:50:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 2353, 2024-04-25T14:50:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-04-25T14:50:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 8, 2024-04-25T14:50:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-04-25T14:50:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-04-25T14:50:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 7, 2024-04-25T14:50:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 2353, 2024-04-25T14:50:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-04-25T14:50:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:50:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:50:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:50:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:50:59.031 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-04-25T14:50:59.783 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:50:59 smithi053 ceph-mon[27773]: pgmap v320: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:50:59.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:50:59 smithi183 ceph-mon[32584]: pgmap v320: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:50:59.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:50:59 smithi150 ceph-mon[32504]: pgmap v320: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:51:00.081 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-04-25T14:51:00.081 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-04-25T14:51:00.081 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-04-25T14:51:00.081 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=6 2024-04-25T14:51:00.081 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-04-25T14:51:00.082 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-04-25T14:51:00.082 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T14:51:00.082 DEBUG:tasks.check_counter:Getting stats from j 2024-04-25T14:51:00.082 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-25T14:51:00.571 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:51:01.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:01 smithi183 ceph-mon[32584]: pgmap v321: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:51:01.958 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:01 smithi053 ceph-mon[27773]: pgmap v321: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:51:01.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:01 smithi150 ceph-mon[32504]: pgmap v321: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:51:02.956 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-04-25T14:51:02.956 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:51:02.956 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 228, 2024-04-25T14:51:02.957 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 216, 2024-04-25T14:51:02.957 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 1729856, 2024-04-25T14:51:02.957 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 133024, 2024-04-25T14:51:02.957 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 4, 2024-04-25T14:51:02.957 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 0, 2024-04-25T14:51:02.957 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.109917437, 2024-04-25T14:51:02.957 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.024531615, 2024-04-25T14:51:02.957 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.109061367, 2024-04-25T14:51:02.958 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.010582160, 2024-04-25T14:51:02.958 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:51:02.958 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 215, 2024-04-25T14:51:02.958 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.022430862, 2024-04-25T14:51:02.958 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000104329 2024-04-25T14:51:02.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:51:02.958 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:51:02.959 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:51:02.959 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:51:02.959 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:51:02.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:51:02.959 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 1729856, 2024-04-25T14:51:02.959 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 133024 2024-04-25T14:51:02.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:51:02.959 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:51:02.960 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 6, 2024-04-25T14:51:02.960 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 167, 2024-04-25T14:51:02.960 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 3312, 2024-04-25T14:51:02.960 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 185568, 2024-04-25T14:51:02.960 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 5, 2024-04-25T14:51:02.960 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 3, 2024-04-25T14:51:02.960 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.036051704, 2024-04-25T14:51:02.960 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.026302699, 2024-04-25T14:51:02.961 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.004050681, 2024-04-25T14:51:02.961 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T14:51:02.961 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:51:02.961 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 165, 2024-04-25T14:51:02.961 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.027319557, 2024-04-25T14:51:02.961 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000165573 2024-04-25T14:51:02.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:51:02.961 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:51:02.962 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:51:02.962 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:51:02.962 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:51:02.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:51:02.962 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-25T14:51:02.962 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 185568 2024-04-25T14:51:02.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:51:02.962 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:51:02.963 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 0, 2024-04-25T14:51:02.963 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 2, 2024-04-25T14:51:02.963 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 0, 2024-04-25T14:51:02.963 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 0, 2024-04-25T14:51:02.963 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 2, 2024-04-25T14:51:02.963 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 1, 2024-04-25T14:51:02.963 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.001180139, 2024-04-25T14:51:02.963 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T14:51:02.963 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.000931481, 2024-04-25T14:51:02.964 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T14:51:02.964 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:51:02.964 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:51:02.964 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:51:02.964 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:51:02.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:51:02.964 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:51:02.964 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:51:02.965 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:51:02.965 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:51:02.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:51:02.965 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T14:51:02.965 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T14:51:02.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:51:02.965 INFO:teuthology.orchestra.run.smithi053.stdout: "cct": { 2024-04-25T14:51:02.965 INFO:teuthology.orchestra.run.smithi053.stdout: "total_workers": 0, 2024-04-25T14:51:02.966 INFO:teuthology.orchestra.run.smithi053.stdout: "unhealthy_workers": 0 2024-04-25T14:51:02.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:51:02.966 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool": { 2024-04-25T14:51:02.966 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:51:02.966 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_items": 0, 2024-04-25T14:51:02.966 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:51:02.966 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:51:02.966 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:51:02.966 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:51:02.967 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:51:02.967 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:51:02.967 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:51:02.967 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:51:02.967 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:51:02.967 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:51:02.967 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:51:02.967 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:51:02.968 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:51:02.968 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_items": 0, 2024-04-25T14:51:02.968 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:51:02.968 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_items": 0, 2024-04-25T14:51:02.968 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:51:02.968 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:51:02.968 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:51:02.968 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:51:02.968 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:51:02.969 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:51:02.969 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:51:02.969 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_items": 0, 2024-04-25T14:51:02.969 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:51:02.969 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:51:02.969 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:51:02.969 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_items": 0, 2024-04-25T14:51:02.969 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_bytes": 0, 2024-04-25T14:51:02.969 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_items": 0, 2024-04-25T14:51:02.969 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:51:02.970 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:51:02.970 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:51:02.970 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:51:02.970 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_bytes": 149745, 2024-04-25T14:51:02.970 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_items": 31, 2024-04-25T14:51:02.970 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_bytes": 0, 2024-04-25T14:51:02.970 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_items": 0, 2024-04-25T14:51:02.970 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bytes": 0, 2024-04-25T14:51:02.971 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_items": 0, 2024-04-25T14:51:02.971 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:51:02.971 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_items": 0, 2024-04-25T14:51:02.971 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:51:02.971 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_items": 0, 2024-04-25T14:51:02.971 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_bytes": 0, 2024-04-25T14:51:02.971 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_items": 0, 2024-04-25T14:51:02.971 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:51:02.972 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:51:02.972 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_bytes": 0, 2024-04-25T14:51:02.972 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_items": 0, 2024-04-25T14:51:02.972 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_bytes": 928, 2024-04-25T14:51:02.973 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_items": 2, 2024-04-25T14:51:02.973 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_bytes": 0, 2024-04-25T14:51:02.973 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_items": 0, 2024-04-25T14:51:02.973 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_bytes": 0, 2024-04-25T14:51:02.973 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_items": 0 2024-04-25T14:51:02.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-25T14:51:02.973 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:51:02.973 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-04-25T14:51:02.973 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-04-25T14:51:02.974 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-04-25T14:51:02.974 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 230, 2024-04-25T14:51:02.974 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 1697716, 2024-04-25T14:51:02.974 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-04-25T14:51:02.974 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 230, 2024-04-25T14:51:02.974 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-04-25T14:51:02.974 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-04-25T14:51:02.974 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 230, 2024-04-25T14:51:02.974 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 1697716, 2024-04-25T14:51:02.975 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-04-25T14:51:02.975 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-25T14:51:02.975 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-25T14:51:02.975 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-25T14:51:02.975 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:51:02.975 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-25T14:51:02.975 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-04-25T14:51:03.685 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:03 smithi053 ceph-mon[27773]: pgmap v322: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T14:51:03.770 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-25T14:51:03.770 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-25T14:51:03.770 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-04-25T14:51:03.770 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-04-25T14:51:03.770 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-25T14:51:03.770 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-25T14:51:03.770 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-25T14:51:03.771 DEBUG:tasks.check_counter:Getting stats from b 2024-04-25T14:51:03.771 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-25T14:51:03.798 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:03 smithi150 ceph-mon[32504]: pgmap v322: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T14:51:03.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:03 smithi183 ceph-mon[32584]: pgmap v322: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T14:51:04.023 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:51:05.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:05 smithi150 ceph-mon[32504]: pgmap v323: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:51:05.733 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-04-25T14:51:05.733 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:51:05.733 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 1036, 2024-04-25T14:51:05.733 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 747, 2024-04-25T14:51:05.734 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 1966797, 2024-04-25T14:51:05.734 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 293651, 2024-04-25T14:51:05.734 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 9, 2024-04-25T14:51:05.734 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 3, 2024-04-25T14:51:05.734 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.129720983, 2024-04-25T14:51:05.734 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.038316070, 2024-04-25T14:51:05.734 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.068445665, 2024-04-25T14:51:05.734 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.029327315, 2024-04-25T14:51:05.735 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:51:05.735 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 750, 2024-04-25T14:51:05.735 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.224336463, 2024-04-25T14:51:05.735 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000299115 2024-04-25T14:51:05.735 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.735 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:51:05.735 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 901, 2024-04-25T14:51:05.735 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000257556, 2024-04-25T14:51:05.736 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000285 2024-04-25T14:51:05.736 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.736 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 1817024, 2024-04-25T14:51:05.736 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 140192 2024-04-25T14:51:05.736 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.736 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:51:05.736 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 784, 2024-04-25T14:51:05.736 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 676, 2024-04-25T14:51:05.736 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 401117, 2024-04-25T14:51:05.737 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 612870, 2024-04-25T14:51:05.737 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 11, 2024-04-25T14:51:05.737 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 5, 2024-04-25T14:51:05.737 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.097612541, 2024-04-25T14:51:05.737 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.028434739, 2024-04-25T14:51:05.737 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.034160055, 2024-04-25T14:51:05.737 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.030895990, 2024-04-25T14:51:05.737 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:51:05.737 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 677, 2024-04-25T14:51:05.738 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 1.270439231, 2024-04-25T14:51:05.738 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.001876571 2024-04-25T14:51:05.738 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.738 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:51:05.738 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 902, 2024-04-25T14:51:05.738 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000233083, 2024-04-25T14:51:05.738 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000258 2024-04-25T14:51:05.738 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.739 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 43456, 2024-04-25T14:51:05.739 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 455904 2024-04-25T14:51:05.739 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.739 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:51:05.739 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 1845, 2024-04-25T14:51:05.739 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 611, 2024-04-25T14:51:05.739 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 729258, 2024-04-25T14:51:05.739 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 422866, 2024-04-25T14:51:05.740 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 18, 2024-04-25T14:51:05.740 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 9, 2024-04-25T14:51:05.740 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.232164877, 2024-04-25T14:51:05.740 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.049548037, 2024-04-25T14:51:05.740 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.086654976, 2024-04-25T14:51:05.740 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.094033450, 2024-04-25T14:51:05.740 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:51:05.741 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 602, 2024-04-25T14:51:05.741 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.562930227, 2024-04-25T14:51:05.741 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000935100 2024-04-25T14:51:05.741 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.741 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:51:05.741 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 1747, 2024-04-25T14:51:05.741 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000336343, 2024-04-25T14:51:05.742 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000192 2024-04-25T14:51:05.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.742 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 377984, 2024-04-25T14:51:05.742 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 127088 2024-04-25T14:51:05.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.742 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-04-25T14:51:05.742 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 1, 2024-04-25T14:51:05.742 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-04-25T14:51:05.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.743 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-MDSRank": { 2024-04-25T14:51:05.743 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-04-25T14:51:05.743 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-04-25T14:51:05.743 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 261, 2024-04-25T14:51:05.743 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.037168943, 2024-04-25T14:51:05.743 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000142409 2024-04-25T14:51:05.743 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T14:51:05.743 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.744 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-PurgeQueue": { 2024-04-25T14:51:05.744 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-04-25T14:51:05.744 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-04-25T14:51:05.744 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2, 2024-04-25T14:51:05.744 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000005957, 2024-04-25T14:51:05.744 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000002978 2024-04-25T14:51:05.744 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T14:51:05.744 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.744 INFO:teuthology.orchestra.run.smithi150.stdout: "mds": { 2024-04-25T14:51:05.745 INFO:teuthology.orchestra.run.smithi150.stdout: "request": 159, 2024-04-25T14:51:05.745 INFO:teuthology.orchestra.run.smithi150.stdout: "reply": 159, 2024-04-25T14:51:05.745 INFO:teuthology.orchestra.run.smithi150.stdout: "reply_latency": { 2024-04-25T14:51:05.745 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 159, 2024-04-25T14:51:05.745 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.145988040, 2024-04-25T14:51:05.745 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000918163 2024-04-25T14:51:05.745 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.745 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_reply": 0, 2024-04-25T14:51:05.745 INFO:teuthology.orchestra.run.smithi150.stdout: "forward": 0, 2024-04-25T14:51:05.746 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_complete": 0, 2024-04-25T14:51:05.746 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_keys": 0, 2024-04-25T14:51:05.746 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_commit": 19, 2024-04-25T14:51:05.746 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_split": 0, 2024-04-25T14:51:05.746 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_merge": 0, 2024-04-25T14:51:05.746 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes": 25, 2024-04-25T14:51:05.746 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_top": 2, 2024-04-25T14:51:05.746 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_bottom": 23, 2024-04-25T14:51:05.746 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pin_tail": 0, 2024-04-25T14:51:05.747 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pinned": 15, 2024-04-25T14:51:05.747 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_expired": 0, 2024-04-25T14:51:05.747 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_with_caps": 2, 2024-04-25T14:51:05.747 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": 2, 2024-04-25T14:51:05.747 INFO:teuthology.orchestra.run.smithi150.stdout: "subtrees": 2, 2024-04-25T14:51:05.747 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse": 198, 2024-04-25T14:51:05.747 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_hit": 189, 2024-04-25T14:51:05.747 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_forward": 0, 2024-04-25T14:51:05.748 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_discover": 0, 2024-04-25T14:51:05.748 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_dir_fetch": 0, 2024-04-25T14:51:05.748 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_remote_ino": 0, 2024-04-25T14:51:05.748 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_lock": 0, 2024-04-25T14:51:05.748 INFO:teuthology.orchestra.run.smithi150.stdout: "load_cent": 0, 2024-04-25T14:51:05.748 INFO:teuthology.orchestra.run.smithi150.stdout: "q": 0, 2024-04-25T14:51:05.748 INFO:teuthology.orchestra.run.smithi150.stdout: "exported": 0, 2024-04-25T14:51:05.748 INFO:teuthology.orchestra.run.smithi150.stdout: "exported_inodes": 0, 2024-04-25T14:51:05.748 INFO:teuthology.orchestra.run.smithi150.stdout: "imported": 0, 2024-04-25T14:51:05.749 INFO:teuthology.orchestra.run.smithi150.stdout: "imported_inodes": 0, 2024-04-25T14:51:05.749 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_dir_fetch": 0, 2024-04-25T14:51:05.749 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_backtrace_fetch": 0, 2024-04-25T14:51:05.749 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_peer_discover": 0, 2024-04-25T14:51:05.749 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rfiles": 4, 2024-04-25T14:51:05.749 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rbytes": 234, 2024-04-25T14:51:05.749 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rsnaps": 6, 2024-04-25T14:51:05.749 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T14:51:05.749 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_set_tag": 0, 2024-04-25T14:51:05.750 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T14:51:05.750 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_inotable_repaired": 0, 2024-04-25T14:51:05.750 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_inodes": 0, 2024-04-25T14:51:05.750 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T14:51:05.750 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T14:51:05.750 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_file_inodes": 0, 2024-04-25T14:51:05.750 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_inode_file_caps": 0, 2024-04-25T14:51:05.750 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_revoke": 8, 2024-04-25T14:51:05.751 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_grant": 5, 2024-04-25T14:51:05.751 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T14:51:05.751 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T14:51:05.751 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T14:51:05.751 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps": 14, 2024-04-25T14:51:05.751 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps_dirty": 2, 2024-04-25T14:51:05.751 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_cap_release": 0, 2024-04-25T14:51:05.751 INFO:teuthology.orchestra.run.smithi150.stdout: "process_request_cap_release": 16 2024-04-25T14:51:05.751 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.752 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_cache": { 2024-04-25T14:51:05.752 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update": 5, 2024-04-25T14:51:05.752 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update_receipt": 0, 2024-04-25T14:51:05.752 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_try_discover": 0, 2024-04-25T14:51:05.752 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_send_discover": 0, 2024-04-25T14:51:05.752 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_handle_discover": 13, 2024-04-25T14:51:05.752 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays": 0, 2024-04-25T14:51:05.752 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_delayed": 0, 2024-04-25T14:51:05.752 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_enqueuing": 0, 2024-04-25T14:51:05.753 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_created": 0, 2024-04-25T14:51:05.753 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_enqueued": 0, 2024-04-25T14:51:05.753 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_reintegrated": 0, 2024-04-25T14:51:05.753 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_migrated": 0, 2024-04-25T14:51:05.753 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_processing": 0, 2024-04-25T14:51:05.753 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_enqueued": 0, 2024-04-25T14:51:05.753 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_prioritized": 0, 2024-04-25T14:51:05.753 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_started": 0, 2024-04-25T14:51:05.754 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_completed": 0, 2024-04-25T14:51:05.754 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_quiesce_path": 0, 2024-04-25T14:51:05.754 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_quiesce_inode": 0, 2024-04-25T14:51:05.754 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T14:51:05.754 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_exportdir": 0, 2024-04-25T14:51:05.754 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_flush": 0, 2024-04-25T14:51:05.754 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragmentdir": 0, 2024-04-25T14:51:05.754 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragstats": 0, 2024-04-25T14:51:05.754 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_inodestats": 0 2024-04-25T14:51:05.755 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.755 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_log": { 2024-04-25T14:51:05.755 INFO:teuthology.orchestra.run.smithi150.stdout: "evlrg": 0, 2024-04-25T14:51:05.755 INFO:teuthology.orchestra.run.smithi150.stdout: "evadd": 109, 2024-04-25T14:51:05.755 INFO:teuthology.orchestra.run.smithi150.stdout: "evex": 37, 2024-04-25T14:51:05.755 INFO:teuthology.orchestra.run.smithi150.stdout: "evtrm": 37, 2024-04-25T14:51:05.755 INFO:teuthology.orchestra.run.smithi150.stdout: "ev": 72, 2024-04-25T14:51:05.755 INFO:teuthology.orchestra.run.smithi150.stdout: "evexg": 0, 2024-04-25T14:51:05.755 INFO:teuthology.orchestra.run.smithi150.stdout: "evexd": 36, 2024-04-25T14:51:05.756 INFO:teuthology.orchestra.run.smithi150.stdout: "segadd": 3, 2024-04-25T14:51:05.756 INFO:teuthology.orchestra.run.smithi150.stdout: "segex": 2, 2024-04-25T14:51:05.756 INFO:teuthology.orchestra.run.smithi150.stdout: "segtrm": 2, 2024-04-25T14:51:05.756 INFO:teuthology.orchestra.run.smithi150.stdout: "seg": 1, 2024-04-25T14:51:05.756 INFO:teuthology.orchestra.run.smithi150.stdout: "segmjr": 1, 2024-04-25T14:51:05.756 INFO:teuthology.orchestra.run.smithi150.stdout: "segexg": 0, 2024-04-25T14:51:05.756 INFO:teuthology.orchestra.run.smithi150.stdout: "segexd": 1, 2024-04-25T14:51:05.756 INFO:teuthology.orchestra.run.smithi150.stdout: "expos": 4262242, 2024-04-25T14:51:05.757 INFO:teuthology.orchestra.run.smithi150.stdout: "wrpos": 4288936, 2024-04-25T14:51:05.757 INFO:teuthology.orchestra.run.smithi150.stdout: "rdpos": 0, 2024-04-25T14:51:05.757 INFO:teuthology.orchestra.run.smithi150.stdout: "jlat": { 2024-04-25T14:51:05.757 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 88, 2024-04-25T14:51:05.757 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.162412976, 2024-04-25T14:51:05.757 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.001845602 2024-04-25T14:51:05.757 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.757 INFO:teuthology.orchestra.run.smithi150.stdout: "replayed": 0 2024-04-25T14:51:05.757 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.758 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_mem": { 2024-04-25T14:51:05.758 INFO:teuthology.orchestra.run.smithi150.stdout: "ino": 23, 2024-04-25T14:51:05.758 INFO:teuthology.orchestra.run.smithi150.stdout: "ino+": 23, 2024-04-25T14:51:05.758 INFO:teuthology.orchestra.run.smithi150.stdout: "ino-": 0, 2024-04-25T14:51:05.758 INFO:teuthology.orchestra.run.smithi150.stdout: "dir": 18, 2024-04-25T14:51:05.758 INFO:teuthology.orchestra.run.smithi150.stdout: "dir+": 18, 2024-04-25T14:51:05.758 INFO:teuthology.orchestra.run.smithi150.stdout: "dir-": 0, 2024-04-25T14:51:05.758 INFO:teuthology.orchestra.run.smithi150.stdout: "dn": 25, 2024-04-25T14:51:05.758 INFO:teuthology.orchestra.run.smithi150.stdout: "dn+": 25, 2024-04-25T14:51:05.759 INFO:teuthology.orchestra.run.smithi150.stdout: "dn-": 0, 2024-04-25T14:51:05.759 INFO:teuthology.orchestra.run.smithi150.stdout: "cap": 2, 2024-04-25T14:51:05.759 INFO:teuthology.orchestra.run.smithi150.stdout: "cap+": 15, 2024-04-25T14:51:05.759 INFO:teuthology.orchestra.run.smithi150.stdout: "cap-": 13, 2024-04-25T14:51:05.759 INFO:teuthology.orchestra.run.smithi150.stdout: "rss": 54208, 2024-04-25T14:51:05.759 INFO:teuthology.orchestra.run.smithi150.stdout: "heap": 231800 2024-04-25T14:51:05.759 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.759 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_server": { 2024-04-25T14:51:05.760 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_client_request": 203, 2024-04-25T14:51:05.760 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_server_request": 1, 2024-04-25T14:51:05.760 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_request": 159, 2024-04-25T14:51:05.760 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_session": 32, 2024-04-25T14:51:05.760 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_peer_request": 2, 2024-04-25T14:51:05.760 INFO:teuthology.orchestra.run.smithi150.stdout: "req_create_latency": { 2024-04-25T14:51:05.760 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 4, 2024-04-25T14:51:05.760 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.003063053, 2024-04-25T14:51:05.760 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000765763 2024-04-25T14:51:05.761 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.761 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getattr_latency": { 2024-04-25T14:51:05.761 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 109, 2024-04-25T14:51:05.761 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.025924186, 2024-04-25T14:51:05.761 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000237836 2024-04-25T14:51:05.761 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.761 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getfilelock_latency": { 2024-04-25T14:51:05.761 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:05.761 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:05.762 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:05.762 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.762 INFO:teuthology.orchestra.run.smithi150.stdout: "req_link_latency": { 2024-04-25T14:51:05.762 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:05.762 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:05.762 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:05.762 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.762 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookup_latency": { 2024-04-25T14:51:05.763 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2, 2024-04-25T14:51:05.763 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000417153, 2024-04-25T14:51:05.763 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000208576 2024-04-25T14:51:05.763 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.763 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookuphash_latency": { 2024-04-25T14:51:05.763 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:05.763 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:05.763 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:05.763 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.764 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupino_latency": { 2024-04-25T14:51:05.764 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:05.764 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:05.764 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:05.764 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.764 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupname_latency": { 2024-04-25T14:51:05.764 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:05.764 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:05.765 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:05.765 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.765 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupparent_latency": { 2024-04-25T14:51:05.765 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:05.765 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:05.765 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:05.765 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.765 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupsnap_latency": { 2024-04-25T14:51:05.765 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:05.766 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:05.766 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:05.766 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.766 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lssnap_latency": { 2024-04-25T14:51:05.766 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 11, 2024-04-25T14:51:05.766 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.002258781, 2024-04-25T14:51:05.766 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000205343 2024-04-25T14:51:05.766 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.767 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mkdir_latency": { 2024-04-25T14:51:05.767 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 6, 2024-04-25T14:51:05.767 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.007532089, 2024-04-25T14:51:05.767 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.001255348 2024-04-25T14:51:05.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.768 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mknod_latency": { 2024-04-25T14:51:05.768 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:05.768 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:05.768 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:05.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.768 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mksnap_latency": { 2024-04-25T14:51:05.768 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 11, 2024-04-25T14:51:05.768 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.058447642, 2024-04-25T14:51:05.769 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.005313422 2024-04-25T14:51:05.769 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.769 INFO:teuthology.orchestra.run.smithi150.stdout: "req_open_latency": { 2024-04-25T14:51:05.769 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 1, 2024-04-25T14:51:05.769 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000188708, 2024-04-25T14:51:05.769 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000188708 2024-04-25T14:51:05.769 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.769 INFO:teuthology.orchestra.run.smithi150.stdout: "req_readdir_latency": { 2024-04-25T14:51:05.769 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:05.770 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:05.770 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:05.770 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.770 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rename_latency": { 2024-04-25T14:51:05.770 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2, 2024-04-25T14:51:05.770 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.003293894, 2024-04-25T14:51:05.770 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.001646947 2024-04-25T14:51:05.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.771 INFO:teuthology.orchestra.run.smithi150.stdout: "req_renamesnap_latency": { 2024-04-25T14:51:05.771 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:05.771 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:05.771 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:05.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.771 INFO:teuthology.orchestra.run.smithi150.stdout: "req_snapdiff_latency": { 2024-04-25T14:51:05.771 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:05.771 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:05.772 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:05.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.772 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmdir_latency": { 2024-04-25T14:51:05.772 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:05.772 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:05.772 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:05.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.772 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmsnap_latency": { 2024-04-25T14:51:05.772 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 5, 2024-04-25T14:51:05.773 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.024691673, 2024-04-25T14:51:05.773 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.004938334 2024-04-25T14:51:05.773 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.773 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmxattr_latency": { 2024-04-25T14:51:05.773 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:05.773 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:05.773 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:05.773 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.774 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setattr_latency": { 2024-04-25T14:51:05.774 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 1, 2024-04-25T14:51:05.774 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.001062451, 2024-04-25T14:51:05.774 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.001062451 2024-04-25T14:51:05.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.774 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setdirlayout_latency": { 2024-04-25T14:51:05.774 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:05.774 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:05.774 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:05.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.775 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setfilelock_latency": { 2024-04-25T14:51:05.775 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:05.775 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:05.775 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:05.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.775 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setlayout_latency": { 2024-04-25T14:51:05.775 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:05.776 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:05.776 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:05.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.776 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setxattr_latency": { 2024-04-25T14:51:05.776 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 7, 2024-04-25T14:51:05.776 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.019108410, 2024-04-25T14:51:05.776 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.002729772 2024-04-25T14:51:05.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.776 INFO:teuthology.orchestra.run.smithi150.stdout: "req_symlink_latency": { 2024-04-25T14:51:05.777 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:05.777 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:05.777 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:05.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.777 INFO:teuthology.orchestra.run.smithi150.stdout: "req_unlink_latency": { 2024-04-25T14:51:05.777 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:05.777 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:05.777 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:05.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.778 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_revoke_eviction": 0, 2024-04-25T14:51:05.778 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_acquisition_throttle": 0, 2024-04-25T14:51:05.778 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getvxattr_latency": { 2024-04-25T14:51:05.778 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:05.778 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:05.778 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:05.778 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T14:51:05.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.779 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_sessions": { 2024-04-25T14:51:05.779 INFO:teuthology.orchestra.run.smithi150.stdout: "session_count": 2, 2024-04-25T14:51:05.779 INFO:teuthology.orchestra.run.smithi150.stdout: "session_add": 5, 2024-04-25T14:51:05.779 INFO:teuthology.orchestra.run.smithi150.stdout: "session_remove": 3, 2024-04-25T14:51:05.779 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_open": 2, 2024-04-25T14:51:05.779 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_stale": 0, 2024-04-25T14:51:05.779 INFO:teuthology.orchestra.run.smithi150.stdout: "total_load": 27, 2024-04-25T14:51:05.779 INFO:teuthology.orchestra.run.smithi150.stdout: "average_load": 13, 2024-04-25T14:51:05.779 INFO:teuthology.orchestra.run.smithi150.stdout: "avg_session_uptime": 615, 2024-04-25T14:51:05.780 INFO:teuthology.orchestra.run.smithi150.stdout: "mdthresh_evicted": 0 2024-04-25T14:51:05.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.780 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-04-25T14:51:05.780 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:51:05.780 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-04-25T14:51:05.780 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:51:05.780 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:51:05.780 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:51:05.780 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:51:05.781 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:51:05.781 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:51:05.781 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:51:05.781 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:51:05.781 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:51:05.781 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:51:05.781 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:51:05.781 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:51:05.782 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:51:05.782 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-04-25T14:51:05.782 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:51:05.782 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-04-25T14:51:05.782 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:51:05.782 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:51:05.782 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:51:05.782 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:51:05.783 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:51:05.783 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:51:05.783 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:51:05.783 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-04-25T14:51:05.783 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:51:05.783 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:51:05.783 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:51:05.783 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-04-25T14:51:05.783 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-04-25T14:51:05.784 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-04-25T14:51:05.784 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:51:05.784 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:51:05.784 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:51:05.784 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:51:05.784 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 358655, 2024-04-25T14:51:05.784 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 140, 2024-04-25T14:51:05.784 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 792, 2024-04-25T14:51:05.784 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 9, 2024-04-25T14:51:05.785 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-04-25T14:51:05.785 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-04-25T14:51:05.785 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:51:05.785 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-04-25T14:51:05.785 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:51:05.785 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-04-25T14:51:05.785 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 8240, 2024-04-25T14:51:05.785 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 173, 2024-04-25T14:51:05.785 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:51:05.786 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:51:05.786 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-04-25T14:51:05.786 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-04-25T14:51:05.786 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 95170, 2024-04-25T14:51:05.786 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 586, 2024-04-25T14:51:05.786 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-04-25T14:51:05.786 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-04-25T14:51:05.786 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-04-25T14:51:05.787 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-04-25T14:51:05.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.787 INFO:teuthology.orchestra.run.smithi150.stdout: "objecter": { 2024-04-25T14:51:05.787 INFO:teuthology.orchestra.run.smithi150.stdout: "op_active": 0, 2024-04-25T14:51:05.787 INFO:teuthology.orchestra.run.smithi150.stdout: "op_laggy": 0, 2024-04-25T14:51:05.787 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send": 676, 2024-04-25T14:51:05.787 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send_bytes": 120126, 2024-04-25T14:51:05.787 INFO:teuthology.orchestra.run.smithi150.stdout: "op_resend": 32, 2024-04-25T14:51:05.787 INFO:teuthology.orchestra.run.smithi150.stdout: "op_reply": 676, 2024-04-25T14:51:05.788 INFO:teuthology.orchestra.run.smithi150.stdout: "op_latency": { 2024-04-25T14:51:05.788 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 676, 2024-04-25T14:51:05.788 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 22.177566014, 2024-04-25T14:51:05.788 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.032807050 2024-04-25T14:51:05.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.788 INFO:teuthology.orchestra.run.smithi150.stdout: "op_inflight": 0, 2024-04-25T14:51:05.788 INFO:teuthology.orchestra.run.smithi150.stdout: "oplen_avg": { 2024-04-25T14:51:05.788 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 676, 2024-04-25T14:51:05.789 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 743 2024-04-25T14:51:05.789 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.789 INFO:teuthology.orchestra.run.smithi150.stdout: "op": 676, 2024-04-25T14:51:05.789 INFO:teuthology.orchestra.run.smithi150.stdout: "op_r": 0, 2024-04-25T14:51:05.789 INFO:teuthology.orchestra.run.smithi150.stdout: "op_w": 676, 2024-04-25T14:51:05.789 INFO:teuthology.orchestra.run.smithi150.stdout: "op_rmw": 0, 2024-04-25T14:51:05.789 INFO:teuthology.orchestra.run.smithi150.stdout: "op_pg": 0, 2024-04-25T14:51:05.789 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_stat": 1, 2024-04-25T14:51:05.789 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_create": 22, 2024-04-25T14:51:05.790 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_read": 0, 2024-04-25T14:51:05.790 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_write": 88, 2024-04-25T14:51:05.790 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writefull": 26, 2024-04-25T14:51:05.790 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writesame": 0, 2024-04-25T14:51:05.790 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_append": 0, 2024-04-25T14:51:05.790 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_zero": 0, 2024-04-25T14:51:05.790 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_truncate": 0, 2024-04-25T14:51:05.790 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_delete": 506, 2024-04-25T14:51:05.791 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_mapext": 0, 2024-04-25T14:51:05.791 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_sparse_read": 0, 2024-04-25T14:51:05.791 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_clonerange": 0, 2024-04-25T14:51:05.791 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_getxattr": 0, 2024-04-25T14:51:05.791 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_setxattr": 44, 2024-04-25T14:51:05.791 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_cmpxattr": 0, 2024-04-25T14:51:05.791 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_rmxattr": 0, 2024-04-25T14:51:05.791 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_resetxattrs": 0, 2024-04-25T14:51:05.791 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_call": 0, 2024-04-25T14:51:05.792 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_watch": 0, 2024-04-25T14:51:05.792 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_notify": 0, 2024-04-25T14:51:05.792 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T14:51:05.792 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls": 0, 2024-04-25T14:51:05.792 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls_filter": 0, 2024-04-25T14:51:05.792 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_other": 0, 2024-04-25T14:51:05.792 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_active": 0, 2024-04-25T14:51:05.792 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_send": 0, 2024-04-25T14:51:05.792 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_resend": 0, 2024-04-25T14:51:05.793 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_ping": 0, 2024-04-25T14:51:05.793 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_active": 0, 2024-04-25T14:51:05.793 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_send": 0, 2024-04-25T14:51:05.793 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_resend": 0, 2024-04-25T14:51:05.793 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_active": 0, 2024-04-25T14:51:05.793 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_send": 0, 2024-04-25T14:51:05.793 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_resend": 0, 2024-04-25T14:51:05.793 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_active": 0, 2024-04-25T14:51:05.794 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_send": 0, 2024-04-25T14:51:05.794 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_resend": 0, 2024-04-25T14:51:05.794 INFO:teuthology.orchestra.run.smithi150.stdout: "command_active": 0, 2024-04-25T14:51:05.794 INFO:teuthology.orchestra.run.smithi150.stdout: "command_send": 0, 2024-04-25T14:51:05.794 INFO:teuthology.orchestra.run.smithi150.stdout: "command_resend": 0, 2024-04-25T14:51:05.794 INFO:teuthology.orchestra.run.smithi150.stdout: "map_epoch": 101, 2024-04-25T14:51:05.794 INFO:teuthology.orchestra.run.smithi150.stdout: "map_full": 0, 2024-04-25T14:51:05.794 INFO:teuthology.orchestra.run.smithi150.stdout: "map_inc": 14, 2024-04-25T14:51:05.794 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_sessions": 12, 2024-04-25T14:51:05.795 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_open": 12, 2024-04-25T14:51:05.795 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_close": 0, 2024-04-25T14:51:05.795 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_laggy": 0, 2024-04-25T14:51:05.795 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_wr": 48, 2024-04-25T14:51:05.795 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_rd": 0, 2024-04-25T14:51:05.795 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_del": 8 2024-04-25T14:51:05.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.795 INFO:teuthology.orchestra.run.smithi150.stdout: "oft": { 2024-04-25T14:51:05.796 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_objs": 1, 2024-04-25T14:51:05.796 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_kv_pairs": 0, 2024-04-25T14:51:05.796 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_updates": 0, 2024-04-25T14:51:05.796 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_removes": 0 2024-04-25T14:51:05.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.796 INFO:teuthology.orchestra.run.smithi150.stdout: "osd-slow-ops": { 2024-04-25T14:51:05.796 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_ops_count": 0 2024-04-25T14:51:05.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.796 INFO:teuthology.orchestra.run.smithi150.stdout: "purge_queue": { 2024-04-25T14:51:05.797 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops": 0, 2024-04-25T14:51:05.797 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T14:51:05.797 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing": 0, 2024-04-25T14:51:05.797 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_high_water": 0, 2024-04-25T14:51:05.797 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed_ops": 0, 2024-04-25T14:51:05.797 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed": 0, 2024-04-25T14:51:05.797 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_item_in_journal": 0 2024-04-25T14:51:05.797 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.797 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:51:05.798 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-25T14:51:05.798 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-04-25T14:51:05.798 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-25T14:51:05.798 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 3661, 2024-04-25T14:51:05.798 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 2701882, 2024-04-25T14:51:05.798 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-25T14:51:05.798 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 3661, 2024-04-25T14:51:05.798 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-04-25T14:51:05.799 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-04-25T14:51:05.799 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 3661, 2024-04-25T14:51:05.799 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 2701882, 2024-04-25T14:51:05.799 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-25T14:51:05.799 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:05.799 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:05.799 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:05.799 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T14:51:05.799 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.800 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_bytes": { 2024-04-25T14:51:05.800 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-25T14:51:05.800 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-04-25T14:51:05.800 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-25T14:51:05.800 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-04-25T14:51:05.800 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-04-25T14:51:05.800 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-25T14:51:05.800 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-04-25T14:51:05.800 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 676, 2024-04-25T14:51:05.801 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 120126, 2024-04-25T14:51:05.801 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 676, 2024-04-25T14:51:05.801 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 120126, 2024-04-25T14:51:05.801 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-25T14:51:05.801 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:05.801 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:05.801 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:05.801 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T14:51:05.801 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.802 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_ops": { 2024-04-25T14:51:05.802 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-25T14:51:05.802 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 1024, 2024-04-25T14:51:05.802 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-25T14:51:05.802 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-04-25T14:51:05.802 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-04-25T14:51:05.802 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-25T14:51:05.802 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-04-25T14:51:05.803 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 676, 2024-04-25T14:51:05.803 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 676, 2024-04-25T14:51:05.803 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 676, 2024-04-25T14:51:05.803 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 676, 2024-04-25T14:51:05.803 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-25T14:51:05.803 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:05.803 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:05.803 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:05.803 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T14:51:05.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.804 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle": { 2024-04-25T14:51:05.804 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-25T14:51:05.804 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-04-25T14:51:05.804 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-25T14:51:05.804 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-04-25T14:51:05.804 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-04-25T14:51:05.804 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-25T14:51:05.805 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-04-25T14:51:05.805 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-04-25T14:51:05.805 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-04-25T14:51:05.805 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 0, 2024-04-25T14:51:05.805 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 0, 2024-04-25T14:51:05.805 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-25T14:51:05.805 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:05.805 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:05.805 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:05.806 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T14:51:05.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:05.806 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle-0x564e2ea3a500": { 2024-04-25T14:51:05.806 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 60, 2024-04-25T14:51:05.806 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-04-25T14:51:05.806 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-25T14:51:05.806 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 109, 2024-04-25T14:51:05.806 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 94632, 2024-04-25T14:51:05.807 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-25T14:51:05.807 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 109, 2024-04-25T14:51:05.807 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-04-25T14:51:05.807 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-04-25T14:51:05.807 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 88, 2024-04-25T14:51:05.807 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 94572, 2024-04-25T14:51:05.807 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-25T14:51:05.807 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:05.807 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:05.808 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:05.808 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T14:51:05.808 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T14:51:05.808 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-04-25T14:51:05.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:05 smithi183 ceph-mon[32584]: pgmap v323: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:51:06.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:05 smithi053 ceph-mon[27773]: pgmap v323: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:51:06.375 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-04-25T14:51:06.375 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-04-25T14:51:06.375 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=5 2024-04-25T14:51:06.376 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-04-25T14:51:06.376 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-25T14:51:06.376 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2024-04-25T14:51:06.376 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=5 2024-04-25T14:51:06.376 DEBUG:tasks.check_counter:Getting stats from e 2024-04-25T14:51:06.376 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-25T14:51:06.686 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:51:06.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:06 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:51:06.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:06 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:51:07.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:06 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:51:07.857 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:07 smithi150 ceph-mon[32504]: pgmap v324: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T14:51:07.857 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:07 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:51:07.857 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:07 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:51:07.857 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:07 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:51:07.857 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:07 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:51:07.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:07 smithi183 ceph-mon[32584]: pgmap v324: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T14:51:07.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:07 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:51:07.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:07 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:51:07.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:07 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:51:07.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:07 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:51:08.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:07 smithi053 ceph-mon[27773]: pgmap v324: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T14:51:08.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:07 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:51:08.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:07 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:51:08.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:07 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:51:08.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:07 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:51:08.398 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-04-25T14:51:08.398 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:51:08.398 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 232, 2024-04-25T14:51:08.399 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 218, 2024-04-25T14:51:08.399 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 1732848, 2024-04-25T14:51:08.399 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 133472, 2024-04-25T14:51:08.399 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 4, 2024-04-25T14:51:08.399 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 0, 2024-04-25T14:51:08.399 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.047585867, 2024-04-25T14:51:08.399 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.014159185, 2024-04-25T14:51:08.399 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.036871869, 2024-04-25T14:51:08.399 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.005394418, 2024-04-25T14:51:08.400 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:51:08.400 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 218, 2024-04-25T14:51:08.400 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.013887705, 2024-04-25T14:51:08.400 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000063705 2024-04-25T14:51:08.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:08.400 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:51:08.400 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:08.400 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:08.400 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:08.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:08.401 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 1732848, 2024-04-25T14:51:08.401 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 133472 2024-04-25T14:51:08.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:08.401 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:51:08.401 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 3, 2024-04-25T14:51:08.401 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 167, 2024-04-25T14:51:08.401 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 480, 2024-04-25T14:51:08.401 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 263600, 2024-04-25T14:51:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 6, 2024-04-25T14:51:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 3, 2024-04-25T14:51:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.011500379, 2024-04-25T14:51:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.007511097, 2024-04-25T14:51:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.001878580, 2024-04-25T14:51:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T14:51:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:51:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 163, 2024-04-25T14:51:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.027096167, 2024-04-25T14:51:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000166234 2024-04-25T14:51:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:51:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-25T14:51:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 263600 2024-04-25T14:51:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:51:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 0, 2024-04-25T14:51:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 2, 2024-04-25T14:51:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 0, 2024-04-25T14:51:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 0, 2024-04-25T14:51:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 2, 2024-04-25T14:51:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 1, 2024-04-25T14:51:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.000235155, 2024-04-25T14:51:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T14:51:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.000179948, 2024-04-25T14:51:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T14:51:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:51:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:51:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T14:51:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T14:51:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-04-25T14:51:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 0, 2024-04-25T14:51:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-04-25T14:51:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-04-25T14:51:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:51:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-04-25T14:51:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:51:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:51:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:51:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:51:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:51:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:51:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:51:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:51:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:51:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:51:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:51:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:51:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:51:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-04-25T14:51:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:51:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-04-25T14:51:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:51:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:51:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:51:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:51:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:51:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:51:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:51:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-04-25T14:51:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:51:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:51:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:51:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-04-25T14:51:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-04-25T14:51:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-04-25T14:51:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:51:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:51:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:51:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:51:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 148955, 2024-04-25T14:51:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 30, 2024-04-25T14:51:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 0, 2024-04-25T14:51:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 0, 2024-04-25T14:51:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-04-25T14:51:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-04-25T14:51:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:51:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-04-25T14:51:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:51:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-04-25T14:51:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 0, 2024-04-25T14:51:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 0, 2024-04-25T14:51:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:51:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:51:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-04-25T14:51:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-04-25T14:51:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 928, 2024-04-25T14:51:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 2, 2024-04-25T14:51:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-04-25T14:51:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-04-25T14:51:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-04-25T14:51:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-04-25T14:51:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:51:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-25T14:51:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-04-25T14:51:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-25T14:51:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 231, 2024-04-25T14:51:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 1697743, 2024-04-25T14:51:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-25T14:51:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 231, 2024-04-25T14:51:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-04-25T14:51:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-04-25T14:51:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 231, 2024-04-25T14:51:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 1697743, 2024-04-25T14:51:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-25T14:51:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T14:51:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T14:51:08.416 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-04-25T14:51:09.107 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2024-04-25T14:51:09.107 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2024-04-25T14:51:09.107 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.e 2024-04-25T14:51:09.108 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.e 2024-04-25T14:51:09.108 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2024-04-25T14:51:09.108 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2024-04-25T14:51:09.108 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2024-04-25T14:51:09.108 DEBUG:tasks.check_counter:Getting stats from h 2024-04-25T14:51:09.108 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-25T14:51:09.445 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:09 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:51:09.445 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:09 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:51:09.445 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:09 smithi150 ceph-mon[32504]: pgmap v325: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:51:09.445 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:09 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:09.445 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:09 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:09.446 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:09 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:09.446 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:09 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:09.446 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:09 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:09.446 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:09 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:09.446 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:09 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:09.446 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:09 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:09.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:09 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:51:09.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:09 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:51:09.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:09 smithi183 ceph-mon[32584]: pgmap v325: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:51:09.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:09 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:09.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:09 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:09.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:09 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:09.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:09 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:09.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:09 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:09.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:09 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:09.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:09 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:09.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:09 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:09.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:09 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:51:09.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:09 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:51:09.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:09 smithi053 ceph-mon[27773]: pgmap v325: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:51:09.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:09 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:09.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:09 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:09.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:09 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:09.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:09 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:09.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:09 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:09.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:09 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:09.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:09 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:09.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:09 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:09.912 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:51:10.294 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:10 smithi150 ceph-mon[32504]: Adjusting osd_memory_target on smithi183 to 176.2M 2024-04-25T14:51:10.295 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:10 smithi150 ceph-mon[32504]: Unable to set osd_memory_target on smithi183 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T14:51:10.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:10 smithi183 ceph-mon[32584]: Adjusting osd_memory_target on smithi183 to 176.2M 2024-04-25T14:51:10.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:10 smithi183 ceph-mon[32584]: Unable to set osd_memory_target on smithi183 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T14:51:10.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:10 smithi053 ceph-mon[27773]: Adjusting osd_memory_target on smithi183 to 176.2M 2024-04-25T14:51:10.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:10 smithi053 ceph-mon[27773]: Unable to set osd_memory_target on smithi183 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T14:51:11.325 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:11 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:51:11.326 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:11 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:51:11.326 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:11 smithi150 ceph-mon[32504]: pgmap v326: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:51:11.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:11 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:51:11.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:11 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:51:11.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:11 smithi183 ceph-mon[32584]: pgmap v326: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:51:11.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:11 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:51:11.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:11 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:51:11.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:11 smithi053 ceph-mon[27773]: pgmap v326: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T14:51:11.582 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-04-25T14:51:11.582 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:51:11.583 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 2244, 2024-04-25T14:51:11.583 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 2245, 2024-04-25T14:51:11.583 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 636629, 2024-04-25T14:51:11.583 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 713281, 2024-04-25T14:51:11.583 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 6, 2024-04-25T14:51:11.583 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 2, 2024-04-25T14:51:11.583 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.172995812, 2024-04-25T14:51:11.584 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.057358026, 2024-04-25T14:51:11.584 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.041835822, 2024-04-25T14:51:11.584 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.059709713, 2024-04-25T14:51:11.584 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:51:11.584 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2243, 2024-04-25T14:51:11.584 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.071514109, 2024-04-25T14:51:11.584 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000031883 2024-04-25T14:51:11.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.585 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:51:11.585 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:11.585 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:11.585 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:11.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.585 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T14:51:11.585 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T14:51:11.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.586 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:51:11.586 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 4, 2024-04-25T14:51:11.586 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 166, 2024-04-25T14:51:11.586 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 876, 2024-04-25T14:51:11.586 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 453066, 2024-04-25T14:51:11.586 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 6, 2024-04-25T14:51:11.587 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 4, 2024-04-25T14:51:11.587 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.009884638, 2024-04-25T14:51:11.587 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.006475770, 2024-04-25T14:51:11.587 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.001643724, 2024-04-25T14:51:11.587 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000046036, 2024-04-25T14:51:11.587 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:51:11.588 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 163, 2024-04-25T14:51:11.588 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.019771087, 2024-04-25T14:51:11.588 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000121295 2024-04-25T14:51:11.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.588 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:51:11.588 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:11.588 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:11.589 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:11.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.589 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T14:51:11.589 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 452368 2024-04-25T14:51:11.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.589 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:51:11.589 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 251, 2024-04-25T14:51:11.590 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 236, 2024-04-25T14:51:11.590 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 1758812, 2024-04-25T14:51:11.590 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 135806, 2024-04-25T14:51:11.590 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 3, 2024-04-25T14:51:11.590 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 2, 2024-04-25T14:51:11.590 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.047756209, 2024-04-25T14:51:11.590 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.014701166, 2024-04-25T14:51:11.590 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.037446646, 2024-04-25T14:51:11.591 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.005472555, 2024-04-25T14:51:11.591 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:51:11.591 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 236, 2024-04-25T14:51:11.591 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.014304795, 2024-04-25T14:51:11.591 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000060613 2024-04-25T14:51:11.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.591 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:51:11.591 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:11.592 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:11.592 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:11.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.592 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 1758544, 2024-04-25T14:51:11.592 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 135488 2024-04-25T14:51:11.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.592 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-04-25T14:51:11.593 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 1, 2024-04-25T14:51:11.593 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-04-25T14:51:11.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.593 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-MDSRank": { 2024-04-25T14:51:11.593 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-04-25T14:51:11.593 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-04-25T14:51:11.593 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 3799, 2024-04-25T14:51:11.593 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.093029474, 2024-04-25T14:51:11.594 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000024487 2024-04-25T14:51:11.594 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T14:51:11.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.594 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-PurgeQueue": { 2024-04-25T14:51:11.594 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-04-25T14:51:11.594 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-04-25T14:51:11.594 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:11.595 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:11.595 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:11.595 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T14:51:11.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.595 INFO:teuthology.orchestra.run.smithi150.stdout: "mds": { 2024-04-25T14:51:11.595 INFO:teuthology.orchestra.run.smithi150.stdout: "request": 0, 2024-04-25T14:51:11.595 INFO:teuthology.orchestra.run.smithi150.stdout: "reply": 0, 2024-04-25T14:51:11.595 INFO:teuthology.orchestra.run.smithi150.stdout: "reply_latency": { 2024-04-25T14:51:11.596 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:11.596 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:11.596 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:11.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.596 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_reply": 0, 2024-04-25T14:51:11.596 INFO:teuthology.orchestra.run.smithi150.stdout: "forward": 0, 2024-04-25T14:51:11.596 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_complete": 0, 2024-04-25T14:51:11.596 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_keys": 0, 2024-04-25T14:51:11.597 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_commit": 0, 2024-04-25T14:51:11.597 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_split": 0, 2024-04-25T14:51:11.597 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_merge": 0, 2024-04-25T14:51:11.597 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes": 0, 2024-04-25T14:51:11.597 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_top": 0, 2024-04-25T14:51:11.597 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_bottom": 0, 2024-04-25T14:51:11.597 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pin_tail": 0, 2024-04-25T14:51:11.598 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pinned": 0, 2024-04-25T14:51:11.598 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_expired": 0, 2024-04-25T14:51:11.598 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_with_caps": 0, 2024-04-25T14:51:11.598 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": 0, 2024-04-25T14:51:11.598 INFO:teuthology.orchestra.run.smithi150.stdout: "subtrees": 1, 2024-04-25T14:51:11.598 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse": 0, 2024-04-25T14:51:11.598 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_hit": 0, 2024-04-25T14:51:11.598 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_forward": 0, 2024-04-25T14:51:11.599 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_discover": 0, 2024-04-25T14:51:11.599 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_dir_fetch": 0, 2024-04-25T14:51:11.599 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_remote_ino": 0, 2024-04-25T14:51:11.599 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_lock": 0, 2024-04-25T14:51:11.599 INFO:teuthology.orchestra.run.smithi150.stdout: "load_cent": 0, 2024-04-25T14:51:11.599 INFO:teuthology.orchestra.run.smithi150.stdout: "q": 0, 2024-04-25T14:51:11.599 INFO:teuthology.orchestra.run.smithi150.stdout: "exported": 0, 2024-04-25T14:51:11.600 INFO:teuthology.orchestra.run.smithi150.stdout: "exported_inodes": 0, 2024-04-25T14:51:11.600 INFO:teuthology.orchestra.run.smithi150.stdout: "imported": 0, 2024-04-25T14:51:11.600 INFO:teuthology.orchestra.run.smithi150.stdout: "imported_inodes": 0, 2024-04-25T14:51:11.600 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_dir_fetch": 0, 2024-04-25T14:51:11.600 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_backtrace_fetch": 0, 2024-04-25T14:51:11.600 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_peer_discover": 0, 2024-04-25T14:51:11.600 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rfiles": 0, 2024-04-25T14:51:11.600 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rbytes": 0, 2024-04-25T14:51:11.601 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rsnaps": 0, 2024-04-25T14:51:11.601 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T14:51:11.601 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_set_tag": 0, 2024-04-25T14:51:11.601 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T14:51:11.601 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_inotable_repaired": 0, 2024-04-25T14:51:11.601 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_inodes": 0, 2024-04-25T14:51:11.601 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T14:51:11.601 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T14:51:11.602 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_file_inodes": 0, 2024-04-25T14:51:11.602 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_inode_file_caps": 0, 2024-04-25T14:51:11.602 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T14:51:11.602 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_grant": 0, 2024-04-25T14:51:11.602 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T14:51:11.602 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T14:51:11.602 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T14:51:11.603 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps": 0, 2024-04-25T14:51:11.603 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps_dirty": 0, 2024-04-25T14:51:11.603 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_cap_release": 0, 2024-04-25T14:51:11.603 INFO:teuthology.orchestra.run.smithi150.stdout: "process_request_cap_release": 0 2024-04-25T14:51:11.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.603 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_cache": { 2024-04-25T14:51:11.603 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update": 0, 2024-04-25T14:51:11.603 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update_receipt": 0, 2024-04-25T14:51:11.604 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_try_discover": 0, 2024-04-25T14:51:11.604 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_send_discover": 0, 2024-04-25T14:51:11.604 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_handle_discover": 0, 2024-04-25T14:51:11.604 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays": 0, 2024-04-25T14:51:11.604 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_delayed": 0, 2024-04-25T14:51:11.604 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_enqueuing": 0, 2024-04-25T14:51:11.604 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_created": 0, 2024-04-25T14:51:11.604 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_enqueued": 0, 2024-04-25T14:51:11.605 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_reintegrated": 0, 2024-04-25T14:51:11.605 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_migrated": 0, 2024-04-25T14:51:11.605 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_processing": 0, 2024-04-25T14:51:11.605 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_enqueued": 0, 2024-04-25T14:51:11.605 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_prioritized": 0, 2024-04-25T14:51:11.605 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_started": 0, 2024-04-25T14:51:11.605 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_completed": 0, 2024-04-25T14:51:11.606 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_quiesce_path": 0, 2024-04-25T14:51:11.606 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_quiesce_inode": 0, 2024-04-25T14:51:11.606 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T14:51:11.606 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_exportdir": 0, 2024-04-25T14:51:11.606 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_flush": 0, 2024-04-25T14:51:11.606 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragmentdir": 0, 2024-04-25T14:51:11.606 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragstats": 0, 2024-04-25T14:51:11.606 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_inodestats": 0 2024-04-25T14:51:11.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.607 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_log": { 2024-04-25T14:51:11.607 INFO:teuthology.orchestra.run.smithi150.stdout: "evlrg": 0, 2024-04-25T14:51:11.607 INFO:teuthology.orchestra.run.smithi150.stdout: "evadd": 0, 2024-04-25T14:51:11.607 INFO:teuthology.orchestra.run.smithi150.stdout: "evex": 0, 2024-04-25T14:51:11.607 INFO:teuthology.orchestra.run.smithi150.stdout: "evtrm": 0, 2024-04-25T14:51:11.607 INFO:teuthology.orchestra.run.smithi150.stdout: "ev": 6, 2024-04-25T14:51:11.608 INFO:teuthology.orchestra.run.smithi150.stdout: "evexg": 0, 2024-04-25T14:51:11.608 INFO:teuthology.orchestra.run.smithi150.stdout: "evexd": 0, 2024-04-25T14:51:11.608 INFO:teuthology.orchestra.run.smithi150.stdout: "segadd": 0, 2024-04-25T14:51:11.608 INFO:teuthology.orchestra.run.smithi150.stdout: "segex": 0, 2024-04-25T14:51:11.608 INFO:teuthology.orchestra.run.smithi150.stdout: "segtrm": 0, 2024-04-25T14:51:11.608 INFO:teuthology.orchestra.run.smithi150.stdout: "seg": 2, 2024-04-25T14:51:11.608 INFO:teuthology.orchestra.run.smithi150.stdout: "segmjr": 2, 2024-04-25T14:51:11.608 INFO:teuthology.orchestra.run.smithi150.stdout: "segexg": 0, 2024-04-25T14:51:11.609 INFO:teuthology.orchestra.run.smithi150.stdout: "segexd": 0, 2024-04-25T14:51:11.609 INFO:teuthology.orchestra.run.smithi150.stdout: "expos": 4194304, 2024-04-25T14:51:11.609 INFO:teuthology.orchestra.run.smithi150.stdout: "wrpos": 4195956, 2024-04-25T14:51:11.609 INFO:teuthology.orchestra.run.smithi150.stdout: "rdpos": 4195815, 2024-04-25T14:51:11.609 INFO:teuthology.orchestra.run.smithi150.stdout: "jlat": { 2024-04-25T14:51:11.609 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:11.609 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:11.609 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:11.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.610 INFO:teuthology.orchestra.run.smithi150.stdout: "replayed": 6 2024-04-25T14:51:11.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.610 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_mem": { 2024-04-25T14:51:11.610 INFO:teuthology.orchestra.run.smithi150.stdout: "ino": 3, 2024-04-25T14:51:11.610 INFO:teuthology.orchestra.run.smithi150.stdout: "ino+": 3, 2024-04-25T14:51:11.610 INFO:teuthology.orchestra.run.smithi150.stdout: "ino-": 0, 2024-04-25T14:51:11.611 INFO:teuthology.orchestra.run.smithi150.stdout: "dir": 1, 2024-04-25T14:51:11.611 INFO:teuthology.orchestra.run.smithi150.stdout: "dir+": 1, 2024-04-25T14:51:11.611 INFO:teuthology.orchestra.run.smithi150.stdout: "dir-": 0, 2024-04-25T14:51:11.611 INFO:teuthology.orchestra.run.smithi150.stdout: "dn": 0, 2024-04-25T14:51:11.611 INFO:teuthology.orchestra.run.smithi150.stdout: "dn+": 0, 2024-04-25T14:51:11.611 INFO:teuthology.orchestra.run.smithi150.stdout: "dn-": 0, 2024-04-25T14:51:11.611 INFO:teuthology.orchestra.run.smithi150.stdout: "cap": 0, 2024-04-25T14:51:11.612 INFO:teuthology.orchestra.run.smithi150.stdout: "cap+": 0, 2024-04-25T14:51:11.612 INFO:teuthology.orchestra.run.smithi150.stdout: "cap-": 0, 2024-04-25T14:51:11.612 INFO:teuthology.orchestra.run.smithi150.stdout: "rss": 49492, 2024-04-25T14:51:11.612 INFO:teuthology.orchestra.run.smithi150.stdout: "heap": 215416 2024-04-25T14:51:11.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.612 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_server": { 2024-04-25T14:51:11.612 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_client_request": 0, 2024-04-25T14:51:11.612 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_server_request": 0, 2024-04-25T14:51:11.613 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_request": 0, 2024-04-25T14:51:11.613 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_session": 0, 2024-04-25T14:51:11.613 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_peer_request": 0, 2024-04-25T14:51:11.613 INFO:teuthology.orchestra.run.smithi150.stdout: "req_create_latency": { 2024-04-25T14:51:11.613 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:11.613 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:11.613 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:11.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.614 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getattr_latency": { 2024-04-25T14:51:11.614 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:11.614 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:11.614 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:11.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.614 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getfilelock_latency": { 2024-04-25T14:51:11.614 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:11.615 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:11.615 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:11.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.615 INFO:teuthology.orchestra.run.smithi150.stdout: "req_link_latency": { 2024-04-25T14:51:11.615 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:11.615 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:11.615 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:11.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.616 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookup_latency": { 2024-04-25T14:51:11.616 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:11.616 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:11.616 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:11.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.616 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookuphash_latency": { 2024-04-25T14:51:11.616 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:11.616 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:11.617 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:11.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.617 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupino_latency": { 2024-04-25T14:51:11.617 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:11.617 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:11.617 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:11.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.618 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupname_latency": { 2024-04-25T14:51:11.618 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:11.618 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:11.618 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:11.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.618 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupparent_latency": { 2024-04-25T14:51:11.618 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:11.618 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:11.619 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:11.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.619 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupsnap_latency": { 2024-04-25T14:51:11.619 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:11.619 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:11.619 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:11.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.620 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lssnap_latency": { 2024-04-25T14:51:11.620 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:11.620 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:11.620 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:11.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.620 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mkdir_latency": { 2024-04-25T14:51:11.620 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:11.621 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:11.621 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:11.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.621 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mknod_latency": { 2024-04-25T14:51:11.622 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:11.622 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:11.622 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:11.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.622 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mksnap_latency": { 2024-04-25T14:51:11.622 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:11.623 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:11.623 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:11.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.623 INFO:teuthology.orchestra.run.smithi150.stdout: "req_open_latency": { 2024-04-25T14:51:11.623 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:11.623 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:11.623 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:11.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.624 INFO:teuthology.orchestra.run.smithi150.stdout: "req_readdir_latency": { 2024-04-25T14:51:11.624 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:11.624 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:11.624 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:11.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.624 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rename_latency": { 2024-04-25T14:51:11.625 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:11.625 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:11.625 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:11.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.625 INFO:teuthology.orchestra.run.smithi150.stdout: "req_renamesnap_latency": { 2024-04-25T14:51:11.625 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:11.625 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:11.625 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:11.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.626 INFO:teuthology.orchestra.run.smithi150.stdout: "req_snapdiff_latency": { 2024-04-25T14:51:11.626 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:11.626 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:11.626 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:11.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.626 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmdir_latency": { 2024-04-25T14:51:11.626 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:11.627 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:11.627 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:11.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.627 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmsnap_latency": { 2024-04-25T14:51:11.627 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:11.627 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:11.627 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:11.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.628 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmxattr_latency": { 2024-04-25T14:51:11.628 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:11.628 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:11.628 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:11.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.628 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setattr_latency": { 2024-04-25T14:51:11.629 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:11.629 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:11.629 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:11.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.629 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setdirlayout_latency": { 2024-04-25T14:51:11.629 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:11.629 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:11.629 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:11.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.630 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setfilelock_latency": { 2024-04-25T14:51:11.630 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:11.630 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:11.630 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:11.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.630 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setlayout_latency": { 2024-04-25T14:51:11.631 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:11.631 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:11.631 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:11.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.631 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setxattr_latency": { 2024-04-25T14:51:11.631 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:11.631 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:11.631 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:11.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.632 INFO:teuthology.orchestra.run.smithi150.stdout: "req_symlink_latency": { 2024-04-25T14:51:11.632 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:11.632 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:11.632 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:11.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.632 INFO:teuthology.orchestra.run.smithi150.stdout: "req_unlink_latency": { 2024-04-25T14:51:11.633 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:11.633 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:11.633 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:11.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.633 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_revoke_eviction": 0, 2024-04-25T14:51:11.633 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_acquisition_throttle": 0, 2024-04-25T14:51:11.633 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getvxattr_latency": { 2024-04-25T14:51:11.633 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:11.634 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:11.634 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:11.634 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T14:51:11.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.634 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_sessions": { 2024-04-25T14:51:11.634 INFO:teuthology.orchestra.run.smithi150.stdout: "session_count": 0, 2024-04-25T14:51:11.634 INFO:teuthology.orchestra.run.smithi150.stdout: "session_add": 2, 2024-04-25T14:51:11.635 INFO:teuthology.orchestra.run.smithi150.stdout: "session_remove": 2, 2024-04-25T14:51:11.635 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_open": 1, 2024-04-25T14:51:11.635 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_stale": 0, 2024-04-25T14:51:11.635 INFO:teuthology.orchestra.run.smithi150.stdout: "total_load": 0, 2024-04-25T14:51:11.635 INFO:teuthology.orchestra.run.smithi150.stdout: "average_load": 0, 2024-04-25T14:51:11.635 INFO:teuthology.orchestra.run.smithi150.stdout: "avg_session_uptime": 1550, 2024-04-25T14:51:11.635 INFO:teuthology.orchestra.run.smithi150.stdout: "mdthresh_evicted": 0 2024-04-25T14:51:11.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.636 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-04-25T14:51:11.636 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:51:11.636 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-04-25T14:51:11.636 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:51:11.636 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:51:11.636 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:51:11.636 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:51:11.637 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:51:11.637 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:51:11.637 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:51:11.637 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:51:11.637 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:51:11.637 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:51:11.637 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:51:11.638 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:51:11.638 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:51:11.638 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-04-25T14:51:11.638 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:51:11.638 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-04-25T14:51:11.638 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:51:11.638 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:51:11.639 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:51:11.639 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:51:11.639 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:51:11.639 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:51:11.639 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:51:11.639 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-04-25T14:51:11.639 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:51:11.640 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:51:11.640 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:51:11.640 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-04-25T14:51:11.640 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-04-25T14:51:11.640 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-04-25T14:51:11.640 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:51:11.640 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:51:11.641 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:51:11.641 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:51:11.641 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 181269, 2024-04-25T14:51:11.641 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 47, 2024-04-25T14:51:11.641 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 0, 2024-04-25T14:51:11.641 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 0, 2024-04-25T14:51:11.641 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-04-25T14:51:11.641 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-04-25T14:51:11.642 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:51:11.642 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-04-25T14:51:11.642 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:51:11.642 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-04-25T14:51:11.642 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 8360, 2024-04-25T14:51:11.642 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 176, 2024-04-25T14:51:11.642 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:51:11.642 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:51:11.643 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-04-25T14:51:11.643 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-04-25T14:51:11.643 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 8512, 2024-04-25T14:51:11.643 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 13, 2024-04-25T14:51:11.643 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-04-25T14:51:11.643 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-04-25T14:51:11.643 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-04-25T14:51:11.643 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-04-25T14:51:11.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.644 INFO:teuthology.orchestra.run.smithi150.stdout: "objecter": { 2024-04-25T14:51:11.644 INFO:teuthology.orchestra.run.smithi150.stdout: "op_active": 0, 2024-04-25T14:51:11.644 INFO:teuthology.orchestra.run.smithi150.stdout: "op_laggy": 0, 2024-04-25T14:51:11.644 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send": 2246, 2024-04-25T14:51:11.644 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send_bytes": 21, 2024-04-25T14:51:11.644 INFO:teuthology.orchestra.run.smithi150.stdout: "op_resend": 3, 2024-04-25T14:51:11.644 INFO:teuthology.orchestra.run.smithi150.stdout: "op_reply": 2246, 2024-04-25T14:51:11.645 INFO:teuthology.orchestra.run.smithi150.stdout: "op_latency": { 2024-04-25T14:51:11.645 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2246, 2024-04-25T14:51:11.645 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 1.436971801, 2024-04-25T14:51:11.645 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000639791 2024-04-25T14:51:11.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.645 INFO:teuthology.orchestra.run.smithi150.stdout: "op_inflight": 0, 2024-04-25T14:51:11.645 INFO:teuthology.orchestra.run.smithi150.stdout: "oplen_avg": { 2024-04-25T14:51:11.646 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2246, 2024-04-25T14:51:11.646 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 2247 2024-04-25T14:51:11.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.646 INFO:teuthology.orchestra.run.smithi150.stdout: "op": 2246, 2024-04-25T14:51:11.646 INFO:teuthology.orchestra.run.smithi150.stdout: "op_r": 2246, 2024-04-25T14:51:11.646 INFO:teuthology.orchestra.run.smithi150.stdout: "op_w": 0, 2024-04-25T14:51:11.646 INFO:teuthology.orchestra.run.smithi150.stdout: "op_rmw": 0, 2024-04-25T14:51:11.646 INFO:teuthology.orchestra.run.smithi150.stdout: "op_pg": 0, 2024-04-25T14:51:11.647 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_stat": 1461, 2024-04-25T14:51:11.647 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_create": 0, 2024-04-25T14:51:11.647 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_read": 783, 2024-04-25T14:51:11.647 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_write": 0, 2024-04-25T14:51:11.647 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writefull": 0, 2024-04-25T14:51:11.647 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writesame": 0, 2024-04-25T14:51:11.647 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_append": 0, 2024-04-25T14:51:11.648 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_zero": 0, 2024-04-25T14:51:11.648 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_truncate": 0, 2024-04-25T14:51:11.648 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_delete": 0, 2024-04-25T14:51:11.648 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_mapext": 0, 2024-04-25T14:51:11.648 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_sparse_read": 0, 2024-04-25T14:51:11.648 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_clonerange": 0, 2024-04-25T14:51:11.648 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_getxattr": 1, 2024-04-25T14:51:11.648 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_setxattr": 0, 2024-04-25T14:51:11.649 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_cmpxattr": 0, 2024-04-25T14:51:11.649 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_rmxattr": 0, 2024-04-25T14:51:11.649 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_resetxattrs": 0, 2024-04-25T14:51:11.649 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_call": 0, 2024-04-25T14:51:11.649 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_watch": 0, 2024-04-25T14:51:11.649 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_notify": 0, 2024-04-25T14:51:11.649 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T14:51:11.650 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls": 0, 2024-04-25T14:51:11.650 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls_filter": 0, 2024-04-25T14:51:11.650 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_other": 0, 2024-04-25T14:51:11.650 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_active": 0, 2024-04-25T14:51:11.650 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_send": 0, 2024-04-25T14:51:11.650 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_resend": 0, 2024-04-25T14:51:11.650 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_ping": 0, 2024-04-25T14:51:11.650 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_active": 0, 2024-04-25T14:51:11.651 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_send": 0, 2024-04-25T14:51:11.651 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_resend": 0, 2024-04-25T14:51:11.651 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_active": 0, 2024-04-25T14:51:11.651 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_send": 0, 2024-04-25T14:51:11.651 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_resend": 0, 2024-04-25T14:51:11.651 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_active": 0, 2024-04-25T14:51:11.651 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_send": 0, 2024-04-25T14:51:11.652 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_resend": 0, 2024-04-25T14:51:11.652 INFO:teuthology.orchestra.run.smithi150.stdout: "command_active": 0, 2024-04-25T14:51:11.652 INFO:teuthology.orchestra.run.smithi150.stdout: "command_send": 0, 2024-04-25T14:51:11.652 INFO:teuthology.orchestra.run.smithi150.stdout: "command_resend": 0, 2024-04-25T14:51:11.652 INFO:teuthology.orchestra.run.smithi150.stdout: "map_epoch": 102, 2024-04-25T14:51:11.652 INFO:teuthology.orchestra.run.smithi150.stdout: "map_full": 0, 2024-04-25T14:51:11.652 INFO:teuthology.orchestra.run.smithi150.stdout: "map_inc": 14, 2024-04-25T14:51:11.653 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_sessions": 4, 2024-04-25T14:51:11.653 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_open": 4, 2024-04-25T14:51:11.653 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_close": 0, 2024-04-25T14:51:11.653 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_laggy": 0, 2024-04-25T14:51:11.653 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_wr": 0, 2024-04-25T14:51:11.653 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_rd": 2, 2024-04-25T14:51:11.653 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_del": 0 2024-04-25T14:51:11.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.654 INFO:teuthology.orchestra.run.smithi150.stdout: "oft": { 2024-04-25T14:51:11.654 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_objs": 0, 2024-04-25T14:51:11.654 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_kv_pairs": 0, 2024-04-25T14:51:11.654 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_updates": 0, 2024-04-25T14:51:11.654 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_removes": 0 2024-04-25T14:51:11.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.654 INFO:teuthology.orchestra.run.smithi150.stdout: "osd-slow-ops": { 2024-04-25T14:51:11.654 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_ops_count": 0 2024-04-25T14:51:11.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.655 INFO:teuthology.orchestra.run.smithi150.stdout: "purge_queue": { 2024-04-25T14:51:11.655 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops": 0, 2024-04-25T14:51:11.655 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T14:51:11.655 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing": 0, 2024-04-25T14:51:11.655 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_high_water": 0, 2024-04-25T14:51:11.655 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed_ops": 0, 2024-04-25T14:51:11.656 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed": 0, 2024-04-25T14:51:11.656 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_item_in_journal": 0 2024-04-25T14:51:11.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.656 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:51:11.656 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-25T14:51:11.656 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-04-25T14:51:11.656 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-25T14:51:11.657 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 2495, 2024-04-25T14:51:11.657 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 2156188, 2024-04-25T14:51:11.657 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-25T14:51:11.657 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 2495, 2024-04-25T14:51:11.657 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-04-25T14:51:11.657 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-04-25T14:51:11.657 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 2495, 2024-04-25T14:51:11.657 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 2156188, 2024-04-25T14:51:11.658 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-25T14:51:11.658 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:11.658 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:11.658 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:11.658 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T14:51:11.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.658 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_bytes": { 2024-04-25T14:51:11.659 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-25T14:51:11.659 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-04-25T14:51:11.659 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-25T14:51:11.659 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-04-25T14:51:11.659 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-04-25T14:51:11.659 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-25T14:51:11.659 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-04-25T14:51:11.659 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 2246, 2024-04-25T14:51:11.660 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 1657, 2024-04-25T14:51:11.660 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 2246, 2024-04-25T14:51:11.660 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 1657, 2024-04-25T14:51:11.660 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-25T14:51:11.660 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:11.660 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:11.660 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:11.661 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T14:51:11.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.661 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_ops": { 2024-04-25T14:51:11.661 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-25T14:51:11.661 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 1024, 2024-04-25T14:51:11.661 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-25T14:51:11.661 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-04-25T14:51:11.661 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-04-25T14:51:11.662 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-25T14:51:11.662 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-04-25T14:51:11.662 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 2246, 2024-04-25T14:51:11.662 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 2246, 2024-04-25T14:51:11.662 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 2246, 2024-04-25T14:51:11.662 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 2246, 2024-04-25T14:51:11.662 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-25T14:51:11.663 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:11.663 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:11.663 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:11.663 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T14:51:11.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.663 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle": { 2024-04-25T14:51:11.663 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-25T14:51:11.663 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-04-25T14:51:11.664 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-25T14:51:11.664 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-04-25T14:51:11.664 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-04-25T14:51:11.664 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-25T14:51:11.664 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-04-25T14:51:11.664 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-04-25T14:51:11.664 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-04-25T14:51:11.665 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 0, 2024-04-25T14:51:11.665 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 0, 2024-04-25T14:51:11.665 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-25T14:51:11.665 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:11.665 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:11.665 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:11.665 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T14:51:11.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:11.666 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle-0x55881c6366e0": { 2024-04-25T14:51:11.666 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-25T14:51:11.666 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-04-25T14:51:11.666 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-25T14:51:11.666 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-04-25T14:51:11.666 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-04-25T14:51:11.667 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-25T14:51:11.667 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-04-25T14:51:11.667 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-04-25T14:51:11.667 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-04-25T14:51:11.667 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 0, 2024-04-25T14:51:11.667 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 0, 2024-04-25T14:51:11.667 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-25T14:51:11.667 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:11.668 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:11.668 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:11.668 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T14:51:11.668 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T14:51:11.668 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-04-25T14:51:12.355 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:12 smithi150 ceph-mon[32504]: osdmap e102: 12 total, 12 up, 12 in 2024-04-25T14:51:12.356 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:12 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:12.356 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:12 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:12.356 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:12 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:12.356 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:12 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:12.356 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:12 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:12.356 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:12 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:12.356 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:12 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:12.356 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:12 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:12.356 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:12 smithi150 ceph-mon[32504]: Adjusting osd_memory_target on smithi053 to 177.5M 2024-04-25T14:51:12.357 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:12 smithi150 ceph-mon[32504]: Unable to set osd_memory_target on smithi053 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-25T14:51:12.382 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-25T14:51:12.382 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-25T14:51:12.382 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-04-25T14:51:12.383 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2024-04-25T14:51:12.383 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2024-04-25T14:51:12.383 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-25T14:51:12.383 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T14:51:12.383 DEBUG:tasks.check_counter:Getting stats from k 2024-04-25T14:51:12.383 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-25T14:51:12.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:12 smithi053 ceph-mon[27773]: osdmap e102: 12 total, 12 up, 12 in 2024-04-25T14:51:12.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:12 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:12.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:12 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:12.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:12 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:12.534 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:12 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:12.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:12 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:12.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:12 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:12.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:12 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:12.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:12 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:12.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:12 smithi053 ceph-mon[27773]: Adjusting osd_memory_target on smithi053 to 177.5M 2024-04-25T14:51:12.535 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:12 smithi053 ceph-mon[27773]: Unable to set osd_memory_target on smithi053 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-25T14:51:12.642 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:51:12.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:12 smithi183 ceph-mon[32584]: osdmap e102: 12 total, 12 up, 12 in 2024-04-25T14:51:12.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:12 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:12.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:12 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:12.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:12 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:12.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:12 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:12.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:12 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:12.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:12 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:12.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:12 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:12.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:12 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:12.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:12 smithi183 ceph-mon[32584]: Adjusting osd_memory_target on smithi053 to 177.5M 2024-04-25T14:51:12.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:12 smithi183 ceph-mon[32584]: Unable to set osd_memory_target on smithi053 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-25T14:51:13.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:13 smithi150 ceph-mon[32504]: pgmap v328: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T14:51:13.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:13 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:51:13.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:13 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:51:13.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:13 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:13.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:13 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:13.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:13 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:13.712 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:13 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:13.712 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:13 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:13.712 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:13 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:13.712 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:13 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:13.712 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:13 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:13.712 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:13 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:51:13.712 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:13 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:51:13.712 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:13 smithi150 ceph-mon[32504]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:51:13.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:13 smithi183 ceph-mon[32584]: pgmap v328: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T14:51:13.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:13 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:51:13.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:13 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:51:13.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:13 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:13.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:13 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:13.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:13 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:13.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:13 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:13.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:13 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:13.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:13 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:13.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:13 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:13.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:13 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:13.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:13 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:51:13.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:13 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:51:13.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:13 smithi183 ceph-mon[32584]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:51:14.034 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:13 smithi053 ceph-mon[27773]: pgmap v328: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T14:51:14.034 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:13 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:51:14.034 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:13 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:51:14.034 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:13 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:14.034 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:13 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:14.034 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:13 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:14.035 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:13 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:14.035 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:13 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:14.035 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:13 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:14.035 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:13 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:14.035 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:13 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:51:14.035 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:13 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:51:14.035 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:13 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:51:14.035 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:13 smithi053 ceph-mon[27773]: from='mgr.15093 ' entity='mgr.x' 2024-04-25T14:51:14.301 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-04-25T14:51:14.301 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:51:14.301 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 4, 2024-04-25T14:51:14.301 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 4, 2024-04-25T14:51:14.302 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 2992, 2024-04-25T14:51:14.302 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 448, 2024-04-25T14:51:14.302 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 4, 2024-04-25T14:51:14.302 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 0, 2024-04-25T14:51:14.302 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.001339164, 2024-04-25T14:51:14.302 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.000039062, 2024-04-25T14:51:14.302 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.000521401, 2024-04-25T14:51:14.302 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T14:51:14.303 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:51:14.303 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 3, 2024-04-25T14:51:14.303 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000723111, 2024-04-25T14:51:14.303 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000241037 2024-04-25T14:51:14.303 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:14.303 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:51:14.303 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:14.303 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:14.303 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:14.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:14.304 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-25T14:51:14.304 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T14:51:14.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:14.304 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:51:14.304 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 2, 2024-04-25T14:51:14.304 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 168, 2024-04-25T14:51:14.304 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 320, 2024-04-25T14:51:14.304 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 185648, 2024-04-25T14:51:14.305 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 5, 2024-04-25T14:51:14.305 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 3, 2024-04-25T14:51:14.305 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.012248594, 2024-04-25T14:51:14.305 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.008127840, 2024-04-25T14:51:14.305 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.001621024, 2024-04-25T14:51:14.305 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T14:51:14.305 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:51:14.305 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 165, 2024-04-25T14:51:14.305 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.023791639, 2024-04-25T14:51:14.306 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000144191 2024-04-25T14:51:14.306 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:14.306 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:51:14.306 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:14.306 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:14.306 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:14.306 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:14.306 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T14:51:14.307 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 185648 2024-04-25T14:51:14.307 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:14.307 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:51:14.307 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 232, 2024-04-25T14:51:14.307 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 220, 2024-04-25T14:51:14.307 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 1730944, 2024-04-25T14:51:14.307 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 135584, 2024-04-25T14:51:14.307 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 2, 2024-04-25T14:51:14.307 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 1, 2024-04-25T14:51:14.308 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.050707958, 2024-04-25T14:51:14.308 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.014711153, 2024-04-25T14:51:14.308 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.041343594, 2024-04-25T14:51:14.308 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.005549083, 2024-04-25T14:51:14.308 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:51:14.308 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 219, 2024-04-25T14:51:14.308 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.012693976, 2024-04-25T14:51:14.308 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000057963 2024-04-25T14:51:14.308 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:14.309 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:51:14.309 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:14.309 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:14.309 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:14.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:14.309 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 1730944, 2024-04-25T14:51:14.309 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 135584 2024-04-25T14:51:14.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:14.309 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-04-25T14:51:14.310 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 0, 2024-04-25T14:51:14.310 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-04-25T14:51:14.310 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:14.310 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-04-25T14:51:14.310 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:51:14.310 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-04-25T14:51:14.310 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:51:14.310 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:51:14.311 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:51:14.311 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:51:14.311 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:51:14.311 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:51:14.311 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:51:14.311 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:51:14.311 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:51:14.311 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:51:14.311 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:51:14.312 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:51:14.312 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:51:14.312 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-04-25T14:51:14.312 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:51:14.312 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-04-25T14:51:14.312 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:51:14.312 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:51:14.312 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:51:14.312 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:51:14.313 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:51:14.313 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:51:14.313 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:51:14.313 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-04-25T14:51:14.313 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:51:14.313 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:51:14.313 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:51:14.313 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-04-25T14:51:14.314 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-04-25T14:51:14.314 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-04-25T14:51:14.314 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:51:14.314 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:51:14.314 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:51:14.314 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:51:14.314 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 148714, 2024-04-25T14:51:14.314 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 30, 2024-04-25T14:51:14.314 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 0, 2024-04-25T14:51:14.315 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 0, 2024-04-25T14:51:14.315 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-04-25T14:51:14.315 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-04-25T14:51:14.315 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:51:14.315 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-04-25T14:51:14.315 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:51:14.315 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-04-25T14:51:14.315 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 0, 2024-04-25T14:51:14.316 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 0, 2024-04-25T14:51:14.316 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:51:14.316 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:51:14.316 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-04-25T14:51:14.316 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-04-25T14:51:14.316 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 928, 2024-04-25T14:51:14.316 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 2, 2024-04-25T14:51:14.316 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-04-25T14:51:14.316 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-04-25T14:51:14.317 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-04-25T14:51:14.317 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-04-25T14:51:14.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T14:51:14.317 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:51:14.317 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-25T14:51:14.317 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-04-25T14:51:14.317 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-25T14:51:14.317 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 234, 2024-04-25T14:51:14.317 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 1698236, 2024-04-25T14:51:14.318 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-25T14:51:14.318 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 234, 2024-04-25T14:51:14.318 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-04-25T14:51:14.318 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-04-25T14:51:14.318 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 234, 2024-04-25T14:51:14.318 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 1698236, 2024-04-25T14:51:14.318 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-25T14:51:14.318 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T14:51:14.319 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T14:51:14.319 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T14:51:14.319 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T14:51:14.319 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T14:51:14.319 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-04-25T14:51:14.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:14 smithi183 ceph-mon[32584]: Adjusting osd_memory_target on smithi150 to 176.2M 2024-04-25T14:51:14.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:14 smithi183 ceph-mon[32584]: Unable to set osd_memory_target on smithi150 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-25T14:51:14.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:14 smithi150 ceph-mon[32504]: Adjusting osd_memory_target on smithi150 to 176.2M 2024-04-25T14:51:14.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:14 smithi150 ceph-mon[32504]: Unable to set osd_memory_target on smithi150 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-25T14:51:14.997 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-04-25T14:51:14.997 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-04-25T14:51:14.998 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-04-25T14:51:14.998 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-04-25T14:51:14.998 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-04-25T14:51:14.998 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-04-25T14:51:14.998 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-04-25T14:51:14.998 DEBUG:tasks.check_counter:Getting stats from c 2024-04-25T14:51:14.999 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-25T14:51:15.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:14 smithi053 ceph-mon[27773]: Adjusting osd_memory_target on smithi150 to 176.2M 2024-04-25T14:51:15.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:14 smithi053 ceph-mon[27773]: Unable to set osd_memory_target on smithi150 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-25T14:51:15.246 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:51:15.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:15 smithi183 ceph-mon[32584]: pgmap v329: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T14:51:15.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:15 smithi150 ceph-mon[32504]: pgmap v329: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T14:51:16.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:15 smithi053 ceph-mon[27773]: pgmap v329: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T14:51:17.023 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-04-25T14:51:17.023 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:51:17.024 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 1614, 2024-04-25T14:51:17.024 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 1616, 2024-04-25T14:51:17.024 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 575089, 2024-04-25T14:51:17.024 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 513262, 2024-04-25T14:51:17.024 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 6, 2024-04-25T14:51:17.024 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 2, 2024-04-25T14:51:17.024 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.158967821, 2024-04-25T14:51:17.024 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.052001231, 2024-04-25T14:51:17.025 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.039707173, 2024-04-25T14:51:17.025 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.051687716, 2024-04-25T14:51:17.025 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:51:17.025 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1614, 2024-04-25T14:51:17.025 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.068005037, 2024-04-25T14:51:17.025 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000042134 2024-04-25T14:51:17.025 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.025 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:51:17.025 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:17.026 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:17.026 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:17.026 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.026 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T14:51:17.026 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T14:51:17.026 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.026 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:51:17.026 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 6, 2024-04-25T14:51:17.027 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 170, 2024-04-25T14:51:17.027 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 1976, 2024-04-25T14:51:17.027 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 455674, 2024-04-25T14:51:17.027 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 7, 2024-04-25T14:51:17.027 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 5, 2024-04-25T14:51:17.027 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.010098714, 2024-04-25T14:51:17.027 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.006372801, 2024-04-25T14:51:17.027 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.001777076, 2024-04-25T14:51:17.028 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000101150, 2024-04-25T14:51:17.028 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:51:17.028 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 167, 2024-04-25T14:51:17.028 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.024937042, 2024-04-25T14:51:17.028 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000149323 2024-04-25T14:51:17.028 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.028 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:51:17.028 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:17.028 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:17.029 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:17.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.029 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T14:51:17.029 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 454336 2024-04-25T14:51:17.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.029 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:51:17.029 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 1016, 2024-04-25T14:51:17.029 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 1000, 2024-04-25T14:51:17.030 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 1947223, 2024-04-25T14:51:17.030 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 379567, 2024-04-25T14:51:17.030 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 3, 2024-04-25T14:51:17.030 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 2, 2024-04-25T14:51:17.030 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.121191040, 2024-04-25T14:51:17.030 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.041870663, 2024-04-25T14:51:17.030 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.052899089, 2024-04-25T14:51:17.030 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.030302696, 2024-04-25T14:51:17.030 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:51:17.031 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1000, 2024-04-25T14:51:17.031 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.038535702, 2024-04-25T14:51:17.031 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000038535 2024-04-25T14:51:17.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.031 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:51:17.031 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:17.031 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:17.031 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:17.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.032 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 1759520, 2024-04-25T14:51:17.032 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 137568 2024-04-25T14:51:17.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.032 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-04-25T14:51:17.032 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-04-25T14:51:17.032 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-04-25T14:51:17.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.032 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-04-25T14:51:17.033 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-04-25T14:51:17.033 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-04-25T14:51:17.033 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3973, 2024-04-25T14:51:17.033 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.114649198, 2024-04-25T14:51:17.033 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000028857 2024-04-25T14:51:17.033 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T14:51:17.033 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.033 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-04-25T14:51:17.033 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-04-25T14:51:17.034 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-04-25T14:51:17.034 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:17.034 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:17.034 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:17.034 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T14:51:17.034 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.034 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-04-25T14:51:17.034 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 0, 2024-04-25T14:51:17.034 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 0, 2024-04-25T14:51:17.035 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-04-25T14:51:17.035 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:17.035 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:17.035 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:17.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.035 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-04-25T14:51:17.035 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 0, 2024-04-25T14:51:17.035 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 0, 2024-04-25T14:51:17.035 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2024-04-25T14:51:17.036 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 0, 2024-04-25T14:51:17.036 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 0, 2024-04-25T14:51:17.036 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2024-04-25T14:51:17.036 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 12, 2024-04-25T14:51:17.036 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 0, 2024-04-25T14:51:17.036 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 12, 2024-04-25T14:51:17.036 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 0, 2024-04-25T14:51:17.036 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 4, 2024-04-25T14:51:17.036 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 0, 2024-04-25T14:51:17.037 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 0, 2024-04-25T14:51:17.037 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 0, 2024-04-25T14:51:17.037 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 2, 2024-04-25T14:51:17.037 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 0, 2024-04-25T14:51:17.037 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 0, 2024-04-25T14:51:17.037 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 0, 2024-04-25T14:51:17.037 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 0, 2024-04-25T14:51:17.037 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2024-04-25T14:51:17.037 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2024-04-25T14:51:17.038 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 0, 2024-04-25T14:51:17.038 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2024-04-25T14:51:17.038 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-04-25T14:51:17.038 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 0, 2024-04-25T14:51:17.038 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 0, 2024-04-25T14:51:17.038 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 0, 2024-04-25T14:51:17.038 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 0, 2024-04-25T14:51:17.038 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-04-25T14:51:17.039 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 0, 2024-04-25T14:51:17.039 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2024-04-25T14:51:17.039 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 4, 2024-04-25T14:51:17.039 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 234, 2024-04-25T14:51:17.039 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 6, 2024-04-25T14:51:17.039 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T14:51:17.039 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-04-25T14:51:17.039 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T14:51:17.039 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-04-25T14:51:17.040 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 0, 2024-04-25T14:51:17.040 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T14:51:17.040 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T14:51:17.040 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 0, 2024-04-25T14:51:17.040 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 0, 2024-04-25T14:51:17.040 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T14:51:17.040 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 0, 2024-04-25T14:51:17.040 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T14:51:17.040 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T14:51:17.041 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T14:51:17.041 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 0, 2024-04-25T14:51:17.041 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 0, 2024-04-25T14:51:17.041 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 0, 2024-04-25T14:51:17.041 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 0 2024-04-25T14:51:17.041 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.041 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-04-25T14:51:17.041 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 0, 2024-04-25T14:51:17.041 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 0, 2024-04-25T14:51:17.042 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 0, 2024-04-25T14:51:17.042 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 0, 2024-04-25T14:51:17.042 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 0, 2024-04-25T14:51:17.042 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 0, 2024-04-25T14:51:17.042 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-04-25T14:51:17.042 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-04-25T14:51:17.042 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 0, 2024-04-25T14:51:17.042 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 0, 2024-04-25T14:51:17.043 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 0, 2024-04-25T14:51:17.043 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2024-04-25T14:51:17.043 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-04-25T14:51:17.043 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-04-25T14:51:17.043 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-04-25T14:51:17.043 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-04-25T14:51:17.043 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-04-25T14:51:17.043 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 0, 2024-04-25T14:51:17.043 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 0, 2024-04-25T14:51:17.044 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T14:51:17.044 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 0, 2024-04-25T14:51:17.044 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-04-25T14:51:17.044 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 0, 2024-04-25T14:51:17.044 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-04-25T14:51:17.044 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0 2024-04-25T14:51:17.044 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.044 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-04-25T14:51:17.044 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2024-04-25T14:51:17.045 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 0, 2024-04-25T14:51:17.045 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 0, 2024-04-25T14:51:17.045 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 0, 2024-04-25T14:51:17.045 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 110, 2024-04-25T14:51:17.045 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-04-25T14:51:17.045 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 0, 2024-04-25T14:51:17.045 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 0, 2024-04-25T14:51:17.045 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 0, 2024-04-25T14:51:17.045 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 0, 2024-04-25T14:51:17.046 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 2, 2024-04-25T14:51:17.046 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 3, 2024-04-25T14:51:17.046 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-04-25T14:51:17.046 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 0, 2024-04-25T14:51:17.046 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 4262242, 2024-04-25T14:51:17.046 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 4289048, 2024-04-25T14:51:17.046 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 4288936, 2024-04-25T14:51:17.046 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-04-25T14:51:17.046 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:17.047 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:17.047 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:17.047 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.047 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 110 2024-04-25T14:51:17.047 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.047 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-04-25T14:51:17.047 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 13, 2024-04-25T14:51:17.047 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 13, 2024-04-25T14:51:17.047 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 0, 2024-04-25T14:51:17.048 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 8, 2024-04-25T14:51:17.048 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 8, 2024-04-25T14:51:17.048 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 0, 2024-04-25T14:51:17.048 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 12, 2024-04-25T14:51:17.048 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 12, 2024-04-25T14:51:17.048 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 0, 2024-04-25T14:51:17.048 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 0, 2024-04-25T14:51:17.048 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 0, 2024-04-25T14:51:17.049 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 0, 2024-04-25T14:51:17.049 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 53044, 2024-04-25T14:51:17.049 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 215416 2024-04-25T14:51:17.049 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.049 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-04-25T14:51:17.049 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 0, 2024-04-25T14:51:17.049 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 0, 2024-04-25T14:51:17.049 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 0, 2024-04-25T14:51:17.049 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 0, 2024-04-25T14:51:17.050 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 0, 2024-04-25T14:51:17.050 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-04-25T14:51:17.050 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:17.050 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:17.050 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:17.050 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.050 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-04-25T14:51:17.050 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:17.050 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:17.051 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:17.051 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.051 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-04-25T14:51:17.051 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:17.051 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:17.051 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:17.051 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.051 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-04-25T14:51:17.052 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:17.052 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:17.052 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:17.052 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.052 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-04-25T14:51:17.052 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:17.052 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:17.052 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:17.052 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.053 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-04-25T14:51:17.053 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:17.053 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:17.053 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:17.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.053 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-04-25T14:51:17.053 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:17.053 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:17.053 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:17.054 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.054 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-04-25T14:51:17.054 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:17.054 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:17.054 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:17.054 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.054 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-04-25T14:51:17.054 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:17.055 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:17.055 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:17.055 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.055 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-04-25T14:51:17.055 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:17.055 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:17.055 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:17.055 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.055 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-04-25T14:51:17.056 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:17.056 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:17.056 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:17.056 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.056 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-04-25T14:51:17.056 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:17.057 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:17.057 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:17.057 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.057 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-04-25T14:51:17.057 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:17.057 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:17.058 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:17.058 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.058 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-04-25T14:51:17.058 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:17.058 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:17.058 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:17.058 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.058 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-04-25T14:51:17.059 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:17.059 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:17.059 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:17.059 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.059 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-04-25T14:51:17.059 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:17.059 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:17.059 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:17.059 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.060 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-04-25T14:51:17.060 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:17.060 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:17.060 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:17.060 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.060 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-04-25T14:51:17.060 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:17.060 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:17.061 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:17.061 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.061 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-04-25T14:51:17.061 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:17.061 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:17.061 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:17.061 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.061 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-04-25T14:51:17.061 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:17.062 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:17.062 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:17.062 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.062 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-04-25T14:51:17.062 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:17.062 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:17.062 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:17.062 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.063 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-04-25T14:51:17.063 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:17.063 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:17.063 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:17.063 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.063 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-04-25T14:51:17.063 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:17.063 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:17.063 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:17.064 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.064 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-04-25T14:51:17.064 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:17.064 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:17.064 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:17.064 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.064 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-04-25T14:51:17.064 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:17.064 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:17.065 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:17.065 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.065 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-04-25T14:51:17.065 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:17.065 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:17.065 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:17.065 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.065 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-04-25T14:51:17.065 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:17.066 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:17.066 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:17.066 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.066 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-04-25T14:51:17.066 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:17.066 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:17.066 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:17.066 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.066 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-04-25T14:51:17.067 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:17.067 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:17.067 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:17.067 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.067 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-04-25T14:51:17.067 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-04-25T14:51:17.067 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-04-25T14:51:17.067 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:17.068 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:17.068 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:17.068 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T14:51:17.068 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.068 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-04-25T14:51:17.068 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 2, 2024-04-25T14:51:17.068 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 5, 2024-04-25T14:51:17.068 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 3, 2024-04-25T14:51:17.068 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 4, 2024-04-25T14:51:17.069 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-04-25T14:51:17.069 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 0, 2024-04-25T14:51:17.069 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 0, 2024-04-25T14:51:17.069 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 3605, 2024-04-25T14:51:17.069 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-04-25T14:51:17.069 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.069 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-04-25T14:51:17.069 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:51:17.069 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-04-25T14:51:17.070 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:51:17.070 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:51:17.070 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:51:17.070 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:51:17.070 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:51:17.070 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:51:17.070 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:51:17.070 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:51:17.070 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:51:17.071 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:51:17.071 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:51:17.071 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:51:17.071 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:51:17.071 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-04-25T14:51:17.071 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:51:17.071 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-04-25T14:51:17.071 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:51:17.071 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:51:17.072 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:51:17.072 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:51:17.072 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:51:17.072 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:51:17.072 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:51:17.072 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-04-25T14:51:17.072 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:51:17.072 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:51:17.072 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:51:17.073 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-04-25T14:51:17.073 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-04-25T14:51:17.073 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-04-25T14:51:17.073 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:51:17.073 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:51:17.073 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:51:17.073 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:51:17.073 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 189371, 2024-04-25T14:51:17.073 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 51, 2024-04-25T14:51:17.074 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-04-25T14:51:17.074 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-04-25T14:51:17.074 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-04-25T14:51:17.074 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-04-25T14:51:17.074 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:51:17.074 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-04-25T14:51:17.074 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:51:17.074 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-04-25T14:51:17.075 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 8360, 2024-04-25T14:51:17.075 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 176, 2024-04-25T14:51:17.075 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:51:17.075 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:51:17.075 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-04-25T14:51:17.075 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-04-25T14:51:17.075 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 47746, 2024-04-25T14:51:17.075 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 220, 2024-04-25T14:51:17.075 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-04-25T14:51:17.076 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-04-25T14:51:17.076 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-04-25T14:51:17.076 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-04-25T14:51:17.076 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.076 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-04-25T14:51:17.076 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-04-25T14:51:17.076 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-04-25T14:51:17.076 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 2379, 2024-04-25T14:51:17.076 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 26, 2024-04-25T14:51:17.077 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 4, 2024-04-25T14:51:17.077 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 2379, 2024-04-25T14:51:17.077 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-04-25T14:51:17.077 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2379, 2024-04-25T14:51:17.077 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 1.280964428, 2024-04-25T14:51:17.077 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000538446 2024-04-25T14:51:17.077 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.077 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-04-25T14:51:17.077 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-04-25T14:51:17.078 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2379, 2024-04-25T14:51:17.078 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 2380 2024-04-25T14:51:17.078 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.078 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 2379, 2024-04-25T14:51:17.078 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 2379, 2024-04-25T14:51:17.078 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 0, 2024-04-25T14:51:17.078 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-04-25T14:51:17.078 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-04-25T14:51:17.078 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 1540, 2024-04-25T14:51:17.079 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 0, 2024-04-25T14:51:17.079 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 836, 2024-04-25T14:51:17.079 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 0, 2024-04-25T14:51:17.079 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 0, 2024-04-25T14:51:17.079 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-04-25T14:51:17.079 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-04-25T14:51:17.079 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2024-04-25T14:51:17.079 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 0, 2024-04-25T14:51:17.079 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 0, 2024-04-25T14:51:17.080 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-04-25T14:51:17.080 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-04-25T14:51:17.080 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-04-25T14:51:17.080 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 2, 2024-04-25T14:51:17.080 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 0, 2024-04-25T14:51:17.080 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-04-25T14:51:17.080 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-04-25T14:51:17.080 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-04-25T14:51:17.080 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-04-25T14:51:17.081 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-04-25T14:51:17.081 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-04-25T14:51:17.081 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T14:51:17.081 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-04-25T14:51:17.081 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-04-25T14:51:17.081 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 0, 2024-04-25T14:51:17.081 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-04-25T14:51:17.081 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-04-25T14:51:17.081 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-04-25T14:51:17.082 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-04-25T14:51:17.082 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-04-25T14:51:17.082 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-04-25T14:51:17.082 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-04-25T14:51:17.082 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-04-25T14:51:17.082 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-04-25T14:51:17.082 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-04-25T14:51:17.082 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-04-25T14:51:17.083 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-04-25T14:51:17.083 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-04-25T14:51:17.083 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-04-25T14:51:17.083 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-04-25T14:51:17.083 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-04-25T14:51:17.083 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 102, 2024-04-25T14:51:17.083 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-04-25T14:51:17.083 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 14, 2024-04-25T14:51:17.083 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 5, 2024-04-25T14:51:17.084 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 5, 2024-04-25T14:51:17.084 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 0, 2024-04-25T14:51:17.084 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-04-25T14:51:17.084 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 0, 2024-04-25T14:51:17.084 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 2, 2024-04-25T14:51:17.084 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 0 2024-04-25T14:51:17.084 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.084 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-04-25T14:51:17.084 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 0, 2024-04-25T14:51:17.085 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-04-25T14:51:17.085 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-04-25T14:51:17.085 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-04-25T14:51:17.085 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.085 INFO:teuthology.orchestra.run.smithi183.stdout: "osd-slow-ops": { 2024-04-25T14:51:17.085 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 0 2024-04-25T14:51:17.085 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.085 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-04-25T14:51:17.085 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-04-25T14:51:17.086 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T14:51:17.086 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-04-25T14:51:17.086 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 0, 2024-04-25T14:51:17.086 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 0, 2024-04-25T14:51:17.086 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 0, 2024-04-25T14:51:17.086 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-04-25T14:51:17.086 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.086 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:51:17.087 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-25T14:51:17.087 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-04-25T14:51:17.087 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-25T14:51:17.087 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 2632, 2024-04-25T14:51:17.087 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 2271627, 2024-04-25T14:51:17.087 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-25T14:51:17.087 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 2632, 2024-04-25T14:51:17.087 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-25T14:51:17.087 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-25T14:51:17.088 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 2632, 2024-04-25T14:51:17.088 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 2271627, 2024-04-25T14:51:17.088 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-25T14:51:17.088 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:17.088 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:17.088 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:17.088 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T14:51:17.088 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.088 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-04-25T14:51:17.089 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-25T14:51:17.089 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-04-25T14:51:17.089 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-25T14:51:17.089 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-25T14:51:17.089 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-25T14:51:17.089 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-25T14:51:17.089 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-25T14:51:17.089 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 2379, 2024-04-25T14:51:17.089 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 94754, 2024-04-25T14:51:17.090 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 2379, 2024-04-25T14:51:17.090 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 94754, 2024-04-25T14:51:17.090 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-25T14:51:17.090 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:17.090 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:17.090 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:17.090 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T14:51:17.090 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.090 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-04-25T14:51:17.091 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-25T14:51:17.091 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-04-25T14:51:17.091 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-25T14:51:17.091 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-25T14:51:17.091 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-25T14:51:17.091 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-25T14:51:17.091 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-25T14:51:17.091 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 2379, 2024-04-25T14:51:17.092 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 2379, 2024-04-25T14:51:17.092 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 2379, 2024-04-25T14:51:17.092 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 2379, 2024-04-25T14:51:17.092 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-25T14:51:17.092 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:17.092 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:17.092 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:17.092 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T14:51:17.092 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.093 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-04-25T14:51:17.093 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-25T14:51:17.093 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-04-25T14:51:17.093 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-25T14:51:17.093 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-25T14:51:17.093 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-25T14:51:17.093 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-25T14:51:17.093 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-25T14:51:17.093 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-25T14:51:17.094 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-25T14:51:17.094 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-04-25T14:51:17.094 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-04-25T14:51:17.094 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-25T14:51:17.094 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:17.094 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:17.094 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:17.094 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T14:51:17.094 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:17.095 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x5577d9cec6e0": { 2024-04-25T14:51:17.095 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-25T14:51:17.095 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-04-25T14:51:17.095 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-25T14:51:17.095 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-25T14:51:17.095 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-25T14:51:17.095 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-25T14:51:17.095 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-25T14:51:17.095 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-25T14:51:17.096 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-25T14:51:17.096 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-04-25T14:51:17.096 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-04-25T14:51:17.096 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-25T14:51:17.096 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:17.096 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:17.096 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:17.096 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T14:51:17.096 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T14:51:17.097 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-04-25T14:51:17.820 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-25T14:51:17.820 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-25T14:51:17.821 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-04-25T14:51:17.821 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-04-25T14:51:17.821 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-04-25T14:51:17.821 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-25T14:51:17.821 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T14:51:17.821 DEBUG:tasks.check_counter:Getting stats from f 2024-04-25T14:51:17.821 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-25T14:51:17.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:17 smithi183 ceph-mon[32584]: pgmap v330: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-25T14:51:17.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:17 smithi150 ceph-mon[32504]: pgmap v330: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-25T14:51:18.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:17 smithi053 ceph-mon[27773]: pgmap v330: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-25T14:51:18.065 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:51:19.739 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:19 smithi183 ceph-mon[32584]: pgmap v331: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-25T14:51:19.740 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-04-25T14:51:19.740 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:51:19.740 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 793, 2024-04-25T14:51:19.741 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 795, 2024-04-25T14:51:19.741 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 210646, 2024-04-25T14:51:19.741 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 252238, 2024-04-25T14:51:19.741 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 6, 2024-04-25T14:51:19.741 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 2, 2024-04-25T14:51:19.741 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.086862956, 2024-04-25T14:51:19.741 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.024055376, 2024-04-25T14:51:19.741 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.024438542, 2024-04-25T14:51:19.741 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.029561086, 2024-04-25T14:51:19.742 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:51:19.742 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 793, 2024-04-25T14:51:19.742 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.033321834, 2024-04-25T14:51:19.742 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000042019 2024-04-25T14:51:19.742 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.742 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:51:19.742 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:19.742 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:19.743 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:19.743 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.743 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T14:51:19.743 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T14:51:19.743 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.743 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:51:19.743 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 698, 2024-04-25T14:51:19.743 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 862, 2024-04-25T14:51:19.743 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 171558, 2024-04-25T14:51:19.744 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 675669, 2024-04-25T14:51:19.744 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 7, 2024-04-25T14:51:19.744 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 5, 2024-04-25T14:51:19.744 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.087697462, 2024-04-25T14:51:19.744 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.025487985, 2024-04-25T14:51:19.744 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.021587168, 2024-04-25T14:51:19.744 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.032459708, 2024-04-25T14:51:19.744 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:51:19.745 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 859, 2024-04-25T14:51:19.745 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.050320280, 2024-04-25T14:51:19.745 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000058580 2024-04-25T14:51:19.745 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.745 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:51:19.745 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:19.745 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:19.745 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:19.745 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.745 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T14:51:19.746 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 454336 2024-04-25T14:51:19.746 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.746 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:51:19.746 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 1037, 2024-04-25T14:51:19.746 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 1022, 2024-04-25T14:51:19.746 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 2022720, 2024-04-25T14:51:19.746 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 387042, 2024-04-25T14:51:19.746 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 3, 2024-04-25T14:51:19.747 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 2, 2024-04-25T14:51:19.747 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.144721761, 2024-04-25T14:51:19.747 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.043191743, 2024-04-25T14:51:19.747 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.062963187, 2024-04-25T14:51:19.747 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.038775858, 2024-04-25T14:51:19.747 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:51:19.747 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1022, 2024-04-25T14:51:19.747 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.058541014, 2024-04-25T14:51:19.747 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000057280 2024-04-25T14:51:19.748 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.748 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:51:19.748 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:19.748 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:19.748 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:19.748 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.748 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 1759632, 2024-04-25T14:51:19.748 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 138048 2024-04-25T14:51:19.749 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.749 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-04-25T14:51:19.749 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-04-25T14:51:19.749 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-04-25T14:51:19.749 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.749 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-04-25T14:51:19.749 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-04-25T14:51:19.749 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-04-25T14:51:19.749 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3795, 2024-04-25T14:51:19.750 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.121771404, 2024-04-25T14:51:19.750 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000032087 2024-04-25T14:51:19.750 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T14:51:19.750 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.750 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-04-25T14:51:19.750 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-04-25T14:51:19.750 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-04-25T14:51:19.750 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:19.751 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:19.751 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:19.751 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T14:51:19.751 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.751 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-04-25T14:51:19.751 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 0, 2024-04-25T14:51:19.751 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 0, 2024-04-25T14:51:19.751 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-04-25T14:51:19.752 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:19.752 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:19.752 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:19.752 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.752 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-04-25T14:51:19.752 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 0, 2024-04-25T14:51:19.752 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 0, 2024-04-25T14:51:19.752 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2024-04-25T14:51:19.753 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 0, 2024-04-25T14:51:19.753 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 0, 2024-04-25T14:51:19.753 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2024-04-25T14:51:19.753 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 0, 2024-04-25T14:51:19.753 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 0, 2024-04-25T14:51:19.753 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 0, 2024-04-25T14:51:19.753 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 0, 2024-04-25T14:51:19.753 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 0, 2024-04-25T14:51:19.753 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 0, 2024-04-25T14:51:19.754 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 0, 2024-04-25T14:51:19.754 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 0, 2024-04-25T14:51:19.754 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 1, 2024-04-25T14:51:19.754 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 0, 2024-04-25T14:51:19.754 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 0, 2024-04-25T14:51:19.754 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 0, 2024-04-25T14:51:19.754 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 0, 2024-04-25T14:51:19.754 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2024-04-25T14:51:19.755 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2024-04-25T14:51:19.755 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 0, 2024-04-25T14:51:19.755 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2024-04-25T14:51:19.755 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-04-25T14:51:19.755 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 0, 2024-04-25T14:51:19.755 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 0, 2024-04-25T14:51:19.755 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 0, 2024-04-25T14:51:19.755 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 0, 2024-04-25T14:51:19.755 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-04-25T14:51:19.756 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 0, 2024-04-25T14:51:19.756 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2024-04-25T14:51:19.756 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 0, 2024-04-25T14:51:19.756 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 0, 2024-04-25T14:51:19.756 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 0, 2024-04-25T14:51:19.756 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T14:51:19.756 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-04-25T14:51:19.756 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T14:51:19.756 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-04-25T14:51:19.757 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 0, 2024-04-25T14:51:19.757 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T14:51:19.757 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T14:51:19.757 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 0, 2024-04-25T14:51:19.757 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 0, 2024-04-25T14:51:19.757 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T14:51:19.757 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 0, 2024-04-25T14:51:19.757 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T14:51:19.758 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T14:51:19.758 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T14:51:19.758 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 0, 2024-04-25T14:51:19.758 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 0, 2024-04-25T14:51:19.758 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 0, 2024-04-25T14:51:19.758 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 0 2024-04-25T14:51:19.758 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.758 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-04-25T14:51:19.759 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 0, 2024-04-25T14:51:19.759 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 0, 2024-04-25T14:51:19.759 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 0, 2024-04-25T14:51:19.759 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 0, 2024-04-25T14:51:19.759 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 0, 2024-04-25T14:51:19.759 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 0, 2024-04-25T14:51:19.759 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-04-25T14:51:19.760 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-04-25T14:51:19.760 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 0, 2024-04-25T14:51:19.760 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 0, 2024-04-25T14:51:19.760 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 0, 2024-04-25T14:51:19.760 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2024-04-25T14:51:19.760 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-04-25T14:51:19.761 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-04-25T14:51:19.761 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-04-25T14:51:19.761 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-04-25T14:51:19.761 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-04-25T14:51:19.761 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 0, 2024-04-25T14:51:19.761 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 0, 2024-04-25T14:51:19.761 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T14:51:19.761 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 0, 2024-04-25T14:51:19.761 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-04-25T14:51:19.762 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 0, 2024-04-25T14:51:19.762 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-04-25T14:51:19.762 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0 2024-04-25T14:51:19.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.762 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-04-25T14:51:19.762 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2024-04-25T14:51:19.762 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 0, 2024-04-25T14:51:19.762 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 0, 2024-04-25T14:51:19.763 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 0, 2024-04-25T14:51:19.763 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 8, 2024-04-25T14:51:19.763 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-04-25T14:51:19.763 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 0, 2024-04-25T14:51:19.763 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 0, 2024-04-25T14:51:19.763 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 0, 2024-04-25T14:51:19.763 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 0, 2024-04-25T14:51:19.763 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 2, 2024-04-25T14:51:19.764 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 2, 2024-04-25T14:51:19.764 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-04-25T14:51:19.764 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 0, 2024-04-25T14:51:19.764 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 4194304, 2024-04-25T14:51:19.764 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 4196657, 2024-04-25T14:51:19.764 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 4196516, 2024-04-25T14:51:19.764 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-04-25T14:51:19.765 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:19.765 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:19.765 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:19.765 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.765 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 8 2024-04-25T14:51:19.765 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.765 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-04-25T14:51:19.765 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 3, 2024-04-25T14:51:19.765 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 3, 2024-04-25T14:51:19.766 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 0, 2024-04-25T14:51:19.766 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 1, 2024-04-25T14:51:19.766 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 1, 2024-04-25T14:51:19.766 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 0, 2024-04-25T14:51:19.766 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 0, 2024-04-25T14:51:19.766 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 0, 2024-04-25T14:51:19.766 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 0, 2024-04-25T14:51:19.766 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 0, 2024-04-25T14:51:19.767 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 0, 2024-04-25T14:51:19.767 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 0, 2024-04-25T14:51:19.767 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 52632, 2024-04-25T14:51:19.767 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 215416 2024-04-25T14:51:19.767 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.767 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-04-25T14:51:19.767 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 0, 2024-04-25T14:51:19.767 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 0, 2024-04-25T14:51:19.768 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 0, 2024-04-25T14:51:19.768 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 0, 2024-04-25T14:51:19.768 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 0, 2024-04-25T14:51:19.768 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-04-25T14:51:19.768 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:19.768 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:19.768 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:19.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.768 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-04-25T14:51:19.769 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:19.769 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:19.769 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:19.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.769 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-04-25T14:51:19.769 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:19.769 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:19.769 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:19.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.770 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-04-25T14:51:19.770 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:19.770 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:19.770 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:19.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.770 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-04-25T14:51:19.770 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:19.770 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:19.771 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:19.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.771 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-04-25T14:51:19.771 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:19.771 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:19.771 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:19.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.771 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-04-25T14:51:19.771 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:19.772 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:19.772 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:19.772 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.772 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-04-25T14:51:19.772 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:19.772 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:19.772 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:19.772 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.773 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-04-25T14:51:19.773 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:19.773 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:19.773 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:19.773 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.773 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-04-25T14:51:19.773 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:19.773 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:19.773 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:19.774 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.774 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-04-25T14:51:19.774 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:19.774 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:19.774 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:19.774 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.774 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-04-25T14:51:19.774 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:19.775 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:19.775 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:19.776 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.776 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-04-25T14:51:19.776 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:19.776 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:19.776 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:19.776 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.776 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-04-25T14:51:19.776 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:19.777 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:19.777 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:19.777 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.777 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-04-25T14:51:19.777 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:19.777 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:19.777 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:19.777 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.777 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-04-25T14:51:19.778 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:19.778 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:19.778 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:19.778 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.778 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-04-25T14:51:19.778 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:19.778 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:19.778 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:19.779 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.779 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-04-25T14:51:19.782 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:19.782 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:19.782 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:19.782 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.782 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-04-25T14:51:19.782 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:19.782 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:19.783 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:19.783 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.783 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-04-25T14:51:19.783 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:19.783 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:19.783 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:19.783 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.783 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-04-25T14:51:19.784 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:19.784 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:19.784 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:19.784 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.784 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-04-25T14:51:19.784 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:19.784 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:19.784 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:19.785 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.785 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-04-25T14:51:19.785 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:19.785 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:19.785 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:19.785 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.785 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-04-25T14:51:19.785 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:19.785 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:19.786 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:19.786 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.786 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-04-25T14:51:19.786 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:19.786 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:19.786 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:19.786 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.786 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-04-25T14:51:19.787 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:19.787 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:19.787 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:19.787 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.787 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-04-25T14:51:19.787 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:19.787 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:19.787 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:19.787 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.788 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-04-25T14:51:19.788 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:19.788 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:19.788 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:19.788 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.788 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-04-25T14:51:19.788 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:19.788 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:19.789 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:19.789 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.789 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-04-25T14:51:19.789 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-04-25T14:51:19.789 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-04-25T14:51:19.789 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:19.789 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:19.789 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:19.789 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T14:51:19.790 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.790 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-04-25T14:51:19.790 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 2, 2024-04-25T14:51:19.790 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 4, 2024-04-25T14:51:19.790 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 2, 2024-04-25T14:51:19.790 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 3, 2024-04-25T14:51:19.790 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-04-25T14:51:19.790 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 0, 2024-04-25T14:51:19.791 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 0, 2024-04-25T14:51:19.791 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 2864, 2024-04-25T14:51:19.791 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-04-25T14:51:19.791 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.791 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-04-25T14:51:19.791 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:51:19.791 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-04-25T14:51:19.791 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:51:19.792 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:51:19.792 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:51:19.792 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:51:19.792 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:51:19.792 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:51:19.792 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:51:19.792 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:51:19.792 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:51:19.792 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:51:19.793 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:51:19.793 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:51:19.793 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:51:19.793 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-04-25T14:51:19.793 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:51:19.793 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-04-25T14:51:19.793 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:51:19.793 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:51:19.794 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:51:19.794 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:51:19.794 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:51:19.794 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:51:19.794 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:51:19.794 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-04-25T14:51:19.794 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:51:19.794 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:51:19.795 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:51:19.795 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-04-25T14:51:19.795 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-04-25T14:51:19.795 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-04-25T14:51:19.795 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:51:19.795 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:51:19.795 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:51:19.795 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:51:19.796 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 189411, 2024-04-25T14:51:19.796 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 52, 2024-04-25T14:51:19.796 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-04-25T14:51:19.796 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-04-25T14:51:19.796 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-04-25T14:51:19.796 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-04-25T14:51:19.796 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:51:19.796 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-04-25T14:51:19.796 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:51:19.797 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-04-25T14:51:19.797 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 8360, 2024-04-25T14:51:19.797 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 176, 2024-04-25T14:51:19.797 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:51:19.797 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:51:19.797 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-04-25T14:51:19.797 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-04-25T14:51:19.797 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 8512, 2024-04-25T14:51:19.797 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 13, 2024-04-25T14:51:19.798 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-04-25T14:51:19.798 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-04-25T14:51:19.798 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-04-25T14:51:19.798 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-04-25T14:51:19.798 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.798 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-04-25T14:51:19.798 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-04-25T14:51:19.798 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-04-25T14:51:19.799 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 2272, 2024-04-25T14:51:19.799 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 21, 2024-04-25T14:51:19.799 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 3, 2024-04-25T14:51:19.799 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 2272, 2024-04-25T14:51:19.799 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-04-25T14:51:19.799 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2272, 2024-04-25T14:51:19.799 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 1.944945985, 2024-04-25T14:51:19.799 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000856050 2024-04-25T14:51:19.800 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.800 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-04-25T14:51:19.800 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-04-25T14:51:19.800 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2272, 2024-04-25T14:51:19.800 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 2273 2024-04-25T14:51:19.800 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.800 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 2272, 2024-04-25T14:51:19.800 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 2272, 2024-04-25T14:51:19.800 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 0, 2024-04-25T14:51:19.801 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-04-25T14:51:19.801 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-04-25T14:51:19.801 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 1477, 2024-04-25T14:51:19.801 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 0, 2024-04-25T14:51:19.801 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 793, 2024-04-25T14:51:19.801 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 0, 2024-04-25T14:51:19.801 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 0, 2024-04-25T14:51:19.801 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-04-25T14:51:19.801 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-04-25T14:51:19.802 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2024-04-25T14:51:19.802 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 0, 2024-04-25T14:51:19.802 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 0, 2024-04-25T14:51:19.802 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-04-25T14:51:19.802 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-04-25T14:51:19.802 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-04-25T14:51:19.802 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 1, 2024-04-25T14:51:19.802 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 0, 2024-04-25T14:51:19.803 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-04-25T14:51:19.803 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-04-25T14:51:19.803 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-04-25T14:51:19.803 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-04-25T14:51:19.803 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-04-25T14:51:19.803 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-04-25T14:51:19.803 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T14:51:19.803 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-04-25T14:51:19.803 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-04-25T14:51:19.803 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 0, 2024-04-25T14:51:19.804 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-04-25T14:51:19.804 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-04-25T14:51:19.804 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-04-25T14:51:19.804 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-04-25T14:51:19.804 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-04-25T14:51:19.804 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-04-25T14:51:19.804 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-04-25T14:51:19.804 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-04-25T14:51:19.804 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-04-25T14:51:19.805 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-04-25T14:51:19.805 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-04-25T14:51:19.805 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-04-25T14:51:19.805 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-04-25T14:51:19.805 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-04-25T14:51:19.805 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-04-25T14:51:19.805 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-04-25T14:51:19.805 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 102, 2024-04-25T14:51:19.805 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-04-25T14:51:19.806 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 14, 2024-04-25T14:51:19.806 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 5, 2024-04-25T14:51:19.806 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 5, 2024-04-25T14:51:19.806 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 0, 2024-04-25T14:51:19.806 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-04-25T14:51:19.806 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 0, 2024-04-25T14:51:19.806 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 2, 2024-04-25T14:51:19.806 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 0 2024-04-25T14:51:19.807 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.807 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-04-25T14:51:19.807 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 0, 2024-04-25T14:51:19.807 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-04-25T14:51:19.807 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-04-25T14:51:19.807 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-04-25T14:51:19.807 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.807 INFO:teuthology.orchestra.run.smithi183.stdout: "osd-slow-ops": { 2024-04-25T14:51:19.808 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 0 2024-04-25T14:51:19.808 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.808 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-04-25T14:51:19.808 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-04-25T14:51:19.808 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T14:51:19.808 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-04-25T14:51:19.808 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 0, 2024-04-25T14:51:19.808 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 0, 2024-04-25T14:51:19.809 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 0, 2024-04-25T14:51:19.809 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-04-25T14:51:19.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.809 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:51:19.809 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-25T14:51:19.809 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-04-25T14:51:19.809 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-25T14:51:19.809 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 2524, 2024-04-25T14:51:19.809 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 2161977, 2024-04-25T14:51:19.810 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-25T14:51:19.810 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 2524, 2024-04-25T14:51:19.810 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-25T14:51:19.810 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-25T14:51:19.810 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 2524, 2024-04-25T14:51:19.810 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 2161977, 2024-04-25T14:51:19.810 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-25T14:51:19.811 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:19.811 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:19.811 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:19.811 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T14:51:19.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.811 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-04-25T14:51:19.811 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-25T14:51:19.811 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-04-25T14:51:19.811 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-25T14:51:19.812 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-25T14:51:19.812 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-25T14:51:19.812 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-25T14:51:19.812 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-25T14:51:19.812 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 2272, 2024-04-25T14:51:19.812 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 2358, 2024-04-25T14:51:19.812 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 2272, 2024-04-25T14:51:19.812 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 2358, 2024-04-25T14:51:19.812 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-25T14:51:19.813 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:19.813 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:19.813 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:19.813 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T14:51:19.813 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.813 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-04-25T14:51:19.813 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-25T14:51:19.813 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-04-25T14:51:19.813 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-25T14:51:19.814 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-25T14:51:19.814 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-25T14:51:19.814 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-25T14:51:19.814 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-25T14:51:19.814 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 2272, 2024-04-25T14:51:19.814 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 2272, 2024-04-25T14:51:19.814 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 2272, 2024-04-25T14:51:19.814 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 2272, 2024-04-25T14:51:19.815 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-25T14:51:19.815 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:19.815 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:19.815 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:19.815 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T14:51:19.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.815 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-04-25T14:51:19.815 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-25T14:51:19.815 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-04-25T14:51:19.816 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-25T14:51:19.816 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-25T14:51:19.816 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-25T14:51:19.816 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-25T14:51:19.816 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-25T14:51:19.816 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-25T14:51:19.816 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-25T14:51:19.816 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-04-25T14:51:19.816 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-04-25T14:51:19.817 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-25T14:51:19.817 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:19.817 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:19.817 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:19.817 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T14:51:19.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:19.817 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x55d910d166e0": { 2024-04-25T14:51:19.817 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-25T14:51:19.817 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-04-25T14:51:19.818 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-25T14:51:19.818 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-25T14:51:19.818 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-25T14:51:19.818 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-25T14:51:19.818 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-25T14:51:19.818 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-25T14:51:19.818 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-25T14:51:19.818 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-04-25T14:51:19.819 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-04-25T14:51:19.819 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-25T14:51:19.819 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:19.819 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:19.819 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:19.819 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T14:51:19.819 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T14:51:19.819 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-04-25T14:51:19.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:19 smithi150 ceph-mon[32504]: pgmap v331: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-25T14:51:20.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:19 smithi053 ceph-mon[27773]: pgmap v331: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-25T14:51:20.511 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-25T14:51:20.511 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-25T14:51:20.511 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-04-25T14:51:20.512 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-04-25T14:51:20.512 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-04-25T14:51:20.512 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-25T14:51:20.512 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T14:51:20.512 DEBUG:tasks.check_counter:Getting stats from i 2024-04-25T14:51:20.512 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-25T14:51:20.750 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:51:21.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:21 smithi183 ceph-mon[32584]: pgmap v332: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-25T14:51:21.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:21 smithi183 ceph-mon[32584]: osdmap e103: 12 total, 12 up, 12 in 2024-04-25T14:51:21.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:21 smithi150 ceph-mon[32504]: pgmap v332: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-25T14:51:21.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:21 smithi150 ceph-mon[32504]: osdmap e103: 12 total, 12 up, 12 in 2024-04-25T14:51:21.964 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:21 smithi053 ceph-mon[27773]: pgmap v332: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-25T14:51:21.964 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:21 smithi053 ceph-mon[27773]: osdmap e103: 12 total, 12 up, 12 in 2024-04-25T14:51:22.374 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-04-25T14:51:22.374 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:51:22.374 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 662, 2024-04-25T14:51:22.374 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 830, 2024-04-25T14:51:22.374 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 1881754, 2024-04-25T14:51:22.375 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 277080, 2024-04-25T14:51:22.375 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 8, 2024-04-25T14:51:22.375 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 3, 2024-04-25T14:51:22.375 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.106006481, 2024-04-25T14:51:22.375 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.038477451, 2024-04-25T14:51:22.375 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.063756231, 2024-04-25T14:51:22.375 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.010869327, 2024-04-25T14:51:22.375 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:51:22.376 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 831, 2024-04-25T14:51:22.376 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.069585389, 2024-04-25T14:51:22.376 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000083736 2024-04-25T14:51:22.376 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.376 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:51:22.376 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 908, 2024-04-25T14:51:22.376 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000257214, 2024-04-25T14:51:22.376 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000283 2024-04-25T14:51:22.376 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.377 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 1767392, 2024-04-25T14:51:22.377 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 141136 2024-04-25T14:51:22.377 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.377 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:51:22.377 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 25, 2024-04-25T14:51:22.377 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 191, 2024-04-25T14:51:22.377 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 5704, 2024-04-25T14:51:22.377 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 588759, 2024-04-25T14:51:22.377 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 16, 2024-04-25T14:51:22.378 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 4, 2024-04-25T14:51:22.378 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.014490752, 2024-04-25T14:51:22.378 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.006872328, 2024-04-25T14:51:22.378 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.005342166, 2024-04-25T14:51:22.378 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000288495, 2024-04-25T14:51:22.378 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:51:22.378 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 187, 2024-04-25T14:51:22.378 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.027888584, 2024-04-25T14:51:22.379 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000149136 2024-04-25T14:51:22.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.379 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:51:22.379 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 20, 2024-04-25T14:51:22.379 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000002536, 2024-04-25T14:51:22.379 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000126 2024-04-25T14:51:22.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.379 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 2336, 2024-04-25T14:51:22.379 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 536208 2024-04-25T14:51:22.380 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.380 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:51:22.380 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 20, 2024-04-25T14:51:22.380 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 22, 2024-04-25T14:51:22.380 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 7355, 2024-04-25T14:51:22.380 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 13319, 2024-04-25T14:51:22.380 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 7, 2024-04-25T14:51:22.380 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 5, 2024-04-25T14:51:22.380 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.003545414, 2024-04-25T14:51:22.381 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.000448619, 2024-04-25T14:51:22.381 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.002463680, 2024-04-25T14:51:22.381 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000451335, 2024-04-25T14:51:22.381 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:51:22.381 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 19, 2024-04-25T14:51:22.381 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.021597433, 2024-04-25T14:51:22.381 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001136707 2024-04-25T14:51:22.381 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.381 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:51:22.382 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:22.382 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:22.382 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:22.382 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.382 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-25T14:51:22.382 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T14:51:22.382 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.382 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-04-25T14:51:22.383 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-04-25T14:51:22.383 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-04-25T14:51:22.383 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.383 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-04-25T14:51:22.383 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-04-25T14:51:22.383 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-04-25T14:51:22.383 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 58, 2024-04-25T14:51:22.383 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.001821221, 2024-04-25T14:51:22.383 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000031400 2024-04-25T14:51:22.384 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T14:51:22.384 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.384 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-04-25T14:51:22.384 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-04-25T14:51:22.384 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-04-25T14:51:22.384 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2, 2024-04-25T14:51:22.384 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000016279, 2024-04-25T14:51:22.384 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000008139 2024-04-25T14:51:22.384 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T14:51:22.384 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.385 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-04-25T14:51:22.385 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 0, 2024-04-25T14:51:22.385 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 0, 2024-04-25T14:51:22.385 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-04-25T14:51:22.385 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:22.385 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:22.385 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:22.385 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.385 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-04-25T14:51:22.386 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 0, 2024-04-25T14:51:22.386 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 0, 2024-04-25T14:51:22.386 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2024-04-25T14:51:22.386 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 11, 2024-04-25T14:51:22.386 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 0, 2024-04-25T14:51:22.386 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2024-04-25T14:51:22.386 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 14, 2024-04-25T14:51:22.386 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 1, 2024-04-25T14:51:22.387 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 13, 2024-04-25T14:51:22.387 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 0, 2024-04-25T14:51:22.387 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 12, 2024-04-25T14:51:22.387 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 0, 2024-04-25T14:51:22.387 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 0, 2024-04-25T14:51:22.387 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 0, 2024-04-25T14:51:22.387 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 2, 2024-04-25T14:51:22.387 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 5, 2024-04-25T14:51:22.387 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 3, 2024-04-25T14:51:22.388 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 0, 2024-04-25T14:51:22.388 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 2, 2024-04-25T14:51:22.388 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2024-04-25T14:51:22.388 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2024-04-25T14:51:22.388 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 0, 2024-04-25T14:51:22.388 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2024-04-25T14:51:22.388 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-04-25T14:51:22.388 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 0, 2024-04-25T14:51:22.388 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 0, 2024-04-25T14:51:22.388 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 0, 2024-04-25T14:51:22.389 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 0, 2024-04-25T14:51:22.389 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-04-25T14:51:22.389 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 0, 2024-04-25T14:51:22.389 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2024-04-25T14:51:22.389 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 4, 2024-04-25T14:51:22.389 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 234, 2024-04-25T14:51:22.389 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 0, 2024-04-25T14:51:22.389 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T14:51:22.389 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-04-25T14:51:22.390 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T14:51:22.390 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-04-25T14:51:22.390 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 0, 2024-04-25T14:51:22.390 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T14:51:22.390 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T14:51:22.390 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 0, 2024-04-25T14:51:22.391 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 0, 2024-04-25T14:51:22.391 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T14:51:22.391 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 0, 2024-04-25T14:51:22.391 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T14:51:22.391 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T14:51:22.391 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T14:51:22.391 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 0, 2024-04-25T14:51:22.391 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 0, 2024-04-25T14:51:22.392 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 0, 2024-04-25T14:51:22.392 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 0 2024-04-25T14:51:22.392 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.392 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-04-25T14:51:22.392 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 0, 2024-04-25T14:51:22.392 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 8, 2024-04-25T14:51:22.392 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 5, 2024-04-25T14:51:22.392 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 6, 2024-04-25T14:51:22.392 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 0, 2024-04-25T14:51:22.393 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 0, 2024-04-25T14:51:22.393 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-04-25T14:51:22.393 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-04-25T14:51:22.393 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 0, 2024-04-25T14:51:22.393 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 0, 2024-04-25T14:51:22.393 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 0, 2024-04-25T14:51:22.393 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2024-04-25T14:51:22.393 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-04-25T14:51:22.393 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-04-25T14:51:22.394 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-04-25T14:51:22.394 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-04-25T14:51:22.394 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-04-25T14:51:22.394 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 0, 2024-04-25T14:51:22.394 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 0, 2024-04-25T14:51:22.394 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T14:51:22.394 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 0, 2024-04-25T14:51:22.394 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-04-25T14:51:22.395 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 0, 2024-04-25T14:51:22.395 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-04-25T14:51:22.395 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0 2024-04-25T14:51:22.395 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.395 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-04-25T14:51:22.395 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2024-04-25T14:51:22.395 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 6, 2024-04-25T14:51:22.395 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 0, 2024-04-25T14:51:22.395 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 0, 2024-04-25T14:51:22.396 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 6, 2024-04-25T14:51:22.396 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-04-25T14:51:22.396 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 0, 2024-04-25T14:51:22.396 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 2, 2024-04-25T14:51:22.396 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 0, 2024-04-25T14:51:22.396 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 0, 2024-04-25T14:51:22.396 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 2, 2024-04-25T14:51:22.396 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 2, 2024-04-25T14:51:22.396 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-04-25T14:51:22.397 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 0, 2024-04-25T14:51:22.397 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 4194304, 2024-04-25T14:51:22.397 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 4195956, 2024-04-25T14:51:22.397 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 0, 2024-04-25T14:51:22.397 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-04-25T14:51:22.397 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 5, 2024-04-25T14:51:22.397 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.008728417, 2024-04-25T14:51:22.397 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001745683 2024-04-25T14:51:22.398 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.398 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 0 2024-04-25T14:51:22.398 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.398 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-04-25T14:51:22.398 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 17, 2024-04-25T14:51:22.398 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 17, 2024-04-25T14:51:22.398 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 0, 2024-04-25T14:51:22.398 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 16, 2024-04-25T14:51:22.399 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 17, 2024-04-25T14:51:22.399 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 1, 2024-04-25T14:51:22.399 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 14, 2024-04-25T14:51:22.399 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 14, 2024-04-25T14:51:22.399 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 0, 2024-04-25T14:51:22.399 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 0, 2024-04-25T14:51:22.399 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 0, 2024-04-25T14:51:22.399 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 0, 2024-04-25T14:51:22.399 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 54804, 2024-04-25T14:51:22.399 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 223608 2024-04-25T14:51:22.400 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.400 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-04-25T14:51:22.400 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 0, 2024-04-25T14:51:22.400 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 0, 2024-04-25T14:51:22.400 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 0, 2024-04-25T14:51:22.400 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 8, 2024-04-25T14:51:22.400 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 0, 2024-04-25T14:51:22.400 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-04-25T14:51:22.400 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:22.401 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:22.401 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:22.401 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.401 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-04-25T14:51:22.401 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:22.401 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:22.401 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:22.401 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.401 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-04-25T14:51:22.402 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:22.402 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:22.402 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:22.402 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.402 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-04-25T14:51:22.402 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:22.402 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:22.402 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:22.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.403 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-04-25T14:51:22.403 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:22.403 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:22.403 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:22.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.403 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-04-25T14:51:22.403 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:22.403 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:22.404 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:22.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.404 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-04-25T14:51:22.404 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:22.404 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:22.404 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:22.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.404 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-04-25T14:51:22.404 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:22.405 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:22.405 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:22.405 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.405 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-04-25T14:51:22.405 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:22.405 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:22.405 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:22.405 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.405 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-04-25T14:51:22.406 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:22.406 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:22.406 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:22.406 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.406 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-04-25T14:51:22.406 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:22.406 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:22.406 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:22.406 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.407 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-04-25T14:51:22.407 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:22.407 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:22.407 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:22.408 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.408 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-04-25T14:51:22.408 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:22.408 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:22.408 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:22.408 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.408 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-04-25T14:51:22.408 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:22.409 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:22.409 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:22.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.409 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-04-25T14:51:22.409 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:22.409 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:22.409 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:22.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.409 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-04-25T14:51:22.410 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:22.410 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:22.410 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:22.410 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.410 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-04-25T14:51:22.410 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:22.410 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:22.410 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:22.411 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.411 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-04-25T14:51:22.411 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:22.411 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:22.411 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:22.411 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.411 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-04-25T14:51:22.411 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:22.411 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:22.412 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:22.412 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.412 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-04-25T14:51:22.412 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:22.412 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:22.412 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:22.412 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.412 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-04-25T14:51:22.413 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:22.413 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:22.413 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:22.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.413 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-04-25T14:51:22.413 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:22.413 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:22.413 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:22.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.414 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-04-25T14:51:22.414 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:22.414 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:22.414 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:22.414 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.414 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-04-25T14:51:22.414 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:22.415 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:22.415 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:22.415 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.415 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-04-25T14:51:22.415 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:22.415 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:22.415 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:22.415 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.415 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-04-25T14:51:22.416 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:22.416 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:22.416 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:22.416 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.416 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-04-25T14:51:22.416 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:22.416 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:22.416 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:22.416 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.417 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-04-25T14:51:22.417 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:22.417 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:22.417 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:22.417 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.417 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-04-25T14:51:22.417 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:22.417 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:22.417 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:22.417 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.418 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-04-25T14:51:22.418 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-04-25T14:51:22.418 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-04-25T14:51:22.418 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:22.418 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:22.418 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:22.418 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T14:51:22.418 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.419 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-04-25T14:51:22.419 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 0, 2024-04-25T14:51:22.419 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 2, 2024-04-25T14:51:22.419 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 2, 2024-04-25T14:51:22.419 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 0, 2024-04-25T14:51:22.419 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-04-25T14:51:22.419 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 0, 2024-04-25T14:51:22.419 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 0, 2024-04-25T14:51:22.419 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 88, 2024-04-25T14:51:22.419 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-04-25T14:51:22.420 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.420 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-04-25T14:51:22.420 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:51:22.420 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-04-25T14:51:22.420 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:51:22.420 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:51:22.420 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:51:22.420 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:51:22.420 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:51:22.421 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:51:22.421 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:51:22.421 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:51:22.421 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:51:22.421 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:51:22.421 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:51:22.421 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:51:22.421 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:51:22.421 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-04-25T14:51:22.422 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:51:22.422 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-04-25T14:51:22.422 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:51:22.422 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:51:22.422 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:51:22.422 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:51:22.422 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:51:22.422 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:51:22.423 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:51:22.423 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-04-25T14:51:22.423 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:51:22.423 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:51:22.423 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:51:22.423 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-04-25T14:51:22.423 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-04-25T14:51:22.423 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-04-25T14:51:22.423 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:51:22.423 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:51:22.424 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:51:22.424 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:51:22.424 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 246007, 2024-04-25T14:51:22.424 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 76, 2024-04-25T14:51:22.424 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-04-25T14:51:22.424 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-04-25T14:51:22.424 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-04-25T14:51:22.424 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-04-25T14:51:22.424 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:51:22.425 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-04-25T14:51:22.425 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:51:22.425 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-04-25T14:51:22.425 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 8360, 2024-04-25T14:51:22.425 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 176, 2024-04-25T14:51:22.425 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:51:22.425 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:51:22.425 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-04-25T14:51:22.425 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-04-25T14:51:22.426 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 69032, 2024-04-25T14:51:22.426 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 252, 2024-04-25T14:51:22.426 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-04-25T14:51:22.426 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-04-25T14:51:22.426 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-04-25T14:51:22.426 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-04-25T14:51:22.426 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.426 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-04-25T14:51:22.427 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-04-25T14:51:22.427 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-04-25T14:51:22.427 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 42, 2024-04-25T14:51:22.427 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 12044, 2024-04-25T14:51:22.427 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 28, 2024-04-25T14:51:22.427 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 42, 2024-04-25T14:51:22.427 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-04-25T14:51:22.427 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 42, 2024-04-25T14:51:22.427 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.283992144, 2024-04-25T14:51:22.428 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.006761717 2024-04-25T14:51:22.428 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.428 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-04-25T14:51:22.428 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-04-25T14:51:22.428 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 42, 2024-04-25T14:51:22.428 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 64 2024-04-25T14:51:22.428 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.428 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 42, 2024-04-25T14:51:22.428 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 0, 2024-04-25T14:51:22.429 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 42, 2024-04-25T14:51:22.429 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-04-25T14:51:22.429 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-04-25T14:51:22.429 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 0, 2024-04-25T14:51:22.429 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 10, 2024-04-25T14:51:22.429 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 0, 2024-04-25T14:51:22.429 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 5, 2024-04-25T14:51:22.429 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 8, 2024-04-25T14:51:22.429 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-04-25T14:51:22.430 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-04-25T14:51:22.430 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2024-04-25T14:51:22.430 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 0, 2024-04-25T14:51:22.430 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 6, 2024-04-25T14:51:22.430 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-04-25T14:51:22.430 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-04-25T14:51:22.430 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-04-25T14:51:22.430 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 0, 2024-04-25T14:51:22.431 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 20, 2024-04-25T14:51:22.431 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-04-25T14:51:22.431 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-04-25T14:51:22.431 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-04-25T14:51:22.431 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-04-25T14:51:22.431 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-04-25T14:51:22.431 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-04-25T14:51:22.431 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T14:51:22.431 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-04-25T14:51:22.432 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-04-25T14:51:22.432 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 0, 2024-04-25T14:51:22.432 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-04-25T14:51:22.432 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-04-25T14:51:22.432 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-04-25T14:51:22.432 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-04-25T14:51:22.432 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-04-25T14:51:22.432 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-04-25T14:51:22.432 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-04-25T14:51:22.433 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-04-25T14:51:22.433 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-04-25T14:51:22.433 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-04-25T14:51:22.433 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-04-25T14:51:22.433 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-04-25T14:51:22.433 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-04-25T14:51:22.433 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-04-25T14:51:22.433 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-04-25T14:51:22.434 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-04-25T14:51:22.434 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 103, 2024-04-25T14:51:22.434 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-04-25T14:51:22.434 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 15, 2024-04-25T14:51:22.434 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 9, 2024-04-25T14:51:22.434 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 9, 2024-04-25T14:51:22.434 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 0, 2024-04-25T14:51:22.434 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-04-25T14:51:22.434 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 14, 2024-04-25T14:51:22.435 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 0, 2024-04-25T14:51:22.435 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 1 2024-04-25T14:51:22.435 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.435 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-04-25T14:51:22.435 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 1, 2024-04-25T14:51:22.435 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-04-25T14:51:22.435 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-04-25T14:51:22.435 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-04-25T14:51:22.435 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.436 INFO:teuthology.orchestra.run.smithi183.stdout: "osd-slow-ops": { 2024-04-25T14:51:22.436 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 0 2024-04-25T14:51:22.436 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.436 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-04-25T14:51:22.436 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-04-25T14:51:22.436 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T14:51:22.436 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-04-25T14:51:22.436 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 0, 2024-04-25T14:51:22.436 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 0, 2024-04-25T14:51:22.436 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 0, 2024-04-25T14:51:22.437 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-04-25T14:51:22.437 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.437 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:51:22.437 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-25T14:51:22.437 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-04-25T14:51:22.437 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-25T14:51:22.437 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 703, 2024-04-25T14:51:22.437 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 1814095, 2024-04-25T14:51:22.437 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-25T14:51:22.438 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 703, 2024-04-25T14:51:22.438 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-25T14:51:22.438 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-25T14:51:22.438 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 703, 2024-04-25T14:51:22.438 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 1814095, 2024-04-25T14:51:22.438 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-25T14:51:22.438 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:22.438 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:22.439 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:22.439 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T14:51:22.439 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.439 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-04-25T14:51:22.439 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-25T14:51:22.442 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-04-25T14:51:22.442 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-25T14:51:22.442 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-25T14:51:22.442 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-25T14:51:22.442 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-25T14:51:22.442 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-25T14:51:22.442 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 42, 2024-04-25T14:51:22.443 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 12044, 2024-04-25T14:51:22.443 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 42, 2024-04-25T14:51:22.443 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 12044, 2024-04-25T14:51:22.443 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-25T14:51:22.443 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:22.443 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:22.443 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:22.443 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T14:51:22.443 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.444 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-04-25T14:51:22.444 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-25T14:51:22.444 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-04-25T14:51:22.444 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-25T14:51:22.444 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-25T14:51:22.444 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-25T14:51:22.444 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-25T14:51:22.444 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-25T14:51:22.444 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 42, 2024-04-25T14:51:22.444 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 42, 2024-04-25T14:51:22.445 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 42, 2024-04-25T14:51:22.445 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 42, 2024-04-25T14:51:22.445 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-25T14:51:22.445 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:22.445 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:22.445 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:22.445 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T14:51:22.445 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.446 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-04-25T14:51:22.446 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-25T14:51:22.446 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-04-25T14:51:22.446 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-25T14:51:22.446 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-25T14:51:22.446 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-25T14:51:22.446 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-25T14:51:22.446 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-25T14:51:22.446 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-25T14:51:22.447 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-25T14:51:22.447 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-04-25T14:51:22.447 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-04-25T14:51:22.447 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-25T14:51:22.447 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:22.447 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:22.447 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:22.447 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T14:51:22.447 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:22.448 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x55daae348500": { 2024-04-25T14:51:22.448 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-25T14:51:22.448 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-04-25T14:51:22.448 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-25T14:51:22.448 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 6, 2024-04-25T14:51:22.448 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 1652, 2024-04-25T14:51:22.448 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-25T14:51:22.448 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 6, 2024-04-25T14:51:22.448 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-25T14:51:22.449 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-25T14:51:22.449 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 5, 2024-04-25T14:51:22.449 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 1652, 2024-04-25T14:51:22.449 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-25T14:51:22.449 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:22.449 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:22.449 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:22.449 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T14:51:22.449 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T14:51:22.449 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-04-25T14:51:23.098 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-25T14:51:23.098 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-04-25T14:51:23.098 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-04-25T14:51:23.098 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=8 2024-04-25T14:51:23.098 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=0 2024-04-25T14:51:23.098 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-25T14:51:23.098 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T14:51:23.099 DEBUG:tasks.check_counter:Getting stats from l 2024-04-25T14:51:23.099 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-25T14:51:23.333 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:51:23.724 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:23 smithi183 ceph-mon[32584]: pgmap v334: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T14:51:23.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:23 smithi150 ceph-mon[32504]: pgmap v334: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T14:51:24.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:23 smithi053 ceph-mon[27773]: pgmap v334: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T14:51:25.064 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-04-25T14:51:25.065 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:51:25.065 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 235, 2024-04-25T14:51:25.065 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 223, 2024-04-25T14:51:25.065 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 1731760, 2024-04-25T14:51:25.065 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 137504, 2024-04-25T14:51:25.065 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 4, 2024-04-25T14:51:25.066 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 0, 2024-04-25T14:51:25.066 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.050463408, 2024-04-25T14:51:25.066 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.014573195, 2024-04-25T14:51:25.066 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.039221866, 2024-04-25T14:51:25.066 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.005915915, 2024-04-25T14:51:25.066 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:51:25.066 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 222, 2024-04-25T14:51:25.066 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.013467854, 2024-04-25T14:51:25.067 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000060666 2024-04-25T14:51:25.067 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:25.067 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:51:25.067 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:25.067 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:25.068 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:25.068 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:25.068 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 1731760, 2024-04-25T14:51:25.068 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 137504 2024-04-25T14:51:25.068 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:25.068 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:51:25.068 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 3, 2024-04-25T14:51:25.068 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 173, 2024-04-25T14:51:25.069 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 480, 2024-04-25T14:51:25.069 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 264656, 2024-04-25T14:51:25.069 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 6, 2024-04-25T14:51:25.069 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 3, 2024-04-25T14:51:25.069 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.012443118, 2024-04-25T14:51:25.069 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.008343426, 2024-04-25T14:51:25.069 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.001736663, 2024-04-25T14:51:25.069 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T14:51:25.070 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:51:25.070 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 169, 2024-04-25T14:51:25.070 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.027049496, 2024-04-25T14:51:25.070 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000160056 2024-04-25T14:51:25.070 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:25.070 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:51:25.071 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:25.071 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:25.071 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:25.071 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:25.071 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-25T14:51:25.071 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 264656 2024-04-25T14:51:25.071 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:25.071 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:51:25.072 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 4, 2024-04-25T14:51:25.072 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 4, 2024-04-25T14:51:25.072 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 2992, 2024-04-25T14:51:25.072 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 448, 2024-04-25T14:51:25.072 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 2, 2024-04-25T14:51:25.072 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 1, 2024-04-25T14:51:25.072 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.000655085, 2024-04-25T14:51:25.072 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.000029189, 2024-04-25T14:51:25.073 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.000610179, 2024-04-25T14:51:25.073 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T14:51:25.073 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:51:25.073 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3, 2024-04-25T14:51:25.073 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000742512, 2024-04-25T14:51:25.073 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000247504 2024-04-25T14:51:25.073 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:25.073 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:51:25.073 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:25.074 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:25.074 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:25.074 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:25.074 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-25T14:51:25.074 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T14:51:25.074 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:25.074 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-04-25T14:51:25.074 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 0, 2024-04-25T14:51:25.075 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-04-25T14:51:25.075 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:25.075 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-04-25T14:51:25.075 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:51:25.075 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-04-25T14:51:25.075 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:51:25.075 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:51:25.075 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:51:25.075 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:51:25.076 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:51:25.076 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:51:25.076 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:51:25.076 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:51:25.076 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:51:25.076 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:51:25.076 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:51:25.076 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:51:25.077 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:51:25.077 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-04-25T14:51:25.077 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:51:25.077 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-04-25T14:51:25.077 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:51:25.077 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:51:25.077 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:51:25.077 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:51:25.077 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:51:25.078 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:51:25.078 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:51:25.078 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-04-25T14:51:25.078 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:51:25.078 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:51:25.078 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:51:25.078 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-04-25T14:51:25.078 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-04-25T14:51:25.079 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-04-25T14:51:25.079 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:51:25.079 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:51:25.079 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:51:25.079 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:51:25.079 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 148692, 2024-04-25T14:51:25.079 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 30, 2024-04-25T14:51:25.079 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-04-25T14:51:25.079 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-04-25T14:51:25.080 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-04-25T14:51:25.080 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-04-25T14:51:25.080 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:51:25.080 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-04-25T14:51:25.080 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:51:25.080 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-04-25T14:51:25.080 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 0, 2024-04-25T14:51:25.080 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 0, 2024-04-25T14:51:25.081 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:51:25.081 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:51:25.081 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-04-25T14:51:25.081 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-04-25T14:51:25.081 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 928, 2024-04-25T14:51:25.081 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 2, 2024-04-25T14:51:25.081 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-04-25T14:51:25.081 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-04-25T14:51:25.081 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-04-25T14:51:25.082 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-04-25T14:51:25.082 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T14:51:25.082 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:51:25.082 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-25T14:51:25.082 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-04-25T14:51:25.082 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-25T14:51:25.082 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 238, 2024-04-25T14:51:25.082 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 1698653, 2024-04-25T14:51:25.083 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-25T14:51:25.083 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 238, 2024-04-25T14:51:25.083 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-25T14:51:25.083 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-25T14:51:25.083 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 238, 2024-04-25T14:51:25.083 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 1698653, 2024-04-25T14:51:25.083 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-25T14:51:25.083 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T14:51:25.083 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T14:51:25.084 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T14:51:25.084 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T14:51:25.084 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T14:51:25.084 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-04-25T14:51:25.723 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-04-25T14:51:25.723 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-04-25T14:51:25.724 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-04-25T14:51:25.724 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-04-25T14:51:25.724 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-04-25T14:51:25.724 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-04-25T14:51:25.724 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-04-25T14:51:25.724 ERROR:teuthology.run_tasks:Manager failed: check-counter Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa/tasks/check_counter.py", line 125, in end raise RuntimeError("The following counters failed to be set " RuntimeError: The following counters failed to be set on mds daemons: {'mds.imported', 'mds.exported'} 2024-04-25T14:51:25.735 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T14:51:25.747 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T14:51:25.747 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-25T14:51:25.747 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:51:25.815 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-25T14:51:25.816 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:51:25.874 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:51:25.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:25 smithi183 ceph-mon[32584]: pgmap v335: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T14:51:25.954 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:25 smithi053 ceph-mon[27773]: pgmap v335: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T14:51:25.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:25 smithi150 ceph-mon[32504]: pgmap v335: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T14:51:26.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.099+0000 7f4432400640 1 -- 172.21.15.53:0/1269826391 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f442c069620 msgr2=0x7f442c069a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:26.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.099+0000 7f4432400640 1 --2- 172.21.15.53:0/1269826391 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f442c069620 0x7f442c069a80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4414009900 tx=0x7f441402a3f0 comp rx=0 tx=0).stop 2024-04-25T14:51:26.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.099+0000 7f4432400640 1 -- 172.21.15.53:0/1269826391 shutdown_connections 2024-04-25T14:51:26.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.099+0000 7f4432400640 1 --2- 172.21.15.53:0/1269826391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f442c10e5b0 0x7f442c1109c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:26.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.099+0000 7f4432400640 1 --2- 172.21.15.53:0/1269826391 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f442c069620 0x7f442c069a80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:26.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.099+0000 7f4432400640 1 --2- 172.21.15.53:0/1269826391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f442c102c50 0x7f442c0690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:26.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.099+0000 7f4432400640 1 -- 172.21.15.53:0/1269826391 >> 172.21.15.53:0/1269826391 conn(0x7f442c076790 msgr2=0x7f442c076ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:26.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.099+0000 7f4432400640 1 -- 172.21.15.53:0/1269826391 shutdown_connections 2024-04-25T14:51:26.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.099+0000 7f4432400640 1 -- 172.21.15.53:0/1269826391 wait complete. 2024-04-25T14:51:26.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.100+0000 7f4432400640 1 Processor -- start 2024-04-25T14:51:26.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.100+0000 7f4432400640 1 -- start start 2024-04-25T14:51:26.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.100+0000 7f4432400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f442c069620 0x7f442c1a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:26.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.100+0000 7f4432400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f442c102c50 0x7f442c1a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:26.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.100+0000 7f4432400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f442c10e5b0 0x7f442c19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:26.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.100+0000 7f4432400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f442c1141b0 con 0x7f442c102c50 2024-04-25T14:51:26.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.100+0000 7f4432400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f442c114030 con 0x7f442c069620 2024-04-25T14:51:26.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.100+0000 7f4432400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f442c114330 con 0x7f442c10e5b0 2024-04-25T14:51:26.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.101+0000 7f442be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f442c102c50 0x7f442c1a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:26.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.101+0000 7f4431000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f442c069620 0x7f442c1a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:26.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.101+0000 7f442be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f442c102c50 0x7f442c1a0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57264/0 (socket says 172.21.15.53:57264) 2024-04-25T14:51:26.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.101+0000 7f4431a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f442c10e5b0 0x7f442c19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:26.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.101+0000 7f442be00640 1 -- 172.21.15.53:0/1556575031 learned_addr learned my addr 172.21.15.53:0/1556575031 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:51:26.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.101+0000 7f442be00640 1 -- 172.21.15.53:0/1556575031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f442c10e5b0 msgr2=0x7f442c19a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:26.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.101+0000 7f442be00640 1 --2- 172.21.15.53:0/1556575031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f442c10e5b0 0x7f442c19a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:26.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.101+0000 7f442be00640 1 -- 172.21.15.53:0/1556575031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f442c069620 msgr2=0x7f442c1a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:26.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.101+0000 7f442be00640 1 --2- 172.21.15.53:0/1556575031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f442c069620 0x7f442c1a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:26.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.101+0000 7f442be00640 1 -- 172.21.15.53:0/1556575031 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f442c19ad30 con 0x7f442c102c50 2024-04-25T14:51:26.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.101+0000 7f442be00640 1 --2- 172.21.15.53:0/1556575031 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f442c102c50 0x7f442c1a0940 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4414005940 tx=0x7f44140040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:26.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.102+0000 7f4429600640 1 -- 172.21.15.53:0/1556575031 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441403f030 con 0x7f442c102c50 2024-04-25T14:51:26.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.102+0000 7f4429600640 1 -- 172.21.15.53:0/1556575031 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44140044c0 con 0x7f442c102c50 2024-04-25T14:51:26.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.102+0000 7f4432400640 1 -- 172.21.15.53:0/1556575031 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f442c19afc0 con 0x7f442c102c50 2024-04-25T14:51:26.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.102+0000 7f4429600640 1 -- 172.21.15.53:0/1556575031 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4414035070 con 0x7f442c102c50 2024-04-25T14:51:26.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.102+0000 7f4432400640 1 -- 172.21.15.53:0/1556575031 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f442c1a72c0 con 0x7f442c102c50 2024-04-25T14:51:26.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.103+0000 7f4432400640 1 -- 172.21.15.53:0/1556575031 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f442c068c90 con 0x7f442c102c50 2024-04-25T14:51:26.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.105+0000 7f4429600640 1 -- 172.21.15.53:0/1556575031 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f4414042030 con 0x7f442c102c50 2024-04-25T14:51:26.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.107+0000 7f4429600640 1 --2- 172.21.15.53:0/1556575031 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f44100a3d20 0x7f44100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:26.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.107+0000 7f4429600640 1 -- 172.21.15.53:0/1556575031 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8587+0+0 (secure 0 0 0) 0x7f44140f0bf0 con 0x7f442c102c50 2024-04-25T14:51:26.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.107+0000 7f4431000640 1 --2- 172.21.15.53:0/1556575031 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f44100a3d20 0x7f44100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:26.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.108+0000 7f4431000640 1 --2- 172.21.15.53:0/1556575031 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f44100a3d20 0x7f44100a61e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f441c009bd0 tx=0x7f441c005f90 comp rx=0 tx=0).ready entity=mgr.15093 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:26.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.112+0000 7f4429600640 1 -- 172.21.15.53:0/1556575031 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f44140bcba0 con 0x7f442c102c50 2024-04-25T14:51:26.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.321+0000 7f4432400640 1 -- 172.21.15.53:0/1556575031 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f442c1a7490 con 0x7f442c102c50 2024-04-25T14:51:26.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.324+0000 7f4429600640 1 -- 172.21.15.53:0/1556575031 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+23201 (secure 0 0 0) 0x7f44140c0ea0 con 0x7f442c102c50 2024-04-25T14:51:26.326 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:51:26.326 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":103,"fsid":"fd30e6ce-030f-11ef-bc93-c7b262605968","created":"2024-04-25T14:28:54.330326+0000","modified":"2024-04-25T14:51:20.935398+0000","last_up_change":"2024-04-25T14:37:10.921148+0000","last_in_change":"2024-04-25T14:36:49.566459+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:33:02.338115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:38:07.517974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:38:08.994122+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aecfbaf0-39da-4749-a49b-c7205dabd4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6803","nonce":464176015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6805","nonce":464176015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6809","nonce":464176015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6807","nonce":464176015}]},"public_addr":"172.21.15.53:6803/464176015","cluster_addr":"172.21.15.53:6805/464176015","heartbeat_back_addr":"172.21.15.53:6809/464176015","heartbeat_front_addr":"172.21.15.53:6807/464176015","state":["exists","up"]},{"osd":1,"uuid":"2c4a21ae-c43e-4839-8a56-c3fc63aeb7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6801","nonce":1304621982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6803","nonce":1304621982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6807","nonce":1304621982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6805","nonce":1304621982}]},"public_addr":"172.21.15.150:6801/1304621982","cluster_addr":"172.21.15.150:6803/1304621982","heartbeat_back_addr":"172.21.15.150:6807/1304621982","heartbeat_front_addr":"172.21.15.150:6805/1304621982","state":["exists","up"]},{"osd":2,"uuid":"e8effd43-adc0-472a-9eed-a9273b06dc0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6801","nonce":574509311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6803","nonce":574509311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6807","nonce":574509311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6805","nonce":574509311}]},"public_addr":"172.21.15.183:6801/574509311","cluster_addr":"172.21.15.183:6803/574509311","heartbeat_back_addr":"172.21.15.183:6807/574509311","heartbeat_front_addr":"172.21.15.183:6805/574509311","state":["exists","up"]},{"osd":3,"uuid":"e7cfced4-9b40-40de-a303-75ea8d5df4a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6811","nonce":570672219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6813","nonce":570672219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6817","nonce":570672219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6815","nonce":570672219}]},"public_addr":"172.21.15.53:6811/570672219","cluster_addr":"172.21.15.53:6813/570672219","heartbeat_back_addr":"172.21.15.53:6817/570672219","heartbeat_front_addr":"172.21.15.53:6815/570672219","state":["exists","up"]},{"osd":4,"uuid":"44a9e070-2282-43f1-9f05-b1f8ff81e540","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6809","nonce":1526850103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6811","nonce":1526850103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6815","nonce":1526850103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6813","nonce":1526850103}]},"public_addr":"172.21.15.150:6809/1526850103","cluster_addr":"172.21.15.150:6811/1526850103","heartbeat_back_addr":"172.21.15.150:6815/1526850103","heartbeat_front_addr":"172.21.15.150:6813/1526850103","state":["exists","up"]},{"osd":5,"uuid":"2c5619d2-c0f9-47d6-bf7e-2f4afd1b09f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6809","nonce":773726576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6811","nonce":773726576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6815","nonce":773726576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6813","nonce":773726576}]},"public_addr":"172.21.15.183:6809/773726576","cluster_addr":"172.21.15.183:6811/773726576","heartbeat_back_addr":"172.21.15.183:6815/773726576","heartbeat_front_addr":"172.21.15.183:6813/773726576","state":["exists","up"]},{"osd":6,"uuid":"1cf74d94-c118-4738-b6db-040af458a8eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6819","nonce":355884319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6821","nonce":355884319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6825","nonce":355884319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6823","nonce":355884319}]},"public_addr":"172.21.15.53:6819/355884319","cluster_addr":"172.21.15.53:6821/355884319","heartbeat_back_addr":"172.21.15.53:6825/355884319","heartbeat_front_addr":"172.21.15.53:6823/355884319","state":["exists","up"]},{"osd":7,"uuid":"0ce97c04-cd41-4080-bbbc-be9cf5fadc02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6817","nonce":229022562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6819","nonce":229022562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6823","nonce":229022562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6821","nonce":229022562}]},"public_addr":"172.21.15.150:6817/229022562","cluster_addr":"172.21.15.150:6819/229022562","heartbeat_back_addr":"172.21.15.150:6823/229022562","heartbeat_front_addr":"172.21.15.150:6821/229022562","state":["exists","up"]},{"osd":8,"uuid":"5d57787d-2a7c-4514-bd01-357ced413f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6817","nonce":301003844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6819","nonce":301003844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6823","nonce":301003844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6821","nonce":301003844}]},"public_addr":"172.21.15.183:6817/301003844","cluster_addr":"172.21.15.183:6819/301003844","heartbeat_back_addr":"172.21.15.183:6823/301003844","heartbeat_front_addr":"172.21.15.183:6821/301003844","state":["exists","up"]},{"osd":9,"uuid":"e11dd8eb-c55a-4d70-8c58-457540b7a50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6827","nonce":3449260978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6829","nonce":3449260978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6833","nonce":3449260978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6831","nonce":3449260978}]},"public_addr":"172.21.15.53:6827/3449260978","cluster_addr":"172.21.15.53:6829/3449260978","heartbeat_back_addr":"172.21.15.53:6833/3449260978","heartbeat_front_addr":"172.21.15.53:6831/3449260978","state":["exists","up"]},{"osd":10,"uuid":"5de4803d-ad37-4a90-add5-0048f44fc4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6825","nonce":3435992308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6827","nonce":3435992308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6831","nonce":3435992308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6829","nonce":3435992308}]},"public_addr":"172.21.15.150:6825/3435992308","cluster_addr":"172.21.15.150:6827/3435992308","heartbeat_back_addr":"172.21.15.150:6831/3435992308","heartbeat_front_addr":"172.21.15.150:6829/3435992308","state":["exists","up"]},{"osd":11,"uuid":"0e819a68-ed56-477e-b9c0-81e8cc0bce11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6825","nonce":1957842686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6827","nonce":1957842686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6831","nonce":1957842686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6829","nonce":1957842686}]},"public_addr":"172.21.15.183:6825/1957842686","cluster_addr":"172.21.15.183:6827/1957842686","heartbeat_back_addr":"172.21.15.183:6831/1957842686","heartbeat_front_addr":"172.21.15.183:6829/1957842686","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:09.840629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:34.769537+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:58.112969+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:33:29.656790+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:33:55.181287+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:34:19.772645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:34:53.833921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:35:19.076733+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:35:43.163390+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:36:16.568340+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:36:42.492768+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:37:07.944639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"3.38","mappings":[{"from":5,"to":7},{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6800/4154933108":"2024-04-26T14:40:40.168940+0000","172.21.15.53:0/3916258673":"2024-04-26T14:40:40.168940+0000","172.21.15.53:0/309091034":"2024-04-26T14:40:40.168940+0000","172.21.15.53:0/4267178590":"2024-04-26T14:40:40.168940+0000","172.21.15.53:0/920465993":"2024-04-26T14:40:40.168940+0000","172.21.15.53:6800/3887954889":"2024-04-26T14:30:14.162383+0000","172.21.15.53:6801/4154933108":"2024-04-26T14:40:40.168940+0000","172.21.15.53:0/1014501022":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/2576091441":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/1233527569":"2024-04-26T14:29:38.106242+0000","172.21.15.53:6801/3887954889":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/931889810":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/177196141":"2024-04-26T14:30:14.162383+0000","172.21.15.53:6800/1722497390":"2024-04-26T14:29:38.106242+0000","172.21.15.53:6801/1722497390":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/1927549198":"2024-04-26T14:30:14.162383+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:51:26.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.327+0000 7f4432400640 1 -- 172.21.15.53:0/1556575031 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f44100a3d20 msgr2=0x7f44100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:26.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.327+0000 7f4432400640 1 --2- 172.21.15.53:0/1556575031 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f44100a3d20 0x7f44100a61e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f441c009bd0 tx=0x7f441c005f90 comp rx=0 tx=0).stop 2024-04-25T14:51:26.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.327+0000 7f4432400640 1 -- 172.21.15.53:0/1556575031 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f442c102c50 msgr2=0x7f442c1a0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:26.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.327+0000 7f4432400640 1 --2- 172.21.15.53:0/1556575031 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f442c102c50 0x7f442c1a0940 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4414005940 tx=0x7f44140040c0 comp rx=0 tx=0).stop 2024-04-25T14:51:26.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.327+0000 7f4432400640 1 -- 172.21.15.53:0/1556575031 shutdown_connections 2024-04-25T14:51:26.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.327+0000 7f4432400640 1 --2- 172.21.15.53:0/1556575031 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f44100a3d20 0x7f44100a61e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:26.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.327+0000 7f4432400640 1 --2- 172.21.15.53:0/1556575031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f442c10e5b0 0x7f442c19a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:26.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.327+0000 7f4432400640 1 --2- 172.21.15.53:0/1556575031 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f442c102c50 0x7f442c1a0940 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:26.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.328+0000 7f4432400640 1 --2- 172.21.15.53:0/1556575031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f442c069620 0x7f442c1a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:26.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.328+0000 7f4432400640 1 -- 172.21.15.53:0/1556575031 >> 172.21.15.53:0/1556575031 conn(0x7f442c076790 msgr2=0x7f442c0fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:26.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.328+0000 7f4432400640 1 -- 172.21.15.53:0/1556575031 shutdown_connections 2024-04-25T14:51:26.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.328+0000 7f4432400640 1 -- 172.21.15.53:0/1556575031 wait complete. 2024-04-25T14:51:26.349 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:33:02.338115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:38:07.517974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:38:08.994122+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 102, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:51:26.350 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:51:26.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.569+0000 7f996fe00640 1 -- 172.21.15.53:0/649971867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f997010c3d0 msgr2=0x7f997010e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:26.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.569+0000 7f996fe00640 1 --2- 172.21.15.53:0/649971867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f997010c3d0 0x7f997010e870 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9968009900 tx=0x7f996802a490 comp rx=0 tx=0).stop 2024-04-25T14:51:26.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.569+0000 7f996fe00640 1 -- 172.21.15.53:0/649971867 shutdown_connections 2024-04-25T14:51:26.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.569+0000 7f996fe00640 1 --2- 172.21.15.53:0/649971867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f997010c3d0 0x7f997010e870 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:26.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.569+0000 7f996fe00640 1 --2- 172.21.15.53:0/649971867 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99701038d0 0x7f997010bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:26.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.569+0000 7f996fe00640 1 --2- 172.21.15.53:0/649971867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9970102fb0 0x7f9970103390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:26.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.569+0000 7f996fe00640 1 -- 172.21.15.53:0/649971867 >> 172.21.15.53:0/649971867 conn(0x7f99700fc7d0 msgr2=0x7f99700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:26.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.570+0000 7f996fe00640 1 -- 172.21.15.53:0/649971867 shutdown_connections 2024-04-25T14:51:26.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.570+0000 7f996fe00640 1 -- 172.21.15.53:0/649971867 wait complete. 2024-04-25T14:51:26.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.570+0000 7f996fe00640 1 Processor -- start 2024-04-25T14:51:26.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.571+0000 7f996fe00640 1 -- start start 2024-04-25T14:51:26.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.571+0000 7f996fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9970102fb0 0x7f9970100c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:26.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.571+0000 7f996fe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99701038d0 0x7f9970101140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:26.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.571+0000 7f996fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f997010c3d0 0x7f9970101680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:26.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.571+0000 7f996fe00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9970112090 con 0x7f9970102fb0 2024-04-25T14:51:26.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.571+0000 7f996fe00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9970111f10 con 0x7f99701038d0 2024-04-25T14:51:26.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.571+0000 7f996fe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9970112210 con 0x7f997010c3d0 2024-04-25T14:51:26.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.571+0000 7f996ea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9970102fb0 0x7f9970100c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:26.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.571+0000 7f996ea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9970102fb0 0x7f9970100c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57292/0 (socket says 172.21.15.53:57292) 2024-04-25T14:51:26.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.571+0000 7f996ea00640 1 -- 172.21.15.53:0/486699184 learned_addr learned my addr 172.21.15.53:0/486699184 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:51:26.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.571+0000 7f996e000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99701038d0 0x7f9970101140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:26.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.572+0000 7f996f400640 1 --2- 172.21.15.53:0/486699184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f997010c3d0 0x7f9970101680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:26.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.572+0000 7f996e000640 1 -- 172.21.15.53:0/486699184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f997010c3d0 msgr2=0x7f9970101680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:26.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.572+0000 7f996e000640 1 --2- 172.21.15.53:0/486699184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f997010c3d0 0x7f9970101680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:26.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.572+0000 7f996e000640 1 -- 172.21.15.53:0/486699184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9970102fb0 msgr2=0x7f9970100c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:26.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.572+0000 7f996e000640 1 --2- 172.21.15.53:0/486699184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9970102fb0 0x7f9970100c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:26.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.572+0000 7f996e000640 1 -- 172.21.15.53:0/486699184 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99701a6a30 con 0x7f99701038d0 2024-04-25T14:51:26.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.572+0000 7f996ea00640 1 --2- 172.21.15.53:0/486699184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9970102fb0 0x7f9970100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:51:26.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.572+0000 7f996e000640 1 --2- 172.21.15.53:0/486699184 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99701038d0 0x7f9970101140 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f995800ea50 tx=0x7f995800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:26.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.573+0000 7f9967400640 1 -- 172.21.15.53:0/486699184 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995800ce20 con 0x7f99701038d0 2024-04-25T14:51:26.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.573+0000 7f996fe00640 1 -- 172.21.15.53:0/486699184 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99701a6d20 con 0x7f99701038d0 2024-04-25T14:51:26.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.573+0000 7f9967400640 1 -- 172.21.15.53:0/486699184 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99580108e0 con 0x7f99701038d0 2024-04-25T14:51:26.577 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.573+0000 7f9967400640 1 -- 172.21.15.53:0/486699184 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9958017a00 con 0x7f99701038d0 2024-04-25T14:51:26.577 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.573+0000 7f996fe00640 1 -- 172.21.15.53:0/486699184 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99701a7210 con 0x7f99701038d0 2024-04-25T14:51:26.577 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.574+0000 7f996fe00640 1 -- 172.21.15.53:0/486699184 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9970068c90 con 0x7f99701038d0 2024-04-25T14:51:26.577 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.576+0000 7f9967400640 1 -- 172.21.15.53:0/486699184 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f9958017ba0 con 0x7f99701038d0 2024-04-25T14:51:26.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.578+0000 7f9967400640 1 --2- 172.21.15.53:0/486699184 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f99500a3d50 0x7f99500a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:26.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.578+0000 7f9967400640 1 -- 172.21.15.53:0/486699184 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8587+0+0 (secure 0 0 0) 0x7f99580d10f0 con 0x7f99701038d0 2024-04-25T14:51:26.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.578+0000 7f996ea00640 1 --2- 172.21.15.53:0/486699184 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f99500a3d50 0x7f99500a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:26.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.579+0000 7f996ea00640 1 --2- 172.21.15.53:0/486699184 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f99500a3d50 0x7f99500a6210 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9960011550 tx=0x7f9960005f90 comp rx=0 tx=0).ready entity=mgr.15093 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:26.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.580+0000 7f9967400640 1 -- 172.21.15.53:0/486699184 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f995809d6f0 con 0x7f99701038d0 2024-04-25T14:51:26.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.789+0000 7f996fe00640 1 -- 172.21.15.53:0/486699184 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f997002d5c0 con 0x7f99701038d0 2024-04-25T14:51:26.792 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.790+0000 7f9967400640 1 -- 172.21.15.53:0/486699184 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) ==== 93+0+10 (secure 0 0 0) 0x7f99580bd020 con 0x7f99701038d0 2024-04-25T14:51:26.792 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2024-04-25T14:51:26.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.793+0000 7f996fe00640 1 -- 172.21.15.53:0/486699184 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f99500a3d50 msgr2=0x7f99500a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:26.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.793+0000 7f996fe00640 1 --2- 172.21.15.53:0/486699184 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f99500a3d50 0x7f99500a6210 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9960011550 tx=0x7f9960005f90 comp rx=0 tx=0).stop 2024-04-25T14:51:26.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.793+0000 7f996fe00640 1 -- 172.21.15.53:0/486699184 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99701038d0 msgr2=0x7f9970101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:26.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.793+0000 7f996fe00640 1 --2- 172.21.15.53:0/486699184 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99701038d0 0x7f9970101140 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f995800ea50 tx=0x7f995800ef10 comp rx=0 tx=0).stop 2024-04-25T14:51:26.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.793+0000 7f996fe00640 1 -- 172.21.15.53:0/486699184 shutdown_connections 2024-04-25T14:51:26.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.793+0000 7f996fe00640 1 --2- 172.21.15.53:0/486699184 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f99500a3d50 0x7f99500a6210 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:26.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.793+0000 7f996fe00640 1 --2- 172.21.15.53:0/486699184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f997010c3d0 0x7f9970101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:26.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.793+0000 7f996fe00640 1 --2- 172.21.15.53:0/486699184 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99701038d0 0x7f9970101140 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:26.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.793+0000 7f996fe00640 1 --2- 172.21.15.53:0/486699184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9970102fb0 0x7f9970100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:26.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.793+0000 7f996fe00640 1 -- 172.21.15.53:0/486699184 >> 172.21.15.53:0/486699184 conn(0x7f99700fc7d0 msgr2=0x7f99700fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:26.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.794+0000 7f996fe00640 1 -- 172.21.15.53:0/486699184 shutdown_connections 2024-04-25T14:51:26.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:26.794+0000 7f996fe00640 1 -- 172.21.15.53:0/486699184 wait complete. 2024-04-25T14:51:26.814 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:51:26.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:26 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/1556575031' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:51:26.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:26 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/1556575031' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:51:27.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:26 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1556575031' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:51:27.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.035+0000 7fe99a200640 1 -- 172.21.15.53:0/2149134604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe994104c20 msgr2=0x7fe994108c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:27.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.035+0000 7fe99a200640 1 --2- 172.21.15.53:0/2149134604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe994104c20 0x7fe994108c60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe980009900 tx=0x7fe98002a3f0 comp rx=0 tx=0).stop 2024-04-25T14:51:27.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.035+0000 7fe99a200640 1 -- 172.21.15.53:0/2149134604 shutdown_connections 2024-04-25T14:51:27.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.035+0000 7fe99a200640 1 --2- 172.21.15.53:0/2149134604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe994109340 0x7fe994110f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:27.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.035+0000 7fe99a200640 1 --2- 172.21.15.53:0/2149134604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe994104c20 0x7fe994108c60 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:27.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.035+0000 7fe99a200640 1 --2- 172.21.15.53:0/2149134604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe994104270 0x7fe994104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:27.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.035+0000 7fe99a200640 1 -- 172.21.15.53:0/2149134604 >> 172.21.15.53:0/2149134604 conn(0x7fe9940ffa90 msgr2=0x7fe994101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:27.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.035+0000 7fe99a200640 1 -- 172.21.15.53:0/2149134604 shutdown_connections 2024-04-25T14:51:27.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.035+0000 7fe99a200640 1 -- 172.21.15.53:0/2149134604 wait complete. 2024-04-25T14:51:27.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.036+0000 7fe99a200640 1 Processor -- start 2024-04-25T14:51:27.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.036+0000 7fe99a200640 1 -- start start 2024-04-25T14:51:27.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.036+0000 7fe99a200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe994104270 0x7fe9941a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:27.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.036+0000 7fe99a200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe994109340 0x7fe9941a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:27.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.036+0000 7fe99a200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe99419a630 0x7fe99419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:27.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.036+0000 7fe99a200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe9941142d0 con 0x7fe994109340 2024-04-25T14:51:27.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.036+0000 7fe99a200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe994114150 con 0x7fe99419a630 2024-04-25T14:51:27.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.037+0000 7fe99a200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe994114450 con 0x7fe994104270 2024-04-25T14:51:27.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.037+0000 7fe993e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe994109340 0x7fe9941a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:27.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.037+0000 7fe993e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe994109340 0x7fe9941a0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57318/0 (socket says 172.21.15.53:57318) 2024-04-25T14:51:27.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.037+0000 7fe993e00640 1 -- 172.21.15.53:0/777997281 learned_addr learned my addr 172.21.15.53:0/777997281 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:51:27.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.037+0000 7fe998e00640 1 --2- 172.21.15.53:0/777997281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe994104270 0x7fe9941a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:27.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.037+0000 7fe999800640 1 --2- 172.21.15.53:0/777997281 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe99419a630 0x7fe99419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:27.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.037+0000 7fe993e00640 1 -- 172.21.15.53:0/777997281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe994104270 msgr2=0x7fe9941a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:27.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.037+0000 7fe993e00640 1 --2- 172.21.15.53:0/777997281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe994104270 0x7fe9941a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:27.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.037+0000 7fe993e00640 1 -- 172.21.15.53:0/777997281 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe99419a630 msgr2=0x7fe99419aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:27.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.037+0000 7fe993e00640 1 --2- 172.21.15.53:0/777997281 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe99419a630 0x7fe99419aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:27.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.037+0000 7fe993e00640 1 -- 172.21.15.53:0/777997281 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe99419b230 con 0x7fe994109340 2024-04-25T14:51:27.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.037+0000 7fe999800640 1 --2- 172.21.15.53:0/777997281 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe99419a630 0x7fe99419aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:51:27.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.038+0000 7fe993e00640 1 --2- 172.21.15.53:0/777997281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe994109340 0x7fe9941a0ab0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe980005ab0 tx=0x7fe980003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:27.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.038+0000 7fe991600640 1 -- 172.21.15.53:0/777997281 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98003f070 con 0x7fe994109340 2024-04-25T14:51:27.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.038+0000 7fe991600640 1 -- 172.21.15.53:0/777997281 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe980031b50 con 0x7fe994109340 2024-04-25T14:51:27.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.038+0000 7fe99a200640 1 -- 172.21.15.53:0/777997281 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe99419b4c0 con 0x7fe994109340 2024-04-25T14:51:27.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.038+0000 7fe991600640 1 -- 172.21.15.53:0/777997281 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe980035030 con 0x7fe994109340 2024-04-25T14:51:27.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.038+0000 7fe99a200640 1 -- 172.21.15.53:0/777997281 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9941a7650 con 0x7fe994109340 2024-04-25T14:51:27.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.039+0000 7fe99a200640 1 -- 172.21.15.53:0/777997281 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe994105740 con 0x7fe994109340 2024-04-25T14:51:27.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.042+0000 7fe991600640 1 -- 172.21.15.53:0/777997281 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fe980042070 con 0x7fe994109340 2024-04-25T14:51:27.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.043+0000 7fe991600640 1 --2- 172.21.15.53:0/777997281 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fe97c0a3da0 0x7fe97c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:27.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.043+0000 7fe991600640 1 -- 172.21.15.53:0/777997281 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8587+0+0 (secure 0 0 0) 0x7fe9800f11b0 con 0x7fe994109340 2024-04-25T14:51:27.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.043+0000 7fe998e00640 1 --2- 172.21.15.53:0/777997281 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fe97c0a3da0 0x7fe97c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:27.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.044+0000 7fe998e00640 1 --2- 172.21.15.53:0/777997281 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fe97c0a3da0 0x7fe97c0a6260 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe988009bd0 tx=0x7fe988005f90 comp rx=0 tx=0).ready entity=mgr.15093 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:27.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.045+0000 7fe991600640 1 -- 172.21.15.53:0/777997281 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe9800bd160 con 0x7fe994109340 2024-04-25T14:51:27.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.251+0000 7fe99a200640 1 -- 172.21.15.53:0/777997281 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe99410c570 con 0x7fe994109340 2024-04-25T14:51:27.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.252+0000 7fe991600640 1 -- 172.21.15.53:0/777997281 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v103) ==== 104+0+11 (secure 0 0 0) 0x7fe9800c1460 con 0x7fe994109340 2024-04-25T14:51:27.254 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2024-04-25T14:51:27.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.255+0000 7fe99a200640 1 -- 172.21.15.53:0/777997281 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fe97c0a3da0 msgr2=0x7fe97c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:27.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.255+0000 7fe99a200640 1 --2- 172.21.15.53:0/777997281 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fe97c0a3da0 0x7fe97c0a6260 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe988009bd0 tx=0x7fe988005f90 comp rx=0 tx=0).stop 2024-04-25T14:51:27.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.255+0000 7fe99a200640 1 -- 172.21.15.53:0/777997281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe994109340 msgr2=0x7fe9941a0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:27.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.255+0000 7fe99a200640 1 --2- 172.21.15.53:0/777997281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe994109340 0x7fe9941a0ab0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe980005ab0 tx=0x7fe980003b90 comp rx=0 tx=0).stop 2024-04-25T14:51:27.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.255+0000 7fe99a200640 1 -- 172.21.15.53:0/777997281 shutdown_connections 2024-04-25T14:51:27.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.255+0000 7fe99a200640 1 --2- 172.21.15.53:0/777997281 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fe97c0a3da0 0x7fe97c0a6260 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:27.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.255+0000 7fe99a200640 1 --2- 172.21.15.53:0/777997281 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe99419a630 0x7fe99419aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:27.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.255+0000 7fe99a200640 1 --2- 172.21.15.53:0/777997281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe994109340 0x7fe9941a0ab0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:27.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.255+0000 7fe99a200640 1 --2- 172.21.15.53:0/777997281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe994104270 0x7fe9941a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:27.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.255+0000 7fe99a200640 1 -- 172.21.15.53:0/777997281 >> 172.21.15.53:0/777997281 conn(0x7fe9940ffa90 msgr2=0x7fe994100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:27.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.255+0000 7fe99a200640 1 -- 172.21.15.53:0/777997281 shutdown_connections 2024-04-25T14:51:27.258 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.255+0000 7fe99a200640 1 -- 172.21.15.53:0/777997281 wait complete. 2024-04-25T14:51:27.276 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:51:27.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.492+0000 7fbf97000640 1 -- 172.21.15.53:0/1107546974 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf90102fb0 msgr2=0x7fbf90103390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:27.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.492+0000 7fbf97000640 1 --2- 172.21.15.53:0/1107546974 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf90102fb0 0x7fbf90103390 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbf84009ca0 tx=0x7fbf8402a950 comp rx=0 tx=0).stop 2024-04-25T14:51:27.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.493+0000 7fbf97000640 1 -- 172.21.15.53:0/1107546974 shutdown_connections 2024-04-25T14:51:27.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.493+0000 7fbf97000640 1 --2- 172.21.15.53:0/1107546974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf9010c3d0 0x7fbf9010e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:27.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.493+0000 7fbf97000640 1 --2- 172.21.15.53:0/1107546974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf901038d0 0x7fbf9010bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:27.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.493+0000 7fbf97000640 1 --2- 172.21.15.53:0/1107546974 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf90102fb0 0x7fbf90103390 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:27.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.493+0000 7fbf97000640 1 -- 172.21.15.53:0/1107546974 >> 172.21.15.53:0/1107546974 conn(0x7fbf900fc7d0 msgr2=0x7fbf900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:27.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.493+0000 7fbf97000640 1 -- 172.21.15.53:0/1107546974 shutdown_connections 2024-04-25T14:51:27.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.493+0000 7fbf97000640 1 -- 172.21.15.53:0/1107546974 wait complete. 2024-04-25T14:51:27.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.494+0000 7fbf97000640 1 Processor -- start 2024-04-25T14:51:27.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.495+0000 7fbf97000640 1 -- start start 2024-04-25T14:51:27.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.495+0000 7fbf97000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf90102fb0 0x7fbf901a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:27.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.495+0000 7fbf97000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf901038d0 0x7fbf901a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:27.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.495+0000 7fbf97000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf9010c3d0 0x7fbf9019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:27.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.495+0000 7fbf97000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbf901120b0 con 0x7fbf9010c3d0 2024-04-25T14:51:27.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.495+0000 7fbf97000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbf90111f30 con 0x7fbf90102fb0 2024-04-25T14:51:27.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.495+0000 7fbf97000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbf90112230 con 0x7fbf901038d0 2024-04-25T14:51:27.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.495+0000 7fbf96600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf9010c3d0 0x7fbf9019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:27.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.495+0000 7fbf95c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf90102fb0 0x7fbf901a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:27.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.496+0000 7fbf95c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf90102fb0 0x7fbf901a0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.53:60594/0 (socket says 172.21.15.53:60594) 2024-04-25T14:51:27.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.496+0000 7fbf96600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf9010c3d0 0x7fbf9019a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57338/0 (socket says 172.21.15.53:57338) 2024-04-25T14:51:27.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.496+0000 7fbf95c00640 1 -- 172.21.15.53:0/3487928868 learned_addr learned my addr 172.21.15.53:0/3487928868 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:51:27.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.496+0000 7fbf95200640 1 --2- 172.21.15.53:0/3487928868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf901038d0 0x7fbf901a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:27.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.496+0000 7fbf95c00640 1 -- 172.21.15.53:0/3487928868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf901038d0 msgr2=0x7fbf901a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:27.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.496+0000 7fbf95c00640 1 --2- 172.21.15.53:0/3487928868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf901038d0 0x7fbf901a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:27.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.496+0000 7fbf95c00640 1 -- 172.21.15.53:0/3487928868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf9010c3d0 msgr2=0x7fbf9019a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:27.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.496+0000 7fbf95c00640 1 --2- 172.21.15.53:0/3487928868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf9010c3d0 0x7fbf9019a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:27.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.496+0000 7fbf95c00640 1 -- 172.21.15.53:0/3487928868 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf9019ad90 con 0x7fbf90102fb0 2024-04-25T14:51:27.500 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.496+0000 7fbf95c00640 1 --2- 172.21.15.53:0/3487928868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf90102fb0 0x7fbf901a0460 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbf84005e30 tx=0x7fbf84003d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:27.500 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.497+0000 7fbf8e000640 1 -- 172.21.15.53:0/3487928868 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf84034040 con 0x7fbf90102fb0 2024-04-25T14:51:27.500 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.497+0000 7fbf8e000640 1 -- 172.21.15.53:0/3487928868 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf84030850 con 0x7fbf90102fb0 2024-04-25T14:51:27.500 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.497+0000 7fbf97000640 1 -- 172.21.15.53:0/3487928868 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf9019b020 con 0x7fbf90102fb0 2024-04-25T14:51:27.500 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.497+0000 7fbf8e000640 1 -- 172.21.15.53:0/3487928868 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf840046e0 con 0x7fbf90102fb0 2024-04-25T14:51:27.500 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.497+0000 7fbf97000640 1 -- 172.21.15.53:0/3487928868 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf901a7320 con 0x7fbf90102fb0 2024-04-25T14:51:27.501 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.498+0000 7fbf97000640 1 -- 172.21.15.53:0/3487928868 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf90068c90 con 0x7fbf90102fb0 2024-04-25T14:51:27.501 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.500+0000 7fbf8e000640 1 -- 172.21.15.53:0/3487928868 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fbf84037020 con 0x7fbf90102fb0 2024-04-25T14:51:27.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.501+0000 7fbf8e000640 1 --2- 172.21.15.53:0/3487928868 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fbf780a3da0 0x7fbf780a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:27.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.502+0000 7fbf8e000640 1 -- 172.21.15.53:0/3487928868 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8587+0+0 (secure 0 0 0) 0x7fbf840f14b0 con 0x7fbf90102fb0 2024-04-25T14:51:27.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.502+0000 7fbf95200640 1 --2- 172.21.15.53:0/3487928868 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fbf780a3da0 0x7fbf780a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:27.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.503+0000 7fbf95200640 1 --2- 172.21.15.53:0/3487928868 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fbf780a3da0 0x7fbf780a6260 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c0095f0 tx=0x7fbf7c005f90 comp rx=0 tx=0).ready entity=mgr.15093 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:27.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.504+0000 7fbf8e000640 1 -- 172.21.15.53:0/3487928868 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbf840bdc50 con 0x7fbf90102fb0 2024-04-25T14:51:27.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.711+0000 7fbf97000640 1 -- 172.21.15.53:0/3487928868 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbf901a74f0 con 0x7fbf90102fb0 2024-04-25T14:51:27.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.712+0000 7fbf8e000640 1 -- 172.21.15.53:0/3487928868 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v103) ==== 100+0+11 (secure 0 0 0) 0x7fbf840098b0 con 0x7fbf90102fb0 2024-04-25T14:51:27.714 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2024-04-25T14:51:27.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.715+0000 7fbf97000640 1 -- 172.21.15.53:0/3487928868 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fbf780a3da0 msgr2=0x7fbf780a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:27.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.715+0000 7fbf97000640 1 --2- 172.21.15.53:0/3487928868 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fbf780a3da0 0x7fbf780a6260 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c0095f0 tx=0x7fbf7c005f90 comp rx=0 tx=0).stop 2024-04-25T14:51:27.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.715+0000 7fbf97000640 1 -- 172.21.15.53:0/3487928868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf90102fb0 msgr2=0x7fbf901a0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:27.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.715+0000 7fbf97000640 1 --2- 172.21.15.53:0/3487928868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf90102fb0 0x7fbf901a0460 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbf84005e30 tx=0x7fbf84003d60 comp rx=0 tx=0).stop 2024-04-25T14:51:27.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.715+0000 7fbf97000640 1 -- 172.21.15.53:0/3487928868 shutdown_connections 2024-04-25T14:51:27.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.715+0000 7fbf97000640 1 --2- 172.21.15.53:0/3487928868 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fbf780a3da0 0x7fbf780a6260 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:27.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.715+0000 7fbf97000640 1 --2- 172.21.15.53:0/3487928868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf9010c3d0 0x7fbf9019a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:27.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.715+0000 7fbf97000640 1 --2- 172.21.15.53:0/3487928868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf901038d0 0x7fbf901a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:27.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.715+0000 7fbf97000640 1 --2- 172.21.15.53:0/3487928868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf90102fb0 0x7fbf901a0460 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:27.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.715+0000 7fbf97000640 1 -- 172.21.15.53:0/3487928868 >> 172.21.15.53:0/3487928868 conn(0x7fbf900fc7d0 msgr2=0x7fbf90101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:27.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.715+0000 7fbf97000640 1 -- 172.21.15.53:0/3487928868 shutdown_connections 2024-04-25T14:51:27.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.715+0000 7fbf97000640 1 -- 172.21.15.53:0/3487928868 wait complete. 2024-04-25T14:51:27.737 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T14:51:27.948 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.946+0000 7f01b1400640 1 -- 172.21.15.53:0/4064278564 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01ac10d3c0 msgr2=0x7f01ac10d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:27.948 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.946+0000 7f01b1400640 1 --2- 172.21.15.53:0/4064278564 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01ac10d3c0 0x7f01ac10d7a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f019c009860 tx=0x7f019c02a3d0 comp rx=0 tx=0).stop 2024-04-25T14:51:27.948 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.946+0000 7f01b1400640 1 -- 172.21.15.53:0/4064278564 shutdown_connections 2024-04-25T14:51:27.948 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.946+0000 7f01b1400640 1 --2- 172.21.15.53:0/4064278564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01ac108e90 0x7f01ac1152b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:27.948 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.946+0000 7f01b1400640 1 --2- 172.21.15.53:0/4064278564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01ac1084f0 0x7f01ac108950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:27.949 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.946+0000 7f01b1400640 1 --2- 172.21.15.53:0/4064278564 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01ac10d3c0 0x7f01ac10d7a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:27.949 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.946+0000 7f01b1400640 1 -- 172.21.15.53:0/4064278564 >> 172.21.15.53:0/4064278564 conn(0x7f01ac077d70 msgr2=0x7f01ac106d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:27.949 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.947+0000 7f01b1400640 1 -- 172.21.15.53:0/4064278564 shutdown_connections 2024-04-25T14:51:27.949 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.947+0000 7f01b1400640 1 -- 172.21.15.53:0/4064278564 wait complete. 2024-04-25T14:51:27.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.947+0000 7f01b1400640 1 Processor -- start 2024-04-25T14:51:27.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.948+0000 7f01b1400640 1 -- start start 2024-04-25T14:51:27.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.948+0000 7f01b1400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01ac1084f0 0x7f01ac1a4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:27.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.948+0000 7f01b1400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01ac108e90 0x7f01ac1a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:27.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.948+0000 7f01b1400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01ac10d3c0 0x7f01ac19eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:27.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.948+0000 7f01b1400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac1186e0 con 0x7f01ac1084f0 2024-04-25T14:51:27.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.948+0000 7f01b1400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac118560 con 0x7f01ac10d3c0 2024-04-25T14:51:27.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.948+0000 7f01b1400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac118860 con 0x7f01ac108e90 2024-04-25T14:51:27.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.948+0000 7f01abe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01ac1084f0 0x7f01ac1a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:27.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.948+0000 7f01abe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01ac1084f0 0x7f01ac1a4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57354/0 (socket says 172.21.15.53:57354) 2024-04-25T14:51:27.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.948+0000 7f01abe00640 1 -- 172.21.15.53:0/1622077263 learned_addr learned my addr 172.21.15.53:0/1622077263 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:51:27.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.949+0000 7f01a3e00640 1 --2- 172.21.15.53:0/1622077263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01ac108e90 0x7f01ac1a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:27.953 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.949+0000 7f01b0a00640 1 --2- 172.21.15.53:0/1622077263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01ac10d3c0 0x7f01ac19eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:27.953 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.949+0000 7f01abe00640 1 -- 172.21.15.53:0/1622077263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01ac108e90 msgr2=0x7f01ac1a4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:27.953 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.949+0000 7f01abe00640 1 --2- 172.21.15.53:0/1622077263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01ac108e90 0x7f01ac1a4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:27.953 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.949+0000 7f01abe00640 1 -- 172.21.15.53:0/1622077263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01ac10d3c0 msgr2=0x7f01ac19eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:27.953 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.949+0000 7f01abe00640 1 --2- 172.21.15.53:0/1622077263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01ac10d3c0 0x7f01ac19eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:27.953 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.949+0000 7f01abe00640 1 -- 172.21.15.53:0/1622077263 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ac19f240 con 0x7f01ac1084f0 2024-04-25T14:51:27.953 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.949+0000 7f01abe00640 1 --2- 172.21.15.53:0/1622077263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01ac1084f0 0x7f01ac1a4910 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f019c009980 tx=0x7f019c030fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:27.953 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.950+0000 7f01a9600640 1 -- 172.21.15.53:0/1622077263 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019c040070 con 0x7f01ac1084f0 2024-04-25T14:51:27.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.950+0000 7f01b1400640 1 -- 172.21.15.53:0/1622077263 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01ac19f4d0 con 0x7f01ac1084f0 2024-04-25T14:51:27.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.950+0000 7f01a9600640 1 -- 172.21.15.53:0/1622077263 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f019c005570 con 0x7f01ac1084f0 2024-04-25T14:51:27.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.950+0000 7f01a9600640 1 -- 172.21.15.53:0/1622077263 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019c0046c0 con 0x7f01ac1084f0 2024-04-25T14:51:27.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.950+0000 7f01b1400640 1 -- 172.21.15.53:0/1622077263 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01ac1ab7d0 con 0x7f01ac1084f0 2024-04-25T14:51:27.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:27 smithi183 ceph-mon[32584]: pgmap v336: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T14:51:27.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:27 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/486699184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:51:27.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:27 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/777997281' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:51:27.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.951+0000 7f01b1400640 1 -- 172.21.15.53:0/1622077263 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01ac0690e0 con 0x7f01ac1084f0 2024-04-25T14:51:27.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.953+0000 7f01a9600640 1 -- 172.21.15.53:0/1622077263 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f019c043070 con 0x7f01ac1084f0 2024-04-25T14:51:27.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.954+0000 7f01a9600640 1 --2- 172.21.15.53:0/1622077263 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f01900a3d20 0x7f01900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:27.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.955+0000 7f01a9600640 1 -- 172.21.15.53:0/1622077263 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8587+0+0 (secure 0 0 0) 0x7f019c0f1330 con 0x7f01ac1084f0 2024-04-25T14:51:27.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.955+0000 7f01a3e00640 1 --2- 172.21.15.53:0/1622077263 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f01900a3d20 0x7f01900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:27.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.955+0000 7f01a3e00640 1 --2- 172.21.15.53:0/1622077263 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f01900a3d20 0x7f01900a61e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f01ac1080b0 tx=0x7f0194009260 comp rx=0 tx=0).ready entity=mgr.15093 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:27.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:27.959+0000 7f01a9600640 1 -- 172.21.15.53:0/1622077263 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f019c0bd9c0 con 0x7f01ac1084f0 2024-04-25T14:51:27.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:27 smithi150 ceph-mon[32504]: pgmap v336: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T14:51:27.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:27 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/486699184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:51:27.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:27 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/777997281' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:51:28.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:27 smithi053 ceph-mon[27773]: pgmap v336: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T14:51:28.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:27 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/486699184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:51:28.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:27 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/777997281' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:51:28.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:28.178+0000 7f01b1400640 1 -- 172.21.15.53:0/1622077263 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f01ac110900 con 0x7f01ac1084f0 2024-04-25T14:51:28.180 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:28.179+0000 7f01a9600640 1 -- 172.21.15.53:0/1622077263 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v103) ==== 81+0+942 (secure 0 0 0) 0x7f019c0bd2c0 con 0x7f01ac1084f0 2024-04-25T14:51:28.180 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6800/4154933108 2024-04-26T14:40:40.168940+0000 2024-04-25T14:51:28.181 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/3916258673 2024-04-26T14:40:40.168940+0000 2024-04-25T14:51:28.181 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/309091034 2024-04-26T14:40:40.168940+0000 2024-04-25T14:51:28.181 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/4267178590 2024-04-26T14:40:40.168940+0000 2024-04-25T14:51:28.181 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/920465993 2024-04-26T14:40:40.168940+0000 2024-04-25T14:51:28.181 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6800/3887954889 2024-04-26T14:30:14.162383+0000 2024-04-25T14:51:28.181 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6801/4154933108 2024-04-26T14:40:40.168940+0000 2024-04-25T14:51:28.181 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/1014501022 2024-04-26T14:30:14.162383+0000 2024-04-25T14:51:28.181 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/2576091441 2024-04-26T14:29:38.106242+0000 2024-04-25T14:51:28.182 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/1233527569 2024-04-26T14:29:38.106242+0000 2024-04-25T14:51:28.182 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6801/3887954889 2024-04-26T14:30:14.162383+0000 2024-04-25T14:51:28.182 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/931889810 2024-04-26T14:29:38.106242+0000 2024-04-25T14:51:28.182 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/177196141 2024-04-26T14:30:14.162383+0000 2024-04-25T14:51:28.182 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6800/1722497390 2024-04-26T14:29:38.106242+0000 2024-04-25T14:51:28.182 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6801/1722497390 2024-04-26T14:29:38.106242+0000 2024-04-25T14:51:28.182 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:0/1927549198 2024-04-26T14:30:14.162383+0000 2024-04-25T14:51:28.186 INFO:teuthology.orchestra.run.smithi053.stderr:listed 16 entries 2024-04-25T14:51:28.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:28.182+0000 7f01b1400640 1 -- 172.21.15.53:0/1622077263 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f01900a3d20 msgr2=0x7f01900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:28.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:28.182+0000 7f01b1400640 1 --2- 172.21.15.53:0/1622077263 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f01900a3d20 0x7f01900a61e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f01ac1080b0 tx=0x7f0194009260 comp rx=0 tx=0).stop 2024-04-25T14:51:28.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:28.182+0000 7f01b1400640 1 -- 172.21.15.53:0/1622077263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01ac1084f0 msgr2=0x7f01ac1a4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:28.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:28.182+0000 7f01b1400640 1 --2- 172.21.15.53:0/1622077263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01ac1084f0 0x7f01ac1a4910 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f019c009980 tx=0x7f019c030fa0 comp rx=0 tx=0).stop 2024-04-25T14:51:28.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:28.182+0000 7f01b1400640 1 -- 172.21.15.53:0/1622077263 shutdown_connections 2024-04-25T14:51:28.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:28.182+0000 7f01b1400640 1 --2- 172.21.15.53:0/1622077263 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f01900a3d20 0x7f01900a61e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:28.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:28.182+0000 7f01b1400640 1 --2- 172.21.15.53:0/1622077263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01ac10d3c0 0x7f01ac19eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:28.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:28.182+0000 7f01b1400640 1 --2- 172.21.15.53:0/1622077263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01ac108e90 0x7f01ac1a4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:28.188 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:28.182+0000 7f01b1400640 1 --2- 172.21.15.53:0/1622077263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01ac1084f0 0x7f01ac1a4910 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:28.188 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:28.182+0000 7f01b1400640 1 -- 172.21.15.53:0/1622077263 >> 172.21.15.53:0/1622077263 conn(0x7f01ac077d70 msgr2=0x7f01ac078490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:28.188 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:28.182+0000 7f01b1400640 1 -- 172.21.15.53:0/1622077263 shutdown_connections 2024-04-25T14:51:28.188 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:28.182+0000 7f01b1400640 1 -- 172.21.15.53:0/1622077263 wait complete. 2024-04-25T14:51:28.203 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T14:51:28.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:51:28.203 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T14:51:28.296 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2024-04-25T14:51:28.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:51:28.297 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T14:51:28.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:51:28.364 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2024-04-25T14:51:28.415 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T14:51:28.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:51:28.416 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T14:51:28.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:51:28.509 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link delete ceph-brx 2024-04-25T14:51:28.634 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T14:51:28.634 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-25T14:51:28.635 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:51:28.677 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/d77c3ececb404fc2c3b55b3f0353eee3f0994e76fe71c05416d30bf5487eb86c/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/ULFA4DGS47HDLEG6MDHKDVVBXL:/var/lib/containers/storage/overlay/l/ETTB45ZDEFWNCLNNHJQDUDFB66,upperdir=/var/lib/containers/storage/overlay/d77c3ececb404fc2c3b55b3f0353eee3f0994e76fe71c05416d30bf5487eb86c/diff,workdir=/var/lib/containers/storage/overlay/d77c3ececb404fc2c3b55b3f0353eee3f0994e76fe71c05416d30bf5487eb86c/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/2ad30826bd8dd99f1044287be5855c9c45c57a502f63dfec29f357272ad3167d/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/ULFA4DGS47HDLEG6MDHKDVVBXL:/var/lib/containers/storage/overlay/l/ETTB45ZDEFWNCLNNHJQDUDFB66,upperdir=/var/lib/containers/storage/overlay/2ad30826bd8dd99f1044287be5855c9c45c57a502f63dfec29f357272ad3167d/diff,workdir=/var/lib/containers/storage/overlay/2ad30826bd8dd99f1044287be5855c9c45c57a502f63dfec29f357272ad3167d/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/00f3bfcb689b3d12ef5c081c03227b2befbd20220ea35801c1a7745d2c7ec29f/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/ULFA4DGS47HDLEG6MDHKDVVBXL:/var/lib/containers/storage/overlay/l/ETTB45ZDEFWNCLNNHJQDUDFB66,upperdir=/var/lib/containers/storage/overlay/00f3bfcb689b3d12ef5c081c03227b2befbd20220ea35801c1a7745d2c7ec29f/diff,workdir=/var/lib/containers/storage/overlay/00f3bfcb689b3d12ef5c081c03227b2befbd20220ea35801c1a7745d2c7ec29f/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/aa778b5538414707828884e89bdb06322280f8ff1f2d74e19afd752aae9400b8/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/ULFA4DGS47HDLEG6MDHKDVVBXL:/var/lib/containers/storage/overlay/l/ETTB45ZDEFWNCLNNHJQDUDFB66,upperdir=/var/lib/containers/storage/overlay/aa778b5538414707828884e89bdb06322280f8ff1f2d74e19afd752aae9400b8/diff,workdir=/var/lib/containers/storage/overlay/aa778b5538414707828884e89bdb06322280f8ff1f2d74e19afd752aae9400b8/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/f6b778fab2a078fad058b7386cbabb635d5fa6f4f9429057764611cbc8638cac/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/ULFA4DGS47HDLEG6MDHKDVVBXL:/var/lib/containers/storage/overlay/l/ETTB45ZDEFWNCLNNHJQDUDFB66,upperdir=/var/lib/containers/storage/overlay/f6b778fab2a078fad058b7386cbabb635d5fa6f4f9429057764611cbc8638cac/diff,workdir=/var/lib/containers/storage/overlay/f6b778fab2a078fad058b7386cbabb635d5fa6f4f9429057764611cbc8638cac/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/17cba2866015cfba5624ae133cf153074774037c6c7b7354e3742b443a1d54e7/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/ULFA4DGS47HDLEG6MDHKDVVBXL:/var/lib/containers/storage/overlay/l/ETTB45ZDEFWNCLNNHJQDUDFB66,upperdir=/var/lib/containers/storage/overlay/17cba2866015cfba5624ae133cf153074774037c6c7b7354e3742b443a1d54e7/diff,workdir=/var/lib/containers/storage/overlay/17cba2866015cfba5624ae133cf153074774037c6c7b7354e3742b443a1d54e7/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/0b8ccd632f6079d1cb303bec02182a6c25699a8369b0d3eb8d4df94a0c02bb9e/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/ULFA4DGS47HDLEG6MDHKDVVBXL:/var/lib/containers/storage/overlay/l/ETTB45ZDEFWNCLNNHJQDUDFB66,upperdir=/var/lib/containers/storage/overlay/0b8ccd632f6079d1cb303bec02182a6c25699a8369b0d3eb8d4df94a0c02bb9e/diff,workdir=/var/lib/containers/storage/overlay/0b8ccd632f6079d1cb303bec02182a6c25699a8369b0d3eb8d4df94a0c02bb9e/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/9e839720dea39662e554a1c5fa76866a5c19762f63d03fdb2332e93267f31369/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/ULFA4DGS47HDLEG6MDHKDVVBXL:/var/lib/containers/storage/overlay/l/ETTB45ZDEFWNCLNNHJQDUDFB66,upperdir=/var/lib/containers/storage/overlay/9e839720dea39662e554a1c5fa76866a5c19762f63d03fdb2332e93267f31369/diff,workdir=/var/lib/containers/storage/overlay/9e839720dea39662e554a1c5fa76866a5c19762f63d03fdb2332e93267f31369/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/b470a3f905939c62a234a911bd6958af57b11e40f56941ccdb8d6caf83a033a2/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/ULFA4DGS47HDLEG6MDHKDVVBXL:/var/lib/containers/storage/overlay/l/ETTB45ZDEFWNCLNNHJQDUDFB66,upperdir=/var/lib/containers/storage/overlay/b470a3f905939c62a234a911bd6958af57b11e40f56941ccdb8d6caf83a033a2/diff,workdir=/var/lib/containers/storage/overlay/b470a3f905939c62a234a911bd6958af57b11e40f56941ccdb8d6caf83a033a2/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/86bab8f5ac0ea8496a6f338a55d129eff8b9566cf42ca0349dbe48b2ff50b129/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/ULFA4DGS47HDLEG6MDHKDVVBXL:/var/lib/containers/storage/overlay/l/ETTB45ZDEFWNCLNNHJQDUDFB66,upperdir=/var/lib/containers/storage/overlay/86bab8f5ac0ea8496a6f338a55d129eff8b9566cf42ca0349dbe48b2ff50b129/diff,workdir=/var/lib/containers/storage/overlay/86bab8f5ac0ea8496a6f338a55d129eff8b9566cf42ca0349dbe48b2ff50b129/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 2024-04-25T14:51:28.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:51:28.678 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2024-04-25T14:51:28.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:51:28.773 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link delete ceph-brx 2024-04-25T14:51:28.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:28 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/3487928868' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:51:28.782 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:28 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1622077263' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T14:51:28.863 INFO:teuthology.orchestra.run.smithi053.stderr:Cannot find device "ceph-brx" 2024-04-25T14:51:28.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:51:28.868 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-25T14:51:28.879 INFO:tasks.cephadm:Teardown begin 2024-04-25T14:51:28.880 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T14:51:28.880 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:51:28.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:28 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/3487928868' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:51:28.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:28 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/1622077263' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T14:51:28.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:28 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/3487928868' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:51:28.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:28 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/1622077263' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T14:51:29.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.132+0000 7fa410e00640 1 -- 172.21.15.53:0/517995209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa40c076480 msgr2=0x7fa40c076860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:29.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.132+0000 7fa410e00640 1 --2- 172.21.15.53:0/517995209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa40c076480 0x7fa40c076860 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc009c10 tx=0x7fa3fc02a6f0 comp rx=0 tx=0).stop 2024-04-25T14:51:29.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.132+0000 7fa410e00640 1 -- 172.21.15.53:0/517995209 shutdown_connections 2024-04-25T14:51:29.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.132+0000 7fa410e00640 1 --2- 172.21.15.53:0/517995209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa40c112bf0 0x7fa40c114fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:29.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.132+0000 7fa410e00640 1 --2- 172.21.15.53:0/517995209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa40c076da0 0x7fa40c1126b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:29.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.132+0000 7fa410e00640 1 --2- 172.21.15.53:0/517995209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa40c076480 0x7fa40c076860 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:29.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.132+0000 7fa410e00640 1 -- 172.21.15.53:0/517995209 >> 172.21.15.53:0/517995209 conn(0x7fa40c0ffa20 msgr2=0x7fa40c101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:29.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.132+0000 7fa410e00640 1 -- 172.21.15.53:0/517995209 shutdown_connections 2024-04-25T14:51:29.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.132+0000 7fa410e00640 1 -- 172.21.15.53:0/517995209 wait complete. 2024-04-25T14:51:29.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.133+0000 7fa410e00640 1 Processor -- start 2024-04-25T14:51:29.137 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.133+0000 7fa410e00640 1 -- start start 2024-04-25T14:51:29.137 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.133+0000 7fa410e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa40c076480 0x7fa40c19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:29.137 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.133+0000 7fa410e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa40c076da0 0x7fa40c19f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:29.137 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.133+0000 7fa410e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa40c112bf0 0x7fa40c1a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:29.137 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.133+0000 7fa410e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c118610 con 0x7fa40c076da0 2024-04-25T14:51:29.137 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.133+0000 7fa410e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c118490 con 0x7fa40c112bf0 2024-04-25T14:51:29.137 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.133+0000 7fa410e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c118790 con 0x7fa40c076480 2024-04-25T14:51:29.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.134+0000 7fa40aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa40c076da0 0x7fa40c19f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:29.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.134+0000 7fa40aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa40c076da0 0x7fa40c19f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57378/0 (socket says 172.21.15.53:57378) 2024-04-25T14:51:29.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.134+0000 7fa40aa00640 1 -- 172.21.15.53:0/602837171 learned_addr learned my addr 172.21.15.53:0/602837171 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:51:29.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.134+0000 7fa40b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa40c076480 0x7fa40c19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:29.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.134+0000 7fa40be00640 1 --2- 172.21.15.53:0/602837171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa40c112bf0 0x7fa40c1a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:29.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.134+0000 7fa40b400640 1 -- 172.21.15.53:0/602837171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa40c112bf0 msgr2=0x7fa40c1a35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:29.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.134+0000 7fa40b400640 1 --2- 172.21.15.53:0/602837171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa40c112bf0 0x7fa40c1a35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:29.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.134+0000 7fa40b400640 1 -- 172.21.15.53:0/602837171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa40c076da0 msgr2=0x7fa40c19f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:29.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.134+0000 7fa40b400640 1 --2- 172.21.15.53:0/602837171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa40c076da0 0x7fa40c19f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:29.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.134+0000 7fa40b400640 1 -- 172.21.15.53:0/602837171 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa40c1a3d10 con 0x7fa40c076480 2024-04-25T14:51:29.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.134+0000 7fa40aa00640 1 --2- 172.21.15.53:0/602837171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa40c076da0 0x7fa40c19f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:51:29.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.135+0000 7fa40b400640 1 --2- 172.21.15.53:0/602837171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa40c076480 0x7fa40c19ed20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc009b30 tx=0x7fa3fc003b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:29.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.135+0000 7fa403e00640 1 -- 172.21.15.53:0/602837171 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc041020 con 0x7fa40c076480 2024-04-25T14:51:29.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.135+0000 7fa410e00640 1 -- 172.21.15.53:0/602837171 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa40c1a3fa0 con 0x7fa40c076480 2024-04-25T14:51:29.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.135+0000 7fa403e00640 1 -- 172.21.15.53:0/602837171 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3fc0318f0 con 0x7fa40c076480 2024-04-25T14:51:29.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.135+0000 7fa403e00640 1 -- 172.21.15.53:0/602837171 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc036040 con 0x7fa40c076480 2024-04-25T14:51:29.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.135+0000 7fa410e00640 1 -- 172.21.15.53:0/602837171 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa40c1ab890 con 0x7fa40c076480 2024-04-25T14:51:29.140 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.136+0000 7fa410e00640 1 -- 172.21.15.53:0/602837171 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa40c0690e0 con 0x7fa40c076480 2024-04-25T14:51:29.140 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.138+0000 7fa403e00640 1 -- 172.21.15.53:0/602837171 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fa3fc044020 con 0x7fa40c076480 2024-04-25T14:51:29.141 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.140+0000 7fa403e00640 1 --2- 172.21.15.53:0/602837171 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fa3f00a3da0 0x7fa3f00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:29.141 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.140+0000 7fa403e00640 1 -- 172.21.15.53:0/602837171 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8587+0+0 (secure 0 0 0) 0x7fa3fc0f1810 con 0x7fa40c076480 2024-04-25T14:51:29.141 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.140+0000 7fa40aa00640 1 --2- 172.21.15.53:0/602837171 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fa3f00a3da0 0x7fa3f00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:29.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.141+0000 7fa40aa00640 1 --2- 172.21.15.53:0/602837171 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fa3f00a3da0 0x7fa3f00a6260 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa40c101f10 tx=0x7fa3f4005f90 comp rx=0 tx=0).ready entity=mgr.15093 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:29.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.143+0000 7fa403e00640 1 -- 172.21.15.53:0/602837171 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa3fc04b020 con 0x7fa40c076480 2024-04-25T14:51:29.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.395+0000 7fa410e00640 1 -- 172.21.15.53:0/602837171 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa40c10dd80 con 0x7fa40c076480 2024-04-25T14:51:29.398 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.397+0000 7fa403e00640 1 -- 172.21.15.53:0/602837171 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11862 (secure 0 0 0) 0x7fa3fc0d5e70 con 0x7fa40c076480 2024-04-25T14:51:29.398 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:51:29.399 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14460,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6841/41110727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6840","nonce":41110727},{"type":"v1","addr":"172.21.15.53:6841","nonce":41110727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2966992472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2966992472},{"type":"v1","addr":"172.21.15.150:6833","nonce":2966992472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/94143643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":94143643},{"type":"v1","addr":"172.21.15.183:6833","nonce":94143643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24470,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/506682943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":506682943},{"type":"v1","addr":"172.21.15.53:6835","nonce":506682943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1369488844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1369488844},{"type":"v1","addr":"172.21.15.150:6835","nonce":1369488844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3786232682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3786232682},{"type":"v1","addr":"172.21.15.53:6837","nonce":3786232682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":22,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T14:38:10.037451+0000","modified":"2024-04-25T14:40:40.222021+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24584,"mds_1":24565,"mds_2":24560},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24532":{"gid":24532,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1189328602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1189328602},{"type":"v1","addr":"172.21.15.183:6835","nonce":1189328602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/2344343627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2344343627},{"type":"v1","addr":"172.21.15.150:6837","nonce":2344343627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24544":{"gid":24544,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1163336384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1163336384},{"type":"v1","addr":"172.21.15.183:6837","nonce":1163336384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6839/570000540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":570000540},{"type":"v1","addr":"172.21.15.53:6839","nonce":570000540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.183:6839/1554258915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1554258915},{"type":"v1","addr":"172.21.15.183:6839","nonce":1554258915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/149573616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":149573616},{"type":"v1","addr":"172.21.15.150:6839","nonce":149573616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24560,24565,24584]},"id":1}]} 2024-04-25T14:51:29.399 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 22 2024-04-25T14:51:29.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.400+0000 7fa410e00640 1 -- 172.21.15.53:0/602837171 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fa3f00a3da0 msgr2=0x7fa3f00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:29.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.400+0000 7fa410e00640 1 --2- 172.21.15.53:0/602837171 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fa3f00a3da0 0x7fa3f00a6260 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa40c101f10 tx=0x7fa3f4005f90 comp rx=0 tx=0).stop 2024-04-25T14:51:29.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.400+0000 7fa410e00640 1 -- 172.21.15.53:0/602837171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa40c076480 msgr2=0x7fa40c19ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:29.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.400+0000 7fa410e00640 1 --2- 172.21.15.53:0/602837171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa40c076480 0x7fa40c19ed20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc009b30 tx=0x7fa3fc003b80 comp rx=0 tx=0).stop 2024-04-25T14:51:29.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.400+0000 7fa410e00640 1 -- 172.21.15.53:0/602837171 shutdown_connections 2024-04-25T14:51:29.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.400+0000 7fa410e00640 1 --2- 172.21.15.53:0/602837171 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fa3f00a3da0 0x7fa3f00a6260 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:29.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.400+0000 7fa410e00640 1 --2- 172.21.15.53:0/602837171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa40c112bf0 0x7fa40c1a35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:29.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.400+0000 7fa410e00640 1 --2- 172.21.15.53:0/602837171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa40c076da0 0x7fa40c19f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:29.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.400+0000 7fa410e00640 1 --2- 172.21.15.53:0/602837171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa40c076480 0x7fa40c19ed20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:29.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.400+0000 7fa410e00640 1 -- 172.21.15.53:0/602837171 >> 172.21.15.53:0/602837171 conn(0x7fa40c0ffa20 msgr2=0x7fa40c1143f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:29.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.400+0000 7fa410e00640 1 -- 172.21.15.53:0/602837171 shutdown_connections 2024-04-25T14:51:29.403 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.400+0000 7fa410e00640 1 -- 172.21.15.53:0/602837171 wait complete. 2024-04-25T14:51:29.422 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:51:29.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.635+0000 7f78bf200640 1 -- 172.21.15.53:0/2333896129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78c01041e0 msgr2=0x7f78c01045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:29.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.635+0000 7f78bf200640 1 --2- 172.21.15.53:0/2333896129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78c01041e0 0x7f78c01045c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f78b0009900 tx=0x7f78b002a3f0 comp rx=0 tx=0).stop 2024-04-25T14:51:29.638 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.636+0000 7f78bf200640 1 -- 172.21.15.53:0/2333896129 shutdown_connections 2024-04-25T14:51:29.638 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.636+0000 7f78bf200640 1 --2- 172.21.15.53:0/2333896129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78c01092b0 0x7f78c0110e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:29.638 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.636+0000 7f78bf200640 1 --2- 172.21.15.53:0/2333896129 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78c0104b90 0x7f78c0108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:29.638 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.636+0000 7f78bf200640 1 --2- 172.21.15.53:0/2333896129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78c01041e0 0x7f78c01045c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:29.638 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.636+0000 7f78bf200640 1 -- 172.21.15.53:0/2333896129 >> 172.21.15.53:0/2333896129 conn(0x7f78c00ffa20 msgr2=0x7f78c0101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:29.638 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.636+0000 7f78bf200640 1 -- 172.21.15.53:0/2333896129 shutdown_connections 2024-04-25T14:51:29.638 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.636+0000 7f78bf200640 1 -- 172.21.15.53:0/2333896129 wait complete. 2024-04-25T14:51:29.640 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.637+0000 7f78bf200640 1 Processor -- start 2024-04-25T14:51:29.640 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.637+0000 7f78bf200640 1 -- start start 2024-04-25T14:51:29.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.637+0000 7f78bf200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78c01041e0 0x7f78c01a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:29.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.637+0000 7f78bf200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78c01092b0 0x7f78c01a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:29.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.637+0000 7f78bf200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78c019a5d0 0x7f78c019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:29.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.637+0000 7f78bf200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f78c0114270 con 0x7f78c01041e0 2024-04-25T14:51:29.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.637+0000 7f78bf200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f78c01140f0 con 0x7f78c019a5d0 2024-04-25T14:51:29.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.637+0000 7f78bf200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f78c01143f0 con 0x7f78c01092b0 2024-04-25T14:51:29.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.637+0000 7f78bde00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78c01041e0 0x7f78c01a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:29.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.638+0000 7f78bde00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78c01041e0 0x7f78c01a0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57400/0 (socket says 172.21.15.53:57400) 2024-04-25T14:51:29.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.638+0000 7f78bde00640 1 -- 172.21.15.53:0/2697174762 learned_addr learned my addr 172.21.15.53:0/2697174762 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:51:29.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.638+0000 7f78be800640 1 --2- 172.21.15.53:0/2697174762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78c019a5d0 0x7f78c019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:29.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.638+0000 7f78bd400640 1 --2- 172.21.15.53:0/2697174762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78c01092b0 0x7f78c01a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:29.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.638+0000 7f78bde00640 1 -- 172.21.15.53:0/2697174762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78c01092b0 msgr2=0x7f78c01a0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:29.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.638+0000 7f78bde00640 1 --2- 172.21.15.53:0/2697174762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78c01092b0 0x7f78c01a0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:29.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.638+0000 7f78bde00640 1 -- 172.21.15.53:0/2697174762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78c019a5d0 msgr2=0x7f78c019aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:29.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.638+0000 7f78bde00640 1 --2- 172.21.15.53:0/2697174762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78c019a5d0 0x7f78c019aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:29.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.638+0000 7f78bde00640 1 -- 172.21.15.53:0/2697174762 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c019b0f0 con 0x7f78c01041e0 2024-04-25T14:51:29.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.638+0000 7f78be800640 1 --2- 172.21.15.53:0/2697174762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78c019a5d0 0x7f78c019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:51:29.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.638+0000 7f78bde00640 1 --2- 172.21.15.53:0/2697174762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78c01041e0 0x7f78c01a0420 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f78b0005ab0 tx=0x7f78b0003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:29.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.639+0000 7f78b6a00640 1 -- 172.21.15.53:0/2697174762 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b003f070 con 0x7f78c01041e0 2024-04-25T14:51:29.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.639+0000 7f78b6a00640 1 -- 172.21.15.53:0/2697174762 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78b0031d60 con 0x7f78c01041e0 2024-04-25T14:51:29.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.639+0000 7f78bf200640 1 -- 172.21.15.53:0/2697174762 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78c019b380 con 0x7f78c01041e0 2024-04-25T14:51:29.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.639+0000 7f78b6a00640 1 -- 172.21.15.53:0/2697174762 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b0035030 con 0x7f78c01041e0 2024-04-25T14:51:29.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.639+0000 7f78bf200640 1 -- 172.21.15.53:0/2697174762 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78c01a7570 con 0x7f78c01041e0 2024-04-25T14:51:29.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.640+0000 7f78bf200640 1 -- 172.21.15.53:0/2697174762 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78c0068c90 con 0x7f78c01041e0 2024-04-25T14:51:29.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.642+0000 7f78b6a00640 1 -- 172.21.15.53:0/2697174762 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f78b0042070 con 0x7f78c01041e0 2024-04-25T14:51:29.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.644+0000 7f78b6a00640 1 --2- 172.21.15.53:0/2697174762 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f78a00a3d20 0x7f78a00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:29.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.644+0000 7f78b6a00640 1 -- 172.21.15.53:0/2697174762 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8587+0+0 (secure 0 0 0) 0x7f78b00f0cf0 con 0x7f78c01041e0 2024-04-25T14:51:29.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.644+0000 7f78bd400640 1 --2- 172.21.15.53:0/2697174762 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f78a00a3d20 0x7f78a00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:29.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.645+0000 7f78bd400640 1 --2- 172.21.15.53:0/2697174762 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f78a00a3d20 0x7f78a00a61e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f78c00ffed0 tx=0x7f78a8009760 comp rx=0 tx=0).ready entity=mgr.15093 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:29.649 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.648+0000 7f78b6a00640 1 -- 172.21.15.53:0/2697174762 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f78b00bcca0 con 0x7f78c01041e0 2024-04-25T14:51:29.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.896+0000 7f78bf200640 1 -- 172.21.15.53:0/2697174762 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78c019bb00 con 0x7f78c01041e0 2024-04-25T14:51:29.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.897+0000 7f78b6a00640 1 -- 172.21.15.53:0/2697174762 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11862 (secure 0 0 0) 0x7f78b00c0fa0 con 0x7f78c01041e0 2024-04-25T14:51:29.899 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:51:29.899 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14460,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6841/41110727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6840","nonce":41110727},{"type":"v1","addr":"172.21.15.53:6841","nonce":41110727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2966992472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2966992472},{"type":"v1","addr":"172.21.15.150:6833","nonce":2966992472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/94143643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":94143643},{"type":"v1","addr":"172.21.15.183:6833","nonce":94143643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24470,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/506682943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":506682943},{"type":"v1","addr":"172.21.15.53:6835","nonce":506682943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1369488844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1369488844},{"type":"v1","addr":"172.21.15.150:6835","nonce":1369488844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3786232682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3786232682},{"type":"v1","addr":"172.21.15.53:6837","nonce":3786232682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":22,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T14:38:10.037451+0000","modified":"2024-04-25T14:40:40.222021+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24584,"mds_1":24565,"mds_2":24560},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24532":{"gid":24532,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1189328602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1189328602},{"type":"v1","addr":"172.21.15.183:6835","nonce":1189328602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/2344343627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2344343627},{"type":"v1","addr":"172.21.15.150:6837","nonce":2344343627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24544":{"gid":24544,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1163336384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1163336384},{"type":"v1","addr":"172.21.15.183:6837","nonce":1163336384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6839/570000540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":570000540},{"type":"v1","addr":"172.21.15.53:6839","nonce":570000540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.183:6839/1554258915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1554258915},{"type":"v1","addr":"172.21.15.183:6839","nonce":1554258915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/149573616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":149573616},{"type":"v1","addr":"172.21.15.150:6839","nonce":149573616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24560,24565,24584]},"id":1}]} 2024-04-25T14:51:29.899 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 22 2024-04-25T14:51:29.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.900+0000 7f78bf200640 1 -- 172.21.15.53:0/2697174762 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f78a00a3d20 msgr2=0x7f78a00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:29.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.900+0000 7f78bf200640 1 --2- 172.21.15.53:0/2697174762 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f78a00a3d20 0x7f78a00a61e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f78c00ffed0 tx=0x7f78a8009760 comp rx=0 tx=0).stop 2024-04-25T14:51:29.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.900+0000 7f78bf200640 1 -- 172.21.15.53:0/2697174762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78c01041e0 msgr2=0x7f78c01a0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:29.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.900+0000 7f78bf200640 1 --2- 172.21.15.53:0/2697174762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78c01041e0 0x7f78c01a0420 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f78b0005ab0 tx=0x7f78b0003c80 comp rx=0 tx=0).stop 2024-04-25T14:51:29.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.900+0000 7f78bf200640 1 -- 172.21.15.53:0/2697174762 shutdown_connections 2024-04-25T14:51:29.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.900+0000 7f78bf200640 1 --2- 172.21.15.53:0/2697174762 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f78a00a3d20 0x7f78a00a61e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:29.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.900+0000 7f78bf200640 1 --2- 172.21.15.53:0/2697174762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78c019a5d0 0x7f78c019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:29.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.900+0000 7f78bf200640 1 --2- 172.21.15.53:0/2697174762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78c01092b0 0x7f78c01a0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:29.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.900+0000 7f78bf200640 1 --2- 172.21.15.53:0/2697174762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78c01041e0 0x7f78c01a0420 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:29.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.900+0000 7f78bf200640 1 -- 172.21.15.53:0/2697174762 >> 172.21.15.53:0/2697174762 conn(0x7f78c00ffa20 msgr2=0x7f78c0101d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:29.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.900+0000 7f78bf200640 1 -- 172.21.15.53:0/2697174762 shutdown_connections 2024-04-25T14:51:29.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:29.900+0000 7f78bf200640 1 -- 172.21.15.53:0/2697174762 wait complete. 2024-04-25T14:51:29.921 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:51:29.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:29 smithi183 ceph-mon[32584]: pgmap v337: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T14:51:29.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:29 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/602837171' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:51:29.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:29 smithi150 ceph-mon[32504]: pgmap v337: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T14:51:29.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:29 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/602837171' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:51:29.991 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:29 smithi053 ceph-mon[27773]: pgmap v337: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T14:51:29.991 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:29 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/602837171' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:51:30.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.127+0000 7f2906000640 1 -- 172.21.15.53:0/518439923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2900104b70 msgr2=0x7f2900108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:30.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.127+0000 7f2906000640 1 --2- 172.21.15.53:0/518439923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2900104b70 0x7f2900108bb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f28ec009900 tx=0x7f28ec02a410 comp rx=0 tx=0).stop 2024-04-25T14:51:30.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.128+0000 7f2906000640 1 -- 172.21.15.53:0/518439923 shutdown_connections 2024-04-25T14:51:30.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.128+0000 7f2906000640 1 --2- 172.21.15.53:0/518439923 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2900109290 0x7f2900110e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:30.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.128+0000 7f2906000640 1 --2- 172.21.15.53:0/518439923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2900104b70 0x7f2900108bb0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:30.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.128+0000 7f2906000640 1 --2- 172.21.15.53:0/518439923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29001041c0 0x7f29001045a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:30.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.128+0000 7f2906000640 1 -- 172.21.15.53:0/518439923 >> 172.21.15.53:0/518439923 conn(0x7f29000ff9e0 msgr2=0x7f2900101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:30.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.128+0000 7f2906000640 1 -- 172.21.15.53:0/518439923 shutdown_connections 2024-04-25T14:51:30.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.128+0000 7f2906000640 1 -- 172.21.15.53:0/518439923 wait complete. 2024-04-25T14:51:30.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.129+0000 7f2906000640 1 Processor -- start 2024-04-25T14:51:30.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.129+0000 7f2906000640 1 -- start start 2024-04-25T14:51:30.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.129+0000 7f2906000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29001041c0 0x7f29001a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:30.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.129+0000 7f2906000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2900109290 0x7f29001a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:30.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.129+0000 7f2906000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f290019a5e0 0x7f290019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:30.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.129+0000 7f2906000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f29001141d0 con 0x7f290019a5e0 2024-04-25T14:51:30.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.129+0000 7f2906000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2900114050 con 0x7f29001041c0 2024-04-25T14:51:30.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.129+0000 7f2906000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2900114350 con 0x7f2900109290 2024-04-25T14:51:30.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.129+0000 7f2905600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f290019a5e0 0x7f290019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:30.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.129+0000 7f2904c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29001041c0 0x7f29001a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:30.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.130+0000 7f2904c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29001041c0 0x7f29001a0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.53:60670/0 (socket says 172.21.15.53:60670) 2024-04-25T14:51:30.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.130+0000 7f2904c00640 1 -- 172.21.15.53:0/176408065 learned_addr learned my addr 172.21.15.53:0/176408065 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:51:30.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.130+0000 7f28ffe00640 1 --2- 172.21.15.53:0/176408065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2900109290 0x7f29001a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:30.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.130+0000 7f2905600640 1 -- 172.21.15.53:0/176408065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2900109290 msgr2=0x7f29001a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:30.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.130+0000 7f2905600640 1 --2- 172.21.15.53:0/176408065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2900109290 0x7f29001a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:30.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.130+0000 7f2905600640 1 -- 172.21.15.53:0/176408065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29001041c0 msgr2=0x7f29001a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:30.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.130+0000 7f2905600640 1 --2- 172.21.15.53:0/176408065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29001041c0 0x7f29001a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:30.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.130+0000 7f2905600640 1 -- 172.21.15.53:0/176408065 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f290019b100 con 0x7f290019a5e0 2024-04-25T14:51:30.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.130+0000 7f2905600640 1 --2- 172.21.15.53:0/176408065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f290019a5e0 0x7f290019aa90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f28f800da10 tx=0x7f28f800ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:30.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.130+0000 7f28fd600640 1 -- 172.21.15.53:0/176408065 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f800bd70 con 0x7f290019a5e0 2024-04-25T14:51:30.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.131+0000 7f28fd600640 1 -- 172.21.15.53:0/176408065 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28f8003e20 con 0x7f290019a5e0 2024-04-25T14:51:30.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.131+0000 7f2906000640 1 -- 172.21.15.53:0/176408065 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f290019b3f0 con 0x7f290019a5e0 2024-04-25T14:51:30.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.131+0000 7f28fd600640 1 -- 172.21.15.53:0/176408065 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f80106c0 con 0x7f290019a5e0 2024-04-25T14:51:30.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.131+0000 7f2906000640 1 -- 172.21.15.53:0/176408065 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29001a75f0 con 0x7f290019a5e0 2024-04-25T14:51:30.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.132+0000 7f2906000640 1 -- 172.21.15.53:0/176408065 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29000690e0 con 0x7f290019a5e0 2024-04-25T14:51:30.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.134+0000 7f28fd600640 1 -- 172.21.15.53:0/176408065 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f28f8004270 con 0x7f290019a5e0 2024-04-25T14:51:30.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.135+0000 7f28fd600640 1 --2- 172.21.15.53:0/176408065 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f28e80a3d20 0x7f28e80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:30.137 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.136+0000 7f28fd600640 1 -- 172.21.15.53:0/176408065 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8587+0+0 (secure 0 0 0) 0x7f28f80d0f40 con 0x7f290019a5e0 2024-04-25T14:51:30.137 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.136+0000 7f2904c00640 1 --2- 172.21.15.53:0/176408065 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f28e80a3d20 0x7f28e80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:30.137 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.136+0000 7f2904c00640 1 --2- 172.21.15.53:0/176408065 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f28e80a3d20 0x7f28e80a61e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f28f4009cd0 tx=0x7f28f4005f90 comp rx=0 tx=0).ready entity=mgr.15093 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:30.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.138+0000 7f28fd600640 1 -- 172.21.15.53:0/176408065 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f28f809d540 con 0x7f290019a5e0 2024-04-25T14:51:30.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.335+0000 7f2906000640 1 -- 172.21.15.53:0/176408065 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f290019bb10 con 0x7f290019a5e0 2024-04-25T14:51:30.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.340+0000 7f28fd600640 1 -- 172.21.15.53:0/176408065 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+23201 (secure 0 0 0) 0x7f290019bb10 con 0x7f290019a5e0 2024-04-25T14:51:30.341 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-25T14:51:30.342 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":103,"fsid":"fd30e6ce-030f-11ef-bc93-c7b262605968","created":"2024-04-25T14:28:54.330326+0000","modified":"2024-04-25T14:51:20.935398+0000","last_up_change":"2024-04-25T14:37:10.921148+0000","last_in_change":"2024-04-25T14:36:49.566459+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:33:02.338115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:38:07.517974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:38:08.994122+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aecfbaf0-39da-4749-a49b-c7205dabd4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6803","nonce":464176015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6805","nonce":464176015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6809","nonce":464176015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":464176015},{"type":"v1","addr":"172.21.15.53:6807","nonce":464176015}]},"public_addr":"172.21.15.53:6803/464176015","cluster_addr":"172.21.15.53:6805/464176015","heartbeat_back_addr":"172.21.15.53:6809/464176015","heartbeat_front_addr":"172.21.15.53:6807/464176015","state":["exists","up"]},{"osd":1,"uuid":"2c4a21ae-c43e-4839-8a56-c3fc63aeb7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6801","nonce":1304621982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6803","nonce":1304621982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6807","nonce":1304621982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1304621982},{"type":"v1","addr":"172.21.15.150:6805","nonce":1304621982}]},"public_addr":"172.21.15.150:6801/1304621982","cluster_addr":"172.21.15.150:6803/1304621982","heartbeat_back_addr":"172.21.15.150:6807/1304621982","heartbeat_front_addr":"172.21.15.150:6805/1304621982","state":["exists","up"]},{"osd":2,"uuid":"e8effd43-adc0-472a-9eed-a9273b06dc0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6801","nonce":574509311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6803","nonce":574509311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6807","nonce":574509311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":574509311},{"type":"v1","addr":"172.21.15.183:6805","nonce":574509311}]},"public_addr":"172.21.15.183:6801/574509311","cluster_addr":"172.21.15.183:6803/574509311","heartbeat_back_addr":"172.21.15.183:6807/574509311","heartbeat_front_addr":"172.21.15.183:6805/574509311","state":["exists","up"]},{"osd":3,"uuid":"e7cfced4-9b40-40de-a303-75ea8d5df4a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6811","nonce":570672219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6813","nonce":570672219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6817","nonce":570672219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":570672219},{"type":"v1","addr":"172.21.15.53:6815","nonce":570672219}]},"public_addr":"172.21.15.53:6811/570672219","cluster_addr":"172.21.15.53:6813/570672219","heartbeat_back_addr":"172.21.15.53:6817/570672219","heartbeat_front_addr":"172.21.15.53:6815/570672219","state":["exists","up"]},{"osd":4,"uuid":"44a9e070-2282-43f1-9f05-b1f8ff81e540","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6809","nonce":1526850103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6811","nonce":1526850103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6815","nonce":1526850103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1526850103},{"type":"v1","addr":"172.21.15.150:6813","nonce":1526850103}]},"public_addr":"172.21.15.150:6809/1526850103","cluster_addr":"172.21.15.150:6811/1526850103","heartbeat_back_addr":"172.21.15.150:6815/1526850103","heartbeat_front_addr":"172.21.15.150:6813/1526850103","state":["exists","up"]},{"osd":5,"uuid":"2c5619d2-c0f9-47d6-bf7e-2f4afd1b09f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6809","nonce":773726576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6811","nonce":773726576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6815","nonce":773726576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":773726576},{"type":"v1","addr":"172.21.15.183:6813","nonce":773726576}]},"public_addr":"172.21.15.183:6809/773726576","cluster_addr":"172.21.15.183:6811/773726576","heartbeat_back_addr":"172.21.15.183:6815/773726576","heartbeat_front_addr":"172.21.15.183:6813/773726576","state":["exists","up"]},{"osd":6,"uuid":"1cf74d94-c118-4738-b6db-040af458a8eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6819","nonce":355884319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6821","nonce":355884319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6825","nonce":355884319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":355884319},{"type":"v1","addr":"172.21.15.53:6823","nonce":355884319}]},"public_addr":"172.21.15.53:6819/355884319","cluster_addr":"172.21.15.53:6821/355884319","heartbeat_back_addr":"172.21.15.53:6825/355884319","heartbeat_front_addr":"172.21.15.53:6823/355884319","state":["exists","up"]},{"osd":7,"uuid":"0ce97c04-cd41-4080-bbbc-be9cf5fadc02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6817","nonce":229022562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6819","nonce":229022562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6823","nonce":229022562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":229022562},{"type":"v1","addr":"172.21.15.150:6821","nonce":229022562}]},"public_addr":"172.21.15.150:6817/229022562","cluster_addr":"172.21.15.150:6819/229022562","heartbeat_back_addr":"172.21.15.150:6823/229022562","heartbeat_front_addr":"172.21.15.150:6821/229022562","state":["exists","up"]},{"osd":8,"uuid":"5d57787d-2a7c-4514-bd01-357ced413f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6817","nonce":301003844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6819","nonce":301003844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6823","nonce":301003844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":301003844},{"type":"v1","addr":"172.21.15.183:6821","nonce":301003844}]},"public_addr":"172.21.15.183:6817/301003844","cluster_addr":"172.21.15.183:6819/301003844","heartbeat_back_addr":"172.21.15.183:6823/301003844","heartbeat_front_addr":"172.21.15.183:6821/301003844","state":["exists","up"]},{"osd":9,"uuid":"e11dd8eb-c55a-4d70-8c58-457540b7a50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6827","nonce":3449260978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6829","nonce":3449260978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6833","nonce":3449260978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3449260978},{"type":"v1","addr":"172.21.15.53:6831","nonce":3449260978}]},"public_addr":"172.21.15.53:6827/3449260978","cluster_addr":"172.21.15.53:6829/3449260978","heartbeat_back_addr":"172.21.15.53:6833/3449260978","heartbeat_front_addr":"172.21.15.53:6831/3449260978","state":["exists","up"]},{"osd":10,"uuid":"5de4803d-ad37-4a90-add5-0048f44fc4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6825","nonce":3435992308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6827","nonce":3435992308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6831","nonce":3435992308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3435992308},{"type":"v1","addr":"172.21.15.150:6829","nonce":3435992308}]},"public_addr":"172.21.15.150:6825/3435992308","cluster_addr":"172.21.15.150:6827/3435992308","heartbeat_back_addr":"172.21.15.150:6831/3435992308","heartbeat_front_addr":"172.21.15.150:6829/3435992308","state":["exists","up"]},{"osd":11,"uuid":"0e819a68-ed56-477e-b9c0-81e8cc0bce11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6825","nonce":1957842686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6827","nonce":1957842686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6831","nonce":1957842686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1957842686},{"type":"v1","addr":"172.21.15.183:6829","nonce":1957842686}]},"public_addr":"172.21.15.183:6825/1957842686","cluster_addr":"172.21.15.183:6827/1957842686","heartbeat_back_addr":"172.21.15.183:6831/1957842686","heartbeat_front_addr":"172.21.15.183:6829/1957842686","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:09.840629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:34.769537+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:32:58.112969+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:33:29.656790+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:33:55.181287+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:34:19.772645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:34:53.833921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:35:19.076733+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:35:43.163390+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:36:16.568340+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:36:42.492768+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:37:07.944639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9},{"from":5,"to":6}]},{"pgid":"3.38","mappings":[{"from":5,"to":7},{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6800/4154933108":"2024-04-26T14:40:40.168940+0000","172.21.15.53:0/3916258673":"2024-04-26T14:40:40.168940+0000","172.21.15.53:0/309091034":"2024-04-26T14:40:40.168940+0000","172.21.15.53:0/4267178590":"2024-04-26T14:40:40.168940+0000","172.21.15.53:0/920465993":"2024-04-26T14:40:40.168940+0000","172.21.15.53:6800/3887954889":"2024-04-26T14:30:14.162383+0000","172.21.15.53:6801/4154933108":"2024-04-26T14:40:40.168940+0000","172.21.15.53:0/1014501022":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/2576091441":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/1233527569":"2024-04-26T14:29:38.106242+0000","172.21.15.53:6801/3887954889":"2024-04-26T14:30:14.162383+0000","172.21.15.53:0/931889810":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/177196141":"2024-04-26T14:30:14.162383+0000","172.21.15.53:6800/1722497390":"2024-04-26T14:29:38.106242+0000","172.21.15.53:6801/1722497390":"2024-04-26T14:29:38.106242+0000","172.21.15.53:0/1927549198":"2024-04-26T14:30:14.162383+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:51:30.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.342+0000 7f2906000640 1 -- 172.21.15.53:0/176408065 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f28e80a3d20 msgr2=0x7f28e80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:30.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.342+0000 7f2906000640 1 --2- 172.21.15.53:0/176408065 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f28e80a3d20 0x7f28e80a61e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f28f4009cd0 tx=0x7f28f4005f90 comp rx=0 tx=0).stop 2024-04-25T14:51:30.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.342+0000 7f2906000640 1 -- 172.21.15.53:0/176408065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f290019a5e0 msgr2=0x7f290019aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:30.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.342+0000 7f2906000640 1 --2- 172.21.15.53:0/176408065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f290019a5e0 0x7f290019aa90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f28f800da10 tx=0x7f28f800ded0 comp rx=0 tx=0).stop 2024-04-25T14:51:30.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.342+0000 7f2906000640 1 -- 172.21.15.53:0/176408065 shutdown_connections 2024-04-25T14:51:30.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.342+0000 7f2906000640 1 --2- 172.21.15.53:0/176408065 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f28e80a3d20 0x7f28e80a61e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:30.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.342+0000 7f2906000640 1 --2- 172.21.15.53:0/176408065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f290019a5e0 0x7f290019aa90 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:30.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.342+0000 7f2906000640 1 --2- 172.21.15.53:0/176408065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2900109290 0x7f29001a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:30.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.342+0000 7f2906000640 1 --2- 172.21.15.53:0/176408065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29001041c0 0x7f29001a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:30.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.342+0000 7f2906000640 1 -- 172.21.15.53:0/176408065 >> 172.21.15.53:0/176408065 conn(0x7f29000ff9e0 msgr2=0x7f2900100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:30.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.343+0000 7f2906000640 1 -- 172.21.15.53:0/176408065 shutdown_connections 2024-04-25T14:51:30.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.343+0000 7f2906000640 1 -- 172.21.15.53:0/176408065 wait complete. 2024-04-25T14:51:30.363 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T14:51:30.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.566+0000 7f938d000640 1 -- 172.21.15.53:0/442414961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9388102fb0 msgr2=0x7f9388103390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:30.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.566+0000 7f938d000640 1 --2- 172.21.15.53:0/442414961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9388102fb0 0x7f9388103390 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9378009ca0 tx=0x7f937802a950 comp rx=0 tx=0).stop 2024-04-25T14:51:30.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.566+0000 7f938d000640 1 -- 172.21.15.53:0/442414961 shutdown_connections 2024-04-25T14:51:30.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.566+0000 7f938d000640 1 --2- 172.21.15.53:0/442414961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f938810c3d0 0x7f938810e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:30.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.566+0000 7f938d000640 1 --2- 172.21.15.53:0/442414961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93881038d0 0x7f938810bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:30.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.566+0000 7f938d000640 1 --2- 172.21.15.53:0/442414961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9388102fb0 0x7f9388103390 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:30.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.566+0000 7f938d000640 1 -- 172.21.15.53:0/442414961 >> 172.21.15.53:0/442414961 conn(0x7f93880fc7d0 msgr2=0x7f93880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:30.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.566+0000 7f938d000640 1 -- 172.21.15.53:0/442414961 shutdown_connections 2024-04-25T14:51:30.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.566+0000 7f938d000640 1 -- 172.21.15.53:0/442414961 wait complete. 2024-04-25T14:51:30.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.568+0000 7f938d000640 1 Processor -- start 2024-04-25T14:51:30.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.568+0000 7f938d000640 1 -- start start 2024-04-25T14:51:30.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.568+0000 7f938d000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9388102fb0 0x7f9388198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:30.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.568+0000 7f938d000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93881038d0 0x7f9388198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:30.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.568+0000 7f938d000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f938810c3d0 0x7f938819cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:30.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.568+0000 7f938d000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f93881120b0 con 0x7f938810c3d0 2024-04-25T14:51:30.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.568+0000 7f938d000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9388111f30 con 0x7f9388102fb0 2024-04-25T14:51:30.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.568+0000 7f938d000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9388112230 con 0x7f93881038d0 2024-04-25T14:51:30.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.569+0000 7f9387e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f938810c3d0 0x7f938819cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:30.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.569+0000 7f9386a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93881038d0 0x7f9388198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:30.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.569+0000 7f9386a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93881038d0 0x7f9388198b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:45324/0 (socket says 172.21.15.53:45324) 2024-04-25T14:51:30.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.569+0000 7f9386a00640 1 -- 172.21.15.53:0/1372451250 learned_addr learned my addr 172.21.15.53:0/1372451250 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:51:30.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.569+0000 7f9387400640 1 --2- 172.21.15.53:0/1372451250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9388102fb0 0x7f9388198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:30.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.569+0000 7f9386a00640 1 -- 172.21.15.53:0/1372451250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9388102fb0 msgr2=0x7f9388198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:30.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.569+0000 7f9386a00640 1 --2- 172.21.15.53:0/1372451250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9388102fb0 0x7f9388198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:30.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.569+0000 7f9386a00640 1 -- 172.21.15.53:0/1372451250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f938810c3d0 msgr2=0x7f938819cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:30.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.569+0000 7f9386a00640 1 --2- 172.21.15.53:0/1372451250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f938810c3d0 0x7f938819cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:30.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.569+0000 7f9386a00640 1 -- 172.21.15.53:0/1372451250 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f938819d620 con 0x7f93881038d0 2024-04-25T14:51:30.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.570+0000 7f9386a00640 1 --2- 172.21.15.53:0/1372451250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93881038d0 0x7f9388198b70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f937000e9f0 tx=0x7f937000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:30.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.570+0000 7f937fe00640 1 -- 172.21.15.53:0/1372451250 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f937000ccc0 con 0x7f93881038d0 2024-04-25T14:51:30.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.570+0000 7f938d000640 1 -- 172.21.15.53:0/1372451250 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f938819d910 con 0x7f93881038d0 2024-04-25T14:51:30.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.570+0000 7f937fe00640 1 -- 172.21.15.53:0/1372451250 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93700108e0 con 0x7f93881038d0 2024-04-25T14:51:30.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.570+0000 7f937fe00640 1 -- 172.21.15.53:0/1372451250 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9370018880 con 0x7f93881038d0 2024-04-25T14:51:30.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.570+0000 7f938d000640 1 -- 172.21.15.53:0/1372451250 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93881a51a0 con 0x7f93881038d0 2024-04-25T14:51:30.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.571+0000 7f938d000640 1 -- 172.21.15.53:0/1372451250 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9388068c90 con 0x7f93881038d0 2024-04-25T14:51:30.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.573+0000 7f937fe00640 1 -- 172.21.15.53:0/1372451250 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f9370010430 con 0x7f93881038d0 2024-04-25T14:51:30.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.575+0000 7f937fe00640 1 --2- 172.21.15.53:0/1372451250 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f936c0a3d20 0x7f936c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:30.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.575+0000 7f937fe00640 1 -- 172.21.15.53:0/1372451250 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8587+0+0 (secure 0 0 0) 0x7f93700d0ff0 con 0x7f93881038d0 2024-04-25T14:51:30.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.575+0000 7f9387400640 1 --2- 172.21.15.53:0/1372451250 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f936c0a3d20 0x7f936c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:30.577 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.576+0000 7f9387400640 1 --2- 172.21.15.53:0/1372451250 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f936c0a3d20 0x7f936c0a61e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f937802af40 tx=0x7f9378005d80 comp rx=0 tx=0).ready entity=mgr.15093 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:30.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.577+0000 7f937fe00640 1 -- 172.21.15.53:0/1372451250 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f937009cfa0 con 0x7f93881038d0 2024-04-25T14:51:30.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.816+0000 7f938d000640 1 -- 172.21.15.53:0/1372451250 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f9388107470 con 0x7f93881038d0 2024-04-25T14:51:30.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.826+0000 7f937fe00640 1 -- 172.21.15.53:0/1372451250 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v23) ==== 161+0+0 (secure 0 0 0) 0x7f93700a12a0 con 0x7f93881038d0 2024-04-25T14:51:30.827 INFO:teuthology.orchestra.run.smithi053.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T14:51:30.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.829+0000 7f938d000640 1 -- 172.21.15.53:0/1372451250 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f936c0a3d20 msgr2=0x7f936c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:30.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.829+0000 7f938d000640 1 --2- 172.21.15.53:0/1372451250 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f936c0a3d20 0x7f936c0a61e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f937802af40 tx=0x7f9378005d80 comp rx=0 tx=0).stop 2024-04-25T14:51:30.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.829+0000 7f938d000640 1 -- 172.21.15.53:0/1372451250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93881038d0 msgr2=0x7f9388198b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:30.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.829+0000 7f938d000640 1 --2- 172.21.15.53:0/1372451250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93881038d0 0x7f9388198b70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f937000e9f0 tx=0x7f937000eeb0 comp rx=0 tx=0).stop 2024-04-25T14:51:30.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.829+0000 7f938d000640 1 -- 172.21.15.53:0/1372451250 shutdown_connections 2024-04-25T14:51:30.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.829+0000 7f938d000640 1 --2- 172.21.15.53:0/1372451250 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f936c0a3d20 0x7f936c0a61e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:30.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.829+0000 7f938d000640 1 --2- 172.21.15.53:0/1372451250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f938810c3d0 0x7f938819cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:30.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.829+0000 7f938d000640 1 --2- 172.21.15.53:0/1372451250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93881038d0 0x7f9388198b70 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:30.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.829+0000 7f938d000640 1 --2- 172.21.15.53:0/1372451250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9388102fb0 0x7f9388198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:30.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.829+0000 7f938d000640 1 -- 172.21.15.53:0/1372451250 >> 172.21.15.53:0/1372451250 conn(0x7f93880fc7d0 msgr2=0x7f93880fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:30.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.829+0000 7f938d000640 1 -- 172.21.15.53:0/1372451250 shutdown_connections 2024-04-25T14:51:30.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:30.829+0000 7f938d000640 1 -- 172.21.15.53:0/1372451250 wait complete. 2024-04-25T14:51:30.851 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T14:51:30.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:30 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/2697174762' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:51:30.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:30 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/176408065' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:51:30.955 INFO:journalctl@ceph.mds.i.smithi183.stdout:Apr 25 14:51:30 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-i[62892]: ignoring --setuser ceph since I am not root 2024-04-25T14:51:30.955 INFO:journalctl@ceph.mds.i.smithi183.stdout:Apr 25 14:51:30 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-i[62892]: ignoring --setgroup ceph since I am not root 2024-04-25T14:51:30.955 INFO:journalctl@ceph.mds.i.smithi183.stdout:Apr 25 14:51:30 smithi183 ceph-mds[62896]: -- 172.21.15.183:0/3349489278 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561030255a40 con 0x561030228800 2024-04-25T14:51:30.955 INFO:journalctl@ceph.mds.i.smithi183.stdout:Apr 25 14:51:30 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-i[62892]: starting mds.i at 2024-04-25T14:51:30.956 INFO:journalctl@ceph.mds.c.smithi183.stdout:Apr 25 14:51:30 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-c[62590]: ignoring --setuser ceph since I am not root 2024-04-25T14:51:30.956 INFO:journalctl@ceph.mds.c.smithi183.stdout:Apr 25 14:51:30 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-c[62590]: ignoring --setgroup ceph since I am not root 2024-04-25T14:51:30.956 INFO:journalctl@ceph.mds.c.smithi183.stdout:Apr 25 14:51:30 smithi183 ceph-mds[62594]: -- 172.21.15.183:0/2261980933 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c4d9289a40 con 0x55c4d925cc00 2024-04-25T14:51:30.956 INFO:journalctl@ceph.mds.c.smithi183.stdout:Apr 25 14:51:30 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-c[62590]: starting mds.c at 2024-04-25T14:51:30.956 INFO:journalctl@ceph.mds.f.smithi183.stdout:Apr 25 14:51:30 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-f[62283]: ignoring --setuser ceph since I am not root 2024-04-25T14:51:30.956 INFO:journalctl@ceph.mds.f.smithi183.stdout:Apr 25 14:51:30 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-f[62283]: ignoring --setgroup ceph since I am not root 2024-04-25T14:51:30.957 INFO:journalctl@ceph.mds.f.smithi183.stdout:Apr 25 14:51:30 smithi183 ceph-mds[62287]: -- 172.21.15.183:0/2629340017 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562a036aba40 con 0x562a0367e800 2024-04-25T14:51:30.957 INFO:journalctl@ceph.mds.f.smithi183.stdout:Apr 25 14:51:30 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-f[62283]: starting mds.f at 2024-04-25T14:51:30.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:30 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/2697174762' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:51:30.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:30 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/176408065' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:51:30.961 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 25 14:51:30 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-h[62587]: ignoring --setuser ceph since I am not root 2024-04-25T14:51:30.961 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 25 14:51:30 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-h[62587]: ignoring --setgroup ceph since I am not root 2024-04-25T14:51:30.961 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 25 14:51:30 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-h[62587]: starting mds.h at 2024-04-25T14:51:30.961 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 25 14:51:30 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-b[62911]: ignoring --setuser ceph since I am not root 2024-04-25T14:51:30.962 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 25 14:51:30 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-b[62911]: ignoring --setgroup ceph since I am not root 2024-04-25T14:51:30.962 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 25 14:51:30 smithi150 ceph-mds[62915]: -- 172.21.15.150:0/1108824066 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560342d21a40 con 0x560342cf4c00 2024-04-25T14:51:30.962 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 25 14:51:30 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-b[62911]: starting mds.b at 2024-04-25T14:51:31.032 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:30 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/2697174762' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:51:31.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:30 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/176408065' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:51:31.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:30 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mon-a[27768]: 2024-04-25T14:51:30.818+0000 7f78bd000700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T14:51:31.033 INFO:journalctl@ceph.mds.g.smithi053.stdout:Apr 25 14:51:30 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-g[68985]: ignoring --setuser ceph since I am not root 2024-04-25T14:51:31.033 INFO:journalctl@ceph.mds.g.smithi053.stdout:Apr 25 14:51:30 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-g[68985]: ignoring --setgroup ceph since I am not root 2024-04-25T14:51:31.033 INFO:journalctl@ceph.mds.g.smithi053.stdout:Apr 25 14:51:30 smithi053 ceph-mds[68989]: -- 172.21.15.53:0/999116055 <== mon.0 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c9a9917a40 con 0x55c9a98ea800 2024-04-25T14:51:31.033 INFO:journalctl@ceph.mds.g.smithi053.stdout:Apr 25 14:51:30 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-g[68985]: starting mds.g at 2024-04-25T14:51:31.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.035+0000 7f2679e00640 1 -- 172.21.15.53:0/622469739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2674101730 msgr2=0x7f2674105bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:31.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.035+0000 7f2679e00640 1 --2- 172.21.15.53:0/622469739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2674101730 0x7f2674105bf0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2660009900 tx=0x7f266002a480 comp rx=0 tx=0).stop 2024-04-25T14:51:31.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.035+0000 7f2679e00640 1 -- 172.21.15.53:0/622469739 shutdown_connections 2024-04-25T14:51:31.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.035+0000 7f2679e00640 1 --2- 172.21.15.53:0/622469739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2674106210 0x7f26741066a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:31.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.035+0000 7f2679e00640 1 --2- 172.21.15.53:0/622469739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2674101730 0x7f2674105bf0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:31.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.035+0000 7f2679e00640 1 --2- 172.21.15.53:0/622469739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2674100e10 0x7f26741011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:31.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.035+0000 7f2679e00640 1 -- 172.21.15.53:0/622469739 >> 172.21.15.53:0/622469739 conn(0x7f26740fc840 msgr2=0x7f26740fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:31.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.036+0000 7f2679e00640 1 -- 172.21.15.53:0/622469739 shutdown_connections 2024-04-25T14:51:31.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.036+0000 7f2679e00640 1 -- 172.21.15.53:0/622469739 wait complete. 2024-04-25T14:51:31.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.036+0000 7f2679e00640 1 Processor -- start 2024-04-25T14:51:31.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.037+0000 7f2679e00640 1 -- start start 2024-04-25T14:51:31.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.037+0000 7f2679e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2674100e10 0x7f26741a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:31.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.037+0000 7f2679e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2674101730 0x7f26741a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:31.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.037+0000 7f2679e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2674106210 0x7f267419eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:31.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.037+0000 7f2679e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f26741186d0 con 0x7f2674106210 2024-04-25T14:51:31.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.037+0000 7f2679e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2674118550 con 0x7f2674100e10 2024-04-25T14:51:31.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.037+0000 7f2679e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2674118850 con 0x7f2674101730 2024-04-25T14:51:31.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.037+0000 7f2673e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2674101730 0x7f26741a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:31.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.037+0000 7f2673e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2674101730 0x7f26741a4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:45354/0 (socket says 172.21.15.53:45354) 2024-04-25T14:51:31.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.037+0000 7f2673e00640 1 -- 172.21.15.53:0/4113991623 learned_addr learned my addr 172.21.15.53:0/4113991623 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:51:31.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.037+0000 7f2679400640 1 --2- 172.21.15.53:0/4113991623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2674106210 0x7f267419eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:31.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.037+0000 7f2678a00640 1 --2- 172.21.15.53:0/4113991623 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2674100e10 0x7f26741a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:31.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.038+0000 7f2678a00640 1 -- 172.21.15.53:0/4113991623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2674101730 msgr2=0x7f26741a4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:31.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.038+0000 7f2678a00640 1 --2- 172.21.15.53:0/4113991623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2674101730 0x7f26741a4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:31.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.038+0000 7f2678a00640 1 -- 172.21.15.53:0/4113991623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2674106210 msgr2=0x7f267419eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:31.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.038+0000 7f2678a00640 1 --2- 172.21.15.53:0/4113991623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2674106210 0x7f267419eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:31.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.038+0000 7f2678a00640 1 -- 172.21.15.53:0/4113991623 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267419f230 con 0x7f2674100e10 2024-04-25T14:51:31.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.038+0000 7f2678a00640 1 --2- 172.21.15.53:0/4113991623 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2674100e10 0x7f26741a4900 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f266800e9f0 tx=0x7f266800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:31.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.038+0000 7f2671600640 1 -- 172.21.15.53:0/4113991623 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266800ccc0 con 0x7f2674100e10 2024-04-25T14:51:31.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.038+0000 7f2671600640 1 -- 172.21.15.53:0/4113991623 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26680108e0 con 0x7f2674100e10 2024-04-25T14:51:31.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.038+0000 7f2679e00640 1 -- 172.21.15.53:0/4113991623 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f267419f520 con 0x7f2674100e10 2024-04-25T14:51:31.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.038+0000 7f2671600640 1 -- 172.21.15.53:0/4113991623 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2668018880 con 0x7f2674100e10 2024-04-25T14:51:31.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.039+0000 7f2679e00640 1 -- 172.21.15.53:0/4113991623 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26741ab7c0 con 0x7f2674100e10 2024-04-25T14:51:31.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.040+0000 7f2679e00640 1 -- 172.21.15.53:0/4113991623 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26740690e0 con 0x7f2674100e10 2024-04-25T14:51:31.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.042+0000 7f2671600640 1 -- 172.21.15.53:0/4113991623 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f2668010430 con 0x7f2674100e10 2024-04-25T14:51:31.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.043+0000 7f2671600640 1 --2- 172.21.15.53:0/4113991623 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f265c0a3da0 0x7f265c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:31.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.043+0000 7f2671600640 1 -- 172.21.15.53:0/4113991623 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8801+0+0 (secure 0 0 0) 0x7f26680d0c00 con 0x7f2674100e10 2024-04-25T14:51:31.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.043+0000 7f2673e00640 1 --2- 172.21.15.53:0/4113991623 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f265c0a3da0 0x7f265c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:31.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.044+0000 7f2673e00640 1 --2- 172.21.15.53:0/4113991623 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f265c0a3da0 0x7f265c0a6260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2660031440 tx=0x7f2660005a20 comp rx=0 tx=0).ready entity=mgr.15093 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:31.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.047+0000 7f2671600640 1 -- 172.21.15.53:0/4113991623 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f266809cad0 con 0x7f2674100e10 2024-04-25T14:51:31.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.295+0000 7f2679e00640 1 -- 172.21.15.53:0/4113991623 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f26741ab9f0 con 0x7f2674100e10 2024-04-25T14:51:31.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.827+0000 7f2671600640 1 -- 172.21.15.53:0/4113991623 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v24) ==== 104+0+0 (secure 0 0 0) 0x7f26680a0dd0 con 0x7f2674100e10 2024-04-25T14:51:31.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.829+0000 7f2679e00640 1 -- 172.21.15.53:0/4113991623 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f265c0a3da0 msgr2=0x7f265c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:31.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.829+0000 7f2679e00640 1 --2- 172.21.15.53:0/4113991623 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f265c0a3da0 0x7f265c0a6260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2660031440 tx=0x7f2660005a20 comp rx=0 tx=0).stop 2024-04-25T14:51:31.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.829+0000 7f2679e00640 1 -- 172.21.15.53:0/4113991623 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2674100e10 msgr2=0x7f26741a4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:31.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.829+0000 7f2679e00640 1 --2- 172.21.15.53:0/4113991623 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2674100e10 0x7f26741a4900 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f266800e9f0 tx=0x7f266800eeb0 comp rx=0 tx=0).stop 2024-04-25T14:51:31.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.830+0000 7f2679e00640 1 -- 172.21.15.53:0/4113991623 shutdown_connections 2024-04-25T14:51:31.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.830+0000 7f2679e00640 1 --2- 172.21.15.53:0/4113991623 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f265c0a3da0 0x7f265c0a6260 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:31.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.830+0000 7f2679e00640 1 --2- 172.21.15.53:0/4113991623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2674106210 0x7f267419eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:31.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.830+0000 7f2679e00640 1 --2- 172.21.15.53:0/4113991623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2674101730 0x7f26741a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:31.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.830+0000 7f2679e00640 1 --2- 172.21.15.53:0/4113991623 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2674100e10 0x7f26741a4900 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:31.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.830+0000 7f2679e00640 1 -- 172.21.15.53:0/4113991623 >> 172.21.15.53:0/4113991623 conn(0x7f26740fc840 msgr2=0x7f26740fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:31.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.831+0000 7f2679e00640 1 -- 172.21.15.53:0/4113991623 shutdown_connections 2024-04-25T14:51:31.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:31.831+0000 7f2679e00640 1 -- 172.21.15.53:0/4113991623 wait complete. 2024-04-25T14:51:31.853 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T14:51:31.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:31 smithi183 ceph-mon[32584]: pgmap v338: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T14:51:31.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:31 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/1372451250' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T14:51:31.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:31 smithi183 ceph-mon[32584]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T14:51:31.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:31 smithi183 ceph-mon[32584]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T14:51:31.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:31 smithi183 ceph-mon[32584]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T14:51:31.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:31 smithi183 ceph-mon[32584]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T14:51:31.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:31 smithi183 ceph-mon[32584]: osdmap e104: 12 total, 12 up, 12 in 2024-04-25T14:51:31.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:31 smithi183 ceph-mon[32584]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-25T14:51:31.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:31 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/4113991623' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:51:31.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:31 smithi183 ceph-mon[32584]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:51:31.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:31 smithi150 ceph-mon[32504]: pgmap v338: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T14:51:31.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:31 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/1372451250' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T14:51:31.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:31 smithi150 ceph-mon[32504]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T14:51:31.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:31 smithi150 ceph-mon[32504]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T14:51:31.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:31 smithi150 ceph-mon[32504]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T14:51:31.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:31 smithi150 ceph-mon[32504]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T14:51:31.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:31 smithi150 ceph-mon[32504]: osdmap e104: 12 total, 12 up, 12 in 2024-04-25T14:51:31.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:31 smithi150 ceph-mon[32504]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-25T14:51:31.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:31 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/4113991623' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:51:31.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:31 smithi150 ceph-mon[32504]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:51:31.962 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:31 smithi053 ceph-mon[27773]: pgmap v338: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T14:51:31.962 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:31 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/1372451250' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T14:51:31.963 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:31 smithi053 ceph-mon[27773]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T14:51:31.963 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:31 smithi053 ceph-mon[27773]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T14:51:31.963 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:31 smithi053 ceph-mon[27773]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T14:51:31.963 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:31 smithi053 ceph-mon[27773]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T14:51:31.963 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:31 smithi053 ceph-mon[27773]: osdmap e104: 12 total, 12 up, 12 in 2024-04-25T14:51:31.963 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:31 smithi053 ceph-mon[27773]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-25T14:51:31.963 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:31 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/4113991623' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:51:31.963 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:31 smithi053 ceph-mon[27773]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:51:32.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.046+0000 7f2d5a000640 1 -- 172.21.15.53:0/695777238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d54074c40 msgr2=0x7f2d5410e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:32.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.046+0000 7f2d5a000640 1 --2- 172.21.15.53:0/695777238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d54074c40 0x7f2d5410e160 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2d40009900 tx=0x7f2d4002a480 comp rx=0 tx=0).stop 2024-04-25T14:51:32.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.046+0000 7f2d5a000640 1 -- 172.21.15.53:0/695777238 shutdown_connections 2024-04-25T14:51:32.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.046+0000 7f2d5a000640 1 --2- 172.21.15.53:0/695777238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2d5410e6a0 0x7f2d54110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:32.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.046+0000 7f2d5a000640 1 --2- 172.21.15.53:0/695777238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d54074c40 0x7f2d5410e160 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:32.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.046+0000 7f2d5a000640 1 --2- 172.21.15.53:0/695777238 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d54074320 0x7f2d54074700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:32.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.046+0000 7f2d5a000640 1 -- 172.21.15.53:0/695777238 >> 172.21.15.53:0/695777238 conn(0x7f2d540fd8f0 msgr2=0x7f2d540ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:32.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.047+0000 7f2d5a000640 1 -- 172.21.15.53:0/695777238 shutdown_connections 2024-04-25T14:51:32.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.047+0000 7f2d5a000640 1 -- 172.21.15.53:0/695777238 wait complete. 2024-04-25T14:51:32.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.048+0000 7f2d5a000640 1 Processor -- start 2024-04-25T14:51:32.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.048+0000 7f2d5a000640 1 -- start start 2024-04-25T14:51:32.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.048+0000 7f2d5a000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d54074320 0x7f2d541a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:32.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.048+0000 7f2d5a000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2d54074c40 0x7f2d541a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:32.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.048+0000 7f2d5a000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d5410e6a0 0x7f2d5419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:32.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.048+0000 7f2d5a000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2d541142b0 con 0x7f2d54074320 2024-04-25T14:51:32.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.048+0000 7f2d5a000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2d54114130 con 0x7f2d5410e6a0 2024-04-25T14:51:32.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.048+0000 7f2d5a000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2d54114430 con 0x7f2d54074c40 2024-04-25T14:51:32.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.048+0000 7f2d58c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d54074320 0x7f2d541a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:32.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.048+0000 7f2d58c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d54074320 0x7f2d541a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57482/0 (socket says 172.21.15.53:57482) 2024-04-25T14:51:32.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.048+0000 7f2d58c00640 1 -- 172.21.15.53:0/728759313 learned_addr learned my addr 172.21.15.53:0/728759313 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:51:32.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.048+0000 7f2d53e00640 1 --2- 172.21.15.53:0/728759313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2d54074c40 0x7f2d541a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:32.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.048+0000 7f2d59600640 1 --2- 172.21.15.53:0/728759313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d5410e6a0 0x7f2d5419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:32.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.049+0000 7f2d58c00640 1 -- 172.21.15.53:0/728759313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2d54074c40 msgr2=0x7f2d541a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:32.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.049+0000 7f2d58c00640 1 --2- 172.21.15.53:0/728759313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2d54074c40 0x7f2d541a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:32.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.049+0000 7f2d58c00640 1 -- 172.21.15.53:0/728759313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d5410e6a0 msgr2=0x7f2d5419a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:32.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.049+0000 7f2d58c00640 1 --2- 172.21.15.53:0/728759313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d5410e6a0 0x7f2d5419a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:32.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.049+0000 7f2d58c00640 1 -- 172.21.15.53:0/728759313 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d5419af90 con 0x7f2d54074320 2024-04-25T14:51:32.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.049+0000 7f2d58c00640 1 --2- 172.21.15.53:0/728759313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d54074320 0x7f2d541a0610 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2d4400ebc0 tx=0x7f2d4400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:32.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.049+0000 7f2d51600640 1 -- 172.21.15.53:0/728759313 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d44010430 con 0x7f2d54074320 2024-04-25T14:51:32.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.049+0000 7f2d51600640 1 -- 172.21.15.53:0/728759313 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d44010a80 con 0x7f2d54074320 2024-04-25T14:51:32.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.049+0000 7f2d5a000640 1 -- 172.21.15.53:0/728759313 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d5419b280 con 0x7f2d54074320 2024-04-25T14:51:32.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.049+0000 7f2d51600640 1 -- 172.21.15.53:0/728759313 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d44018a50 con 0x7f2d54074320 2024-04-25T14:51:32.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.050+0000 7f2d5a000640 1 -- 172.21.15.53:0/728759313 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d541a7590 con 0x7f2d54074320 2024-04-25T14:51:32.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.051+0000 7f2d5a000640 1 -- 172.21.15.53:0/728759313 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d5419b5c0 con 0x7f2d54074320 2024-04-25T14:51:32.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.053+0000 7f2d51600640 1 -- 172.21.15.53:0/728759313 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f2d44010c30 con 0x7f2d54074320 2024-04-25T14:51:32.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.054+0000 7f2d51600640 1 --2- 172.21.15.53:0/728759313 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f2d380a3d20 0x7f2d380a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:32.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.054+0000 7f2d51600640 1 -- 172.21.15.53:0/728759313 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8801+0+0 (secure 0 0 0) 0x7f2d440d0fe0 con 0x7f2d54074320 2024-04-25T14:51:32.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.055+0000 7f2d53e00640 1 --2- 172.21.15.53:0/728759313 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f2d380a3d20 0x7f2d380a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:32.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.055+0000 7f2d53e00640 1 --2- 172.21.15.53:0/728759313 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f2d380a3d20 0x7f2d380a61e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2d40031440 tx=0x7f2d40005b50 comp rx=0 tx=0).ready entity=mgr.15093 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:32.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.059+0000 7f2d51600640 1 -- 172.21.15.53:0/728759313 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2d4409d560 con 0x7f2d54074320 2024-04-25T14:51:32.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.266+0000 7f2d5a000640 1 -- 172.21.15.53:0/728759313 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2d5419bc00 con 0x7f2d54074320 2024-04-25T14:51:32.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.834+0000 7f2d51600640 1 -- 172.21.15.53:0/728759313 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v105) ==== 181+0+0 (secure 0 0 0) 0x7f2d4409ce60 con 0x7f2d54074320 2024-04-25T14:51:32.836 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' removed 2024-04-25T14:51:32.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.838+0000 7f2d5a000640 1 -- 172.21.15.53:0/728759313 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f2d380a3d20 msgr2=0x7f2d380a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:32.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.838+0000 7f2d5a000640 1 --2- 172.21.15.53:0/728759313 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f2d380a3d20 0x7f2d380a61e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2d40031440 tx=0x7f2d40005b50 comp rx=0 tx=0).stop 2024-04-25T14:51:32.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.838+0000 7f2d5a000640 1 -- 172.21.15.53:0/728759313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d54074320 msgr2=0x7f2d541a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:32.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.838+0000 7f2d5a000640 1 --2- 172.21.15.53:0/728759313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d54074320 0x7f2d541a0610 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2d4400ebc0 tx=0x7f2d4400c2d0 comp rx=0 tx=0).stop 2024-04-25T14:51:32.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.839+0000 7f2d5a000640 1 -- 172.21.15.53:0/728759313 shutdown_connections 2024-04-25T14:51:32.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.839+0000 7f2d5a000640 1 --2- 172.21.15.53:0/728759313 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7f2d380a3d20 0x7f2d380a61e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:32.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.839+0000 7f2d5a000640 1 --2- 172.21.15.53:0/728759313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d5410e6a0 0x7f2d5419a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:32.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.839+0000 7f2d5a000640 1 --2- 172.21.15.53:0/728759313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2d54074c40 0x7f2d541a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:32.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.839+0000 7f2d5a000640 1 --2- 172.21.15.53:0/728759313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d54074320 0x7f2d541a0610 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:32.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.839+0000 7f2d5a000640 1 -- 172.21.15.53:0/728759313 >> 172.21.15.53:0/728759313 conn(0x7f2d540fd8f0 msgr2=0x7f2d540ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:32.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.840+0000 7f2d5a000640 1 -- 172.21.15.53:0/728759313 shutdown_connections 2024-04-25T14:51:32.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:32.840+0000 7f2d5a000640 1 -- 172.21.15.53:0/728759313 wait complete. 2024-04-25T14:51:32.866 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T14:51:32.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:32 smithi150 ceph-mon[32504]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T14:51:32.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:32 smithi150 ceph-mon[32504]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T14:51:32.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:32 smithi150 ceph-mon[32504]: Cluster is now healthy 2024-04-25T14:51:32.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:32 smithi150 ceph-mon[32504]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T14:51:32.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:32 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T14:51:32.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:32 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T14:51:32.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:32 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T14:51:32.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:32 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T14:51:32.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:32 smithi150 ceph-mon[32504]: mds.? [v2:172.21.15.183:6835/2506706606,v1:172.21.15.183:6838/2506706606] up:boot 2024-04-25T14:51:32.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:32 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T14:51:32.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:32 smithi150 ceph-mon[32504]: mds.? [v2:172.21.15.183:6834/1772067544,v1:172.21.15.183:6837/1772067544] up:boot 2024-04-25T14:51:32.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:32 smithi150 ceph-mon[32504]: mds.? [v2:172.21.15.183:6836/1322022574,v1:172.21.15.183:6839/1322022574] up:boot 2024-04-25T14:51:32.963 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:32 smithi150 ceph-mon[32504]: mds.? [v2:172.21.15.150:6836/231022422,v1:172.21.15.150:6838/231022422] up:boot 2024-04-25T14:51:32.963 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:32 smithi150 ceph-mon[32504]: mds.? [v2:172.21.15.150:6837/4014742536,v1:172.21.15.150:6839/4014742536] up:boot 2024-04-25T14:51:32.963 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:32 smithi150 ceph-mon[32504]: mds.? [v2:172.21.15.53:6838/819672128,v1:172.21.15.53:6839/819672128] up:boot 2024-04-25T14:51:32.963 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:32 smithi150 ceph-mon[32504]: fsmap 12 up:standby 2024-04-25T14:51:32.963 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:32 smithi150 ceph-mon[32504]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T14:51:32.963 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:32 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/728759313' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T14:51:33.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:32 smithi053 ceph-mon[27773]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T14:51:33.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:32 smithi053 ceph-mon[27773]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T14:51:33.034 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:32 smithi053 ceph-mon[27773]: Cluster is now healthy 2024-04-25T14:51:33.034 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:32 smithi053 ceph-mon[27773]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T14:51:33.034 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:32 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T14:51:33.034 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:32 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T14:51:33.034 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:32 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T14:51:33.034 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:32 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T14:51:33.034 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:32 smithi053 ceph-mon[27773]: mds.? [v2:172.21.15.183:6835/2506706606,v1:172.21.15.183:6838/2506706606] up:boot 2024-04-25T14:51:33.034 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:32 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T14:51:33.035 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:32 smithi053 ceph-mon[27773]: mds.? [v2:172.21.15.183:6834/1772067544,v1:172.21.15.183:6837/1772067544] up:boot 2024-04-25T14:51:33.035 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:32 smithi053 ceph-mon[27773]: mds.? [v2:172.21.15.183:6836/1322022574,v1:172.21.15.183:6839/1322022574] up:boot 2024-04-25T14:51:33.035 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:32 smithi053 ceph-mon[27773]: mds.? [v2:172.21.15.150:6836/231022422,v1:172.21.15.150:6838/231022422] up:boot 2024-04-25T14:51:33.035 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:32 smithi053 ceph-mon[27773]: mds.? [v2:172.21.15.150:6837/4014742536,v1:172.21.15.150:6839/4014742536] up:boot 2024-04-25T14:51:33.035 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:32 smithi053 ceph-mon[27773]: mds.? [v2:172.21.15.53:6838/819672128,v1:172.21.15.53:6839/819672128] up:boot 2024-04-25T14:51:33.035 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:32 smithi053 ceph-mon[27773]: fsmap 12 up:standby 2024-04-25T14:51:33.035 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:32 smithi053 ceph-mon[27773]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T14:51:33.035 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:32 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/728759313' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T14:51:33.058 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:32 smithi183 ceph-mon[32584]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T14:51:33.058 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:32 smithi183 ceph-mon[32584]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T14:51:33.058 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:32 smithi183 ceph-mon[32584]: Cluster is now healthy 2024-04-25T14:51:33.058 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:32 smithi183 ceph-mon[32584]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T14:51:33.058 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:32 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T14:51:33.058 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:32 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T14:51:33.058 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:32 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T14:51:33.059 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:32 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T14:51:33.059 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:32 smithi183 ceph-mon[32584]: mds.? [v2:172.21.15.183:6835/2506706606,v1:172.21.15.183:6838/2506706606] up:boot 2024-04-25T14:51:33.059 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:32 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T14:51:33.059 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:32 smithi183 ceph-mon[32584]: mds.? [v2:172.21.15.183:6834/1772067544,v1:172.21.15.183:6837/1772067544] up:boot 2024-04-25T14:51:33.059 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:32 smithi183 ceph-mon[32584]: mds.? [v2:172.21.15.183:6836/1322022574,v1:172.21.15.183:6839/1322022574] up:boot 2024-04-25T14:51:33.059 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:32 smithi183 ceph-mon[32584]: mds.? [v2:172.21.15.150:6836/231022422,v1:172.21.15.150:6838/231022422] up:boot 2024-04-25T14:51:33.059 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:32 smithi183 ceph-mon[32584]: mds.? [v2:172.21.15.150:6837/4014742536,v1:172.21.15.150:6839/4014742536] up:boot 2024-04-25T14:51:33.059 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:32 smithi183 ceph-mon[32584]: mds.? [v2:172.21.15.53:6838/819672128,v1:172.21.15.53:6839/819672128] up:boot 2024-04-25T14:51:33.059 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:32 smithi183 ceph-mon[32584]: fsmap 12 up:standby 2024-04-25T14:51:33.060 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:32 smithi183 ceph-mon[32584]: from='mgr.15093 172.21.15.53:0/4181373446' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T14:51:33.060 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:32 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/728759313' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T14:51:33.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.071+0000 7fc0aae00640 1 -- 172.21.15.53:0/567413397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0a4104260 msgr2=0x7fc0a4104640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:33.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.071+0000 7fc0aae00640 1 --2- 172.21.15.53:0/567413397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0a4104260 0x7fc0a4104640 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc098009900 tx=0x7fc09802a3f0 comp rx=0 tx=0).stop 2024-04-25T14:51:33.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.071+0000 7fc0aae00640 1 -- 172.21.15.53:0/567413397 shutdown_connections 2024-04-25T14:51:33.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.071+0000 7fc0aae00640 1 --2- 172.21.15.53:0/567413397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0a4109330 0x7fc0a4110f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:33.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.071+0000 7fc0aae00640 1 --2- 172.21.15.53:0/567413397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0a4104c10 0x7fc0a4108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:33.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.071+0000 7fc0aae00640 1 --2- 172.21.15.53:0/567413397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0a4104260 0x7fc0a4104640 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:33.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.071+0000 7fc0aae00640 1 -- 172.21.15.53:0/567413397 >> 172.21.15.53:0/567413397 conn(0x7fc0a40ffa60 msgr2=0x7fc0a4101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:33.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.071+0000 7fc0aae00640 1 -- 172.21.15.53:0/567413397 shutdown_connections 2024-04-25T14:51:33.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.071+0000 7fc0aae00640 1 -- 172.21.15.53:0/567413397 wait complete. 2024-04-25T14:51:33.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.073+0000 7fc0aae00640 1 Processor -- start 2024-04-25T14:51:33.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.073+0000 7fc0aae00640 1 -- start start 2024-04-25T14:51:33.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.073+0000 7fc0aae00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0a4104260 0x7fc0a41a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:33.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.073+0000 7fc0aae00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0a4109330 0x7fc0a41a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:33.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.073+0000 7fc0aae00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0a419a5e0 0x7fc0a419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:33.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.073+0000 7fc0aae00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a41142c0 con 0x7fc0a4104260 2024-04-25T14:51:33.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.073+0000 7fc0aae00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a4114140 con 0x7fc0a419a5e0 2024-04-25T14:51:33.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.073+0000 7fc0aae00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a4114440 con 0x7fc0a4109330 2024-04-25T14:51:33.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.074+0000 7fc0a9000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0a4109330 0x7fc0a41a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:33.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.074+0000 7fc0aa400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0a419a5e0 0x7fc0a419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:33.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.074+0000 7fc0a9000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0a4109330 0x7fc0a41a0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.53:50036/0 (socket says 172.21.15.53:50036) 2024-04-25T14:51:33.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.074+0000 7fc0a9000640 1 -- 172.21.15.53:0/2366333884 learned_addr learned my addr 172.21.15.53:0/2366333884 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-25T14:51:33.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.074+0000 7fc0a9a00640 1 --2- 172.21.15.53:0/2366333884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0a4104260 0x7fc0a41a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:33.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.074+0000 7fc0aa400640 1 -- 172.21.15.53:0/2366333884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0a4109330 msgr2=0x7fc0a41a0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:33.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.074+0000 7fc0aa400640 1 --2- 172.21.15.53:0/2366333884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0a4109330 0x7fc0a41a0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:33.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.074+0000 7fc0aa400640 1 -- 172.21.15.53:0/2366333884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0a4104260 msgr2=0x7fc0a41a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:33.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.074+0000 7fc0aa400640 1 --2- 172.21.15.53:0/2366333884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0a4104260 0x7fc0a41a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:33.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.074+0000 7fc0aa400640 1 -- 172.21.15.53:0/2366333884 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0a419b100 con 0x7fc0a419a5e0 2024-04-25T14:51:33.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.075+0000 7fc0aa400640 1 --2- 172.21.15.53:0/2366333884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0a419a5e0 0x7fc0a419aa90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc09c009fd0 tx=0x7fc09c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:33.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.075+0000 7fc0a2000640 1 -- 172.21.15.53:0/2366333884 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09c00ce20 con 0x7fc0a419a5e0 2024-04-25T14:51:33.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.075+0000 7fc0aae00640 1 -- 172.21.15.53:0/2366333884 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0a419b3f0 con 0x7fc0a419a5e0 2024-04-25T14:51:33.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.075+0000 7fc0a2000640 1 -- 172.21.15.53:0/2366333884 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc09c005020 con 0x7fc0a419a5e0 2024-04-25T14:51:33.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.075+0000 7fc0a2000640 1 -- 172.21.15.53:0/2366333884 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09c010690 con 0x7fc0a419a5e0 2024-04-25T14:51:33.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.075+0000 7fc0aae00640 1 -- 172.21.15.53:0/2366333884 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0a41a75f0 con 0x7fc0a419a5e0 2024-04-25T14:51:33.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.077+0000 7fc0aae00640 1 -- 172.21.15.53:0/2366333884 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0a419bac0 con 0x7fc0a419a5e0 2024-04-25T14:51:33.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.079+0000 7fc0a2000640 1 -- 172.21.15.53:0/2366333884 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fc09c0054d0 con 0x7fc0a419a5e0 2024-04-25T14:51:33.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.080+0000 7fc0a2000640 1 --2- 172.21.15.53:0/2366333884 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fc08c0a3d20 0x7fc08c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:33.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.080+0000 7fc0a2000640 1 -- 172.21.15.53:0/2366333884 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8247+0+0 (secure 0 0 0) 0x7fc09c0d0dd0 con 0x7fc0a419a5e0 2024-04-25T14:51:33.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.080+0000 7fc0a9a00640 1 --2- 172.21.15.53:0/2366333884 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fc08c0a3d20 0x7fc08c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:33.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.081+0000 7fc0a9a00640 1 --2- 172.21.15.53:0/2366333884 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fc08c0a3d20 0x7fc08c0a61e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc098005ab0 tx=0x7fc098005a20 comp rx=0 tx=0).ready entity=mgr.15093 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:33.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.084+0000 7fc0a2000640 1 -- 172.21.15.53:0/2366333884 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc09c09d520 con 0x7fc0a419a5e0 2024-04-25T14:51:33.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.294+0000 7fc0aae00640 1 -- 172.21.15.53:0/2366333884 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc0a410c560 con 0x7fc0a419a5e0 2024-04-25T14:51:33.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.842+0000 7fc0a2000640 1 -- 172.21.15.53:0/2366333884 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v106) ==== 169+0+0 (secure 0 0 0) 0x7fc09c09ce20 con 0x7fc0a419a5e0 2024-04-25T14:51:33.847 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' removed 2024-04-25T14:51:33.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.850+0000 7fc0aae00640 1 -- 172.21.15.53:0/2366333884 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fc08c0a3d20 msgr2=0x7fc08c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:33.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.850+0000 7fc0aae00640 1 --2- 172.21.15.53:0/2366333884 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fc08c0a3d20 0x7fc08c0a61e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc098005ab0 tx=0x7fc098005a20 comp rx=0 tx=0).stop 2024-04-25T14:51:33.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.850+0000 7fc0aae00640 1 -- 172.21.15.53:0/2366333884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0a419a5e0 msgr2=0x7fc0a419aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:33.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.850+0000 7fc0aae00640 1 --2- 172.21.15.53:0/2366333884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0a419a5e0 0x7fc0a419aa90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc09c009fd0 tx=0x7fc09c00ef10 comp rx=0 tx=0).stop 2024-04-25T14:51:33.852 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.851+0000 7fc0aae00640 1 -- 172.21.15.53:0/2366333884 shutdown_connections 2024-04-25T14:51:33.852 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.851+0000 7fc0aae00640 1 --2- 172.21.15.53:0/2366333884 >> [v2:172.21.15.53:6800/802683714,v1:172.21.15.53:6801/802683714] conn(0x7fc08c0a3d20 0x7fc08c0a61e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:33.852 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.851+0000 7fc0aae00640 1 --2- 172.21.15.53:0/2366333884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0a419a5e0 0x7fc0a419aa90 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:33.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.851+0000 7fc0aae00640 1 --2- 172.21.15.53:0/2366333884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0a4109330 0x7fc0a41a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:33.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.851+0000 7fc0aae00640 1 --2- 172.21.15.53:0/2366333884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0a4104260 0x7fc0a41a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:33.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.851+0000 7fc0aae00640 1 -- 172.21.15.53:0/2366333884 >> 172.21.15.53:0/2366333884 conn(0x7fc0a40ffa60 msgr2=0x7fc0a4101dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:33.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.851+0000 7fc0aae00640 1 -- 172.21.15.53:0/2366333884 shutdown_connections 2024-04-25T14:51:33.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-25T14:51:33.851+0000 7fc0aae00640 1 -- 172.21.15.53:0/2366333884 wait complete. 2024-04-25T14:51:33.877 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T14:51:33.952 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T14:51:33.984 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T14:51:34.017 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-25T14:51:34.017 DEBUG:teuthology.orchestra.run.smithi053:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T14:51:34.079 DEBUG:teuthology.orchestra.run.smithi150:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T14:51:34.096 DEBUG:teuthology.orchestra.run.smithi183:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T14:51:34.114 INFO:tasks.cephadm:Stopping all daemons... 2024-04-25T14:51:34.114 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-25T14:51:34.114 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mon.a 2024-04-25T14:51:34.187 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:33 smithi053 ceph-mon[27773]: pgmap v340: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-25T14:51:34.187 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:33 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/728759313' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T14:51:34.187 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:33 smithi053 ceph-mon[27773]: osdmap e105: 12 total, 12 up, 12 in 2024-04-25T14:51:34.187 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:33 smithi053 ceph-mon[27773]: from='client.? 172.21.15.53:0/2366333884' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T14:51:34.187 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:33 smithi053 ceph-mon[27773]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T14:51:34.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:33 smithi183 ceph-mon[32584]: pgmap v340: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-25T14:51:34.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:33 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/728759313' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T14:51:34.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:33 smithi183 ceph-mon[32584]: osdmap e105: 12 total, 12 up, 12 in 2024-04-25T14:51:34.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:33 smithi183 ceph-mon[32584]: from='client.? 172.21.15.53:0/2366333884' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T14:51:34.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:33 smithi183 ceph-mon[32584]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T14:51:34.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:33 smithi150 ceph-mon[32504]: pgmap v340: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-25T14:51:34.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:33 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/728759313' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T14:51:34.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:33 smithi150 ceph-mon[32504]: osdmap e105: 12 total, 12 up, 12 in 2024-04-25T14:51:34.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:33 smithi150 ceph-mon[32504]: from='client.? 172.21.15.53:0/2366333884' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T14:51:34.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:33 smithi150 ceph-mon[32504]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T14:51:34.533 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:34 smithi053 systemd[1]: Stopping Ceph mon.a for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:51:35.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:34 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mon-a[27768]: 2024-04-25T14:51:34.739+0000 7f78c3e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:51:35.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:34 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mon-a[27768]: 2024-04-25T14:51:34.739+0000 7f78c3e00700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-25T14:51:35.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:34 smithi053 podman[85990]: 2024-04-25 14:51:34.970287019 +0000 UTC m=+0.380782144 container died fc4336fa1ec1552dd465a162c199d50f3682c6bbffff41df7bd13f1574ee5f8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-25T14:51:35.737 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:35 smithi053 podman[85990]: 2024-04-25 14:51:35.485218873 +0000 UTC m=+0.895713995 container cleanup fc4336fa1ec1552dd465a162c199d50f3682c6bbffff41df7bd13f1574ee5f8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T14:51:35.737 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:35 smithi053 bash[85990]: ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mon-a 2024-04-25T14:51:36.033 INFO:journalctl@ceph.mon.a.smithi053.stdout:Apr 25 14:51:35 smithi053 podman[86004]: 2024-04-25 14:51:35.736310721 +0000 UTC m=+0.749394495 container remove fc4336fa1ec1552dd465a162c199d50f3682c6bbffff41df7bd13f1574ee5f8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-25T14:51:36.299 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mon.a.service' 2024-04-25T14:51:36.438 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:51:36.438 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-25T14:51:36.438 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-25T14:51:36.439 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mon.b 2024-04-25T14:51:36.750 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:36 smithi150 systemd[1]: Stopping Ceph mon.b for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:51:37.084 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:36 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mon-b[32500]: 2024-04-25T14:51:36.748+0000 7f075a9ee700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:51:37.085 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:36 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mon-b[32500]: 2024-04-25T14:51:36.748+0000 7f075a9ee700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-25T14:51:37.085 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:36 smithi150 podman[68328]: 2024-04-25 14:51:36.770453954 +0000 UTC m=+0.134239286 container died 807bc911e52dd35645dcff82f412ead739e9c66f97bcbcd0d7b1850ae8baa005 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mon-b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T14:51:37.376 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:37 smithi150 podman[68328]: 2024-04-25 14:51:37.17555411 +0000 UTC m=+0.539339482 container cleanup 807bc911e52dd35645dcff82f412ead739e9c66f97bcbcd0d7b1850ae8baa005 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T14:51:37.376 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 14:51:37 smithi150 bash[68328]: ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mon-b 2024-04-25T14:51:37.516 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mon.b.service' 2024-04-25T14:51:37.562 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:51:37.563 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-25T14:51:37.563 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-25T14:51:37.563 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mon.c 2024-04-25T14:51:37.838 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:37 smithi183 systemd[1]: Stopping Ceph mon.c for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:51:37.839 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:37 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mon-c[32567]: 2024-04-25T14:51:37.836+0000 7fa7af242700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:51:38.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:37 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mon-c[32567]: 2024-04-25T14:51:37.836+0000 7fa7af242700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-25T14:51:38.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:37 smithi183 podman[68269]: 2024-04-25 14:51:37.980641621 +0000 UTC m=+0.221086208 container died d39383c27b474810cd5eaf27f0f64e3f221be15ed885b44e32452b018b8e4a0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mon-c, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-25T14:51:38.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:38 smithi183 podman[68269]: 2024-04-25 14:51:38.45372109 +0000 UTC m=+0.694165687 container cleanup d39383c27b474810cd5eaf27f0f64e3f221be15ed885b44e32452b018b8e4a0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mon-c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-25T14:51:38.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Apr 25 14:51:38 smithi183 bash[68269]: ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mon-c 2024-04-25T14:51:38.880 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mon.c.service' 2024-04-25T14:51:38.923 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:51:38.923 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-25T14:51:38.924 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-25T14:51:38.924 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mgr.x 2024-04-25T14:51:39.283 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:51:39 smithi053 systemd[1]: Stopping Ceph mgr.x for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:51:39.784 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:51:39 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:51:39.495+0000 7f48f5c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:51:39.784 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:51:39 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x[28005]: 2024-04-25T14:51:39.495+0000 7f48f5c00700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-25T14:51:39.784 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:51:39 smithi053 podman[86110]: 2024-04-25 14:51:39.59000446 +0000 UTC m=+0.218121925 container died a15886d6773de773af9639b0035bf7487d6de774c4071c5a178d9b6df94bf2da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-25T14:51:40.283 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:51:40 smithi053 podman[86110]: 2024-04-25 14:51:40.130316743 +0000 UTC m=+0.758434238 container cleanup a15886d6773de773af9639b0035bf7487d6de774c4071c5a178d9b6df94bf2da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-25T14:51:40.283 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:51:40 smithi053 bash[86110]: ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x 2024-04-25T14:51:40.783 INFO:journalctl@ceph.mgr.x.smithi053.stdout:Apr 25 14:51:40 smithi053 podman[86124]: 2024-04-25 14:51:40.466066654 +0000 UTC m=+0.860379053 container remove a15886d6773de773af9639b0035bf7487d6de774c4071c5a178d9b6df94bf2da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-x, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-25T14:51:41.069 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-25T14:51:41.183 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:51:41.184 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-25T14:51:41.184 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-25T14:51:41.184 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mgr.y 2024-04-25T14:51:41.517 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:51:41 smithi150 systemd[1]: Stopping Ceph mgr.y for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:51:41.822 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:51:41 smithi150 podman[68452]: 2024-04-25 14:51:41.516946626 +0000 UTC m=+0.137711094 container died 637d815f7da0dd9a02b553b1676c9e15a6046e303f368cbadecfa5628dc2bf63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T14:51:42.164 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:51:41 smithi150 podman[68452]: 2024-04-25 14:51:41.921549549 +0000 UTC m=+0.542314017 container cleanup 637d815f7da0dd9a02b553b1676c9e15a6046e303f368cbadecfa5628dc2bf63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-25T14:51:42.164 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 14:51:41 smithi150 bash[68452]: ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-y 2024-04-25T14:51:42.281 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-25T14:51:42.313 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:51:42.313 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-25T14:51:42.313 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-25T14:51:42.314 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mgr.z 2024-04-25T14:51:42.654 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:51:42 smithi183 systemd[1]: Stopping Ceph mgr.z for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:51:42.954 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:51:42 smithi183 podman[68391]: 2024-04-25 14:51:42.65435613 +0000 UTC m=+0.152170739 container died 71b0b1274392928cdc0a289efbf34084202f5a30ff113243781c471e4d48d73e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:51:43.387 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:51:42 smithi183 podman[68391]: 2024-04-25 14:51:42.99159934 +0000 UTC m=+0.489413959 container cleanup 71b0b1274392928cdc0a289efbf34084202f5a30ff113243781c471e4d48d73e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-25T14:51:43.387 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:51:42 smithi183 bash[68391]: ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z 2024-04-25T14:51:43.388 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:51:43 smithi183 podman[68405]: 2024-04-25 14:51:43.167774345 +0000 UTC m=+0.508524545 container remove 71b0b1274392928cdc0a289efbf34084202f5a30ff113243781c471e4d48d73e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mgr-z, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T14:51:43.388 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Apr 25 14:51:43 smithi183 systemd[1]: ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-04-25T14:51:43.402 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-25T14:51:43.435 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:51:43.436 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-25T14:51:43.436 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-25T14:51:43.436 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@osd.0 2024-04-25T14:51:43.783 INFO:journalctl@ceph.osd.0.smithi053.stdout:Apr 25 14:51:43 smithi053 systemd[1]: Stopping Ceph osd.0 for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:51:44.533 INFO:journalctl@ceph.osd.0.smithi053.stdout:Apr 25 14:51:44 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-0[42448]: 2024-04-25T14:51:44.088+0000 7fa1a8800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:51:44.533 INFO:journalctl@ceph.osd.0.smithi053.stdout:Apr 25 14:51:44 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-0[42448]: 2024-04-25T14:51:44.088+0000 7fa1a8800700 -1 osd.0 106 *** Got signal Terminated *** 2024-04-25T14:51:44.533 INFO:journalctl@ceph.osd.0.smithi053.stdout:Apr 25 14:51:44 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-0[42448]: 2024-04-25T14:51:44.088+0000 7fa1a8800700 -1 osd.0 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:51:49.908 INFO:journalctl@ceph.osd.0.smithi053.stdout:Apr 25 14:51:49 smithi053 podman[86231]: 2024-04-25 14:51:49.601127649 +0000 UTC m=+5.708632965 container died 46b6b22d17bd2e487e13823bafa3edeaaac90e73362199e4a9ce15fab34052ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:51:50.283 INFO:journalctl@ceph.osd.0.smithi053.stdout:Apr 25 14:51:50 smithi053 podman[86231]: 2024-04-25 14:51:50.035436201 +0000 UTC m=+6.142941513 container cleanup 46b6b22d17bd2e487e13823bafa3edeaaac90e73362199e4a9ce15fab34052ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-25T14:51:50.283 INFO:journalctl@ceph.osd.0.smithi053.stdout:Apr 25 14:51:50 smithi053 bash[86231]: ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-0 2024-04-25T14:51:50.783 INFO:journalctl@ceph.osd.0.smithi053.stdout:Apr 25 14:51:50 smithi053 podman[86252]: 2024-04-25 14:51:50.344894391 +0000 UTC m=+0.725511481 container remove 46b6b22d17bd2e487e13823bafa3edeaaac90e73362199e4a9ce15fab34052ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:51:51.392 INFO:journalctl@ceph.osd.0.smithi053.stdout:Apr 25 14:51:51 smithi053 podman[86345]: 2024-04-25 14:51:51.04284342 +0000 UTC m=+0.041805452 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T14:51:51.393 INFO:journalctl@ceph.osd.0.smithi053.stdout:Apr 25 14:51:51 smithi053 podman[86345]: 2024-04-25 14:51:51.17083181 +0000 UTC m=+0.169793856 container create b2755818d8a8376c0200dab5c0ea52041960c13a625ce677cdf8bf4ab6bb505d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2) 2024-04-25T14:51:51.783 INFO:journalctl@ceph.osd.0.smithi053.stdout:Apr 25 14:51:51 smithi053 podman[86345]: 2024-04-25 14:51:51.49961178 +0000 UTC m=+0.498573832 container init b2755818d8a8376c0200dab5c0ea52041960c13a625ce677cdf8bf4ab6bb505d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-0-deactivate, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-25T14:51:51.783 INFO:journalctl@ceph.osd.0.smithi053.stdout:Apr 25 14:51:51 smithi053 podman[86345]: 2024-04-25 14:51:51.511752142 +0000 UTC m=+0.510714195 container start b2755818d8a8376c0200dab5c0ea52041960c13a625ce677cdf8bf4ab6bb505d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-25T14:51:51.783 INFO:journalctl@ceph.osd.0.smithi053.stdout:Apr 25 14:51:51 smithi053 podman[86345]: 2024-04-25 14:51:51.590132496 +0000 UTC m=+0.589094598 container attach b2755818d8a8376c0200dab5c0ea52041960c13a625ce677cdf8bf4ab6bb505d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-25T14:51:52.283 INFO:journalctl@ceph.osd.0.smithi053.stdout:Apr 25 14:51:51 smithi053 podman[86345]: 2024-04-25 14:51:51.989799078 +0000 UTC m=+0.988761178 container died b2755818d8a8376c0200dab5c0ea52041960c13a625ce677cdf8bf4ab6bb505d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T14:51:52.942 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@osd.0.service' 2024-04-25T14:51:53.055 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:51:53.055 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-25T14:51:53.055 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-25T14:51:53.056 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@osd.1 2024-04-25T14:51:53.361 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 14:51:53 smithi150 systemd[1]: Stopping Ceph osd.1 for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:51:53.634 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 14:51:53 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-1[37094]: 2024-04-25T14:51:53.359+0000 7f316ea2e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:51:53.634 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 14:51:53 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-1[37094]: 2024-04-25T14:51:53.359+0000 7f316ea2e700 -1 osd.1 106 *** Got signal Terminated *** 2024-04-25T14:51:53.635 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 14:51:53 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-1[37094]: 2024-04-25T14:51:53.359+0000 7f316ea2e700 -1 osd.1 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:51:59.103 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 14:51:58 smithi150 podman[68577]: 2024-04-25 14:51:58.822289419 +0000 UTC m=+5.570341844 container died c82f8eed2e4fa25e1dba399e08223fad4cae1f34331d990831305350c23fe629 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-1, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-25T14:51:59.461 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 14:51:59 smithi150 podman[68577]: 2024-04-25 14:51:59.19362387 +0000 UTC m=+5.941676287 container cleanup c82f8eed2e4fa25e1dba399e08223fad4cae1f34331d990831305350c23fe629 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416) 2024-04-25T14:51:59.461 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 14:51:59 smithi150 bash[68577]: ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-1 2024-04-25T14:51:59.812 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 14:51:59 smithi150 podman[68607]: 2024-04-25 14:51:59.470614163 +0000 UTC m=+0.638841173 container remove c82f8eed2e4fa25e1dba399e08223fad4cae1f34331d990831305350c23fe629 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-1, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-25T14:52:00.116 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 14:51:59 smithi150 podman[68697]: 2024-04-25 14:51:59.715341484 +0000 UTC m=+0.017614747 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T14:52:00.117 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 14:51:59 smithi150 podman[68697]: 2024-04-25 14:51:59.829806754 +0000 UTC m=+0.132080001 container create 4d3b7764fe8f1c330082bc75bc7a6d5d537a10a0895f6f377d022bb918e1590a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T14:52:00.422 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 14:52:00 smithi150 podman[68697]: 2024-04-25 14:52:00.116673958 +0000 UTC m=+0.418947205 container init 4d3b7764fe8f1c330082bc75bc7a6d5d537a10a0895f6f377d022bb918e1590a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True) 2024-04-25T14:52:00.422 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 14:52:00 smithi150 podman[68697]: 2024-04-25 14:52:00.120006321 +0000 UTC m=+0.422279569 container start 4d3b7764fe8f1c330082bc75bc7a6d5d537a10a0895f6f377d022bb918e1590a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-1-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-25T14:52:00.422 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 14:52:00 smithi150 podman[68697]: 2024-04-25 14:52:00.193890206 +0000 UTC m=+0.496163463 container attach 4d3b7764fe8f1c330082bc75bc7a6d5d537a10a0895f6f377d022bb918e1590a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD) 2024-04-25T14:52:00.711 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 14:52:00 smithi150 podman[68697]: 2024-04-25 14:52:00.422483607 +0000 UTC m=+0.724756857 container died 4d3b7764fe8f1c330082bc75bc7a6d5d537a10a0895f6f377d022bb918e1590a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-25T14:52:01.017 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@osd.1.service' 2024-04-25T14:52:01.058 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:52:01.059 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-25T14:52:01.059 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-25T14:52:01.059 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@osd.2 2024-04-25T14:52:01.369 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 25 14:52:01 smithi183 systemd[1]: Stopping Ceph osd.2 for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:52:01.633 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 25 14:52:01 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-2[37718]: 2024-04-25T14:52:01.368+0000 7f4b67705700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:52:01.633 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 25 14:52:01 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-2[37718]: 2024-04-25T14:52:01.368+0000 7f4b67705700 -1 osd.2 106 *** Got signal Terminated *** 2024-04-25T14:52:01.633 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 25 14:52:01 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-2[37718]: 2024-04-25T14:52:01.368+0000 7f4b67705700 -1 osd.2 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:52:07.020 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 25 14:52:06 smithi183 podman[68513]: 2024-04-25 14:52:06.743285144 +0000 UTC m=+5.495513915 container died 4ad7b8c31b096fd53c71ab1e2ad572195001596071bc1ef42de153588704c74a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:52:07.386 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 25 14:52:07 smithi183 podman[68513]: 2024-04-25 14:52:07.118981472 +0000 UTC m=+5.871210252 container cleanup 4ad7b8c31b096fd53c71ab1e2ad572195001596071bc1ef42de153588704c74a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, ceph=True) 2024-04-25T14:52:07.387 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 25 14:52:07 smithi183 bash[68513]: ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-2 2024-04-25T14:52:07.704 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 25 14:52:07 smithi183 podman[68542]: 2024-04-25 14:52:07.386849139 +0000 UTC m=+0.644452897 container remove 4ad7b8c31b096fd53c71ab1e2ad572195001596071bc1ef42de153588704c74a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T14:52:08.056 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 25 14:52:07 smithi183 podman[68634]: 2024-04-25 14:52:07.737935089 +0000 UTC m=+0.112206000 container create 45dc7c6fadba13cf9c6a3eaaf13bc3401afdf12e7a4cb2ef1de49660033ead7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-25T14:52:08.057 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 25 14:52:07 smithi183 podman[68634]: 2024-04-25 14:52:07.643665616 +0000 UTC m=+0.017936500 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T14:52:08.057 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 25 14:52:07 smithi183 podman[68634]: 2024-04-25 14:52:07.957056009 +0000 UTC m=+0.331326886 container init 45dc7c6fadba13cf9c6a3eaaf13bc3401afdf12e7a4cb2ef1de49660033ead7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-2-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:52:08.057 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 25 14:52:07 smithi183 podman[68634]: 2024-04-25 14:52:07.960561234 +0000 UTC m=+0.334832100 container start 45dc7c6fadba13cf9c6a3eaaf13bc3401afdf12e7a4cb2ef1de49660033ead7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-25T14:52:08.057 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 25 14:52:08 smithi183 podman[68634]: 2024-04-25 14:52:08.056567052 +0000 UTC m=+0.430837923 container attach 45dc7c6fadba13cf9c6a3eaaf13bc3401afdf12e7a4cb2ef1de49660033ead7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-2-deactivate, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-25T14:52:08.454 INFO:journalctl@ceph.osd.2.smithi183.stdout:Apr 25 14:52:08 smithi183 podman[68634]: 2024-04-25 14:52:08.250907728 +0000 UTC m=+0.625178586 container died 45dc7c6fadba13cf9c6a3eaaf13bc3401afdf12e7a4cb2ef1de49660033ead7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T14:52:08.900 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@osd.2.service' 2024-04-25T14:52:08.937 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:52:08.937 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-25T14:52:08.937 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-25T14:52:08.938 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@osd.3 2024-04-25T14:52:09.283 INFO:journalctl@ceph.osd.3.smithi053.stdout:Apr 25 14:52:09 smithi053 systemd[1]: Stopping Ceph osd.3 for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:52:09.783 INFO:journalctl@ceph.osd.3.smithi053.stdout:Apr 25 14:52:09 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-3[48936]: 2024-04-25T14:52:09.509+0000 7f06ddc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:52:09.783 INFO:journalctl@ceph.osd.3.smithi053.stdout:Apr 25 14:52:09 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-3[48936]: 2024-04-25T14:52:09.509+0000 7f06ddc00700 -1 osd.3 106 *** Got signal Terminated *** 2024-04-25T14:52:09.784 INFO:journalctl@ceph.osd.3.smithi053.stdout:Apr 25 14:52:09 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-3[48936]: 2024-04-25T14:52:09.509+0000 7f06ddc00700 -1 osd.3 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:52:11.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:11 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:11.042+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:12.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:12 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:12.050+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:12.454 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:11 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5[44266]: 2024-04-25T14:52:11.969+0000 7f2f23eda700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:45.896685+0000 front 2024-04-25T14:51:45.896774+0000 (oldest deadline 2024-04-25T14:52:11.196601+0000) 2024-04-25T14:52:13.204 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:12 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5[44266]: 2024-04-25T14:52:12.926+0000 7f2f23eda700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:45.896685+0000 front 2024-04-25T14:51:45.896774+0000 (oldest deadline 2024-04-25T14:52:11.196601+0000) 2024-04-25T14:52:13.205 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:13 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:13.084+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:13.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:13 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:13.020+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:14.454 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:13 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5[44266]: 2024-04-25T14:52:13.968+0000 7f2f23eda700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:45.896685+0000 front 2024-04-25T14:51:45.896774+0000 (oldest deadline 2024-04-25T14:52:11.196601+0000) 2024-04-25T14:52:14.455 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:14 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:14.122+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:14.533 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:14 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:14.046+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:14.961 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:14 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:14.497+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:15.204 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:14 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5[44266]: 2024-04-25T14:52:14.935+0000 7f2f23eda700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:45.896685+0000 front 2024-04-25T14:51:45.896774+0000 (oldest deadline 2024-04-25T14:52:11.196601+0000) 2024-04-25T14:52:15.205 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:15 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:15.097+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:15.229 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:15 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:15.049+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:15.230 INFO:journalctl@ceph.osd.3.smithi053.stdout:Apr 25 14:52:14 smithi053 podman[86448]: 2024-04-25 14:52:14.893960746 +0000 UTC m=+5.502571906 container died 5867d9706c0699d52cc70d1ff26f7356c3d151c4a5d74a681ce9936b43ef7e02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-3, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T14:52:15.516 INFO:journalctl@ceph.osd.3.smithi053.stdout:Apr 25 14:52:15 smithi053 podman[86448]: 2024-04-25 14:52:15.337168714 +0000 UTC m=+5.945779889 container cleanup 5867d9706c0699d52cc70d1ff26f7356c3d151c4a5d74a681ce9936b43ef7e02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:52:15.516 INFO:journalctl@ceph.osd.3.smithi053.stdout:Apr 25 14:52:15 smithi053 bash[86448]: ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-3 2024-04-25T14:52:15.783 INFO:journalctl@ceph.osd.3.smithi053.stdout:Apr 25 14:52:15 smithi053 podman[86479]: 2024-04-25 14:52:15.738332258 +0000 UTC m=+0.833271869 container remove 5867d9706c0699d52cc70d1ff26f7356c3d151c4a5d74a681ce9936b43ef7e02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-25T14:52:15.784 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:15 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:15.515+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:48.965340+0000 front 2024-04-25T14:51:48.964926+0000 (oldest deadline 2024-04-25T14:52:14.864930+0000) 2024-04-25T14:52:15.961 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:15 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:15.522+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:16.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:16 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:16.010+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:16.454 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:15 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5[44266]: 2024-04-25T14:52:15.983+0000 7f2f23eda700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:45.896685+0000 front 2024-04-25T14:51:45.896774+0000 (oldest deadline 2024-04-25T14:52:11.196601+0000) 2024-04-25T14:52:16.455 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:16 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:16.140+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:16.820 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:16 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:16.556+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:48.965340+0000 front 2024-04-25T14:51:48.964926+0000 (oldest deadline 2024-04-25T14:52:14.864930+0000) 2024-04-25T14:52:16.820 INFO:journalctl@ceph.osd.3.smithi053.stdout:Apr 25 14:52:16 smithi053 podman[86565]: 2024-04-25 14:52:16.550955643 +0000 UTC m=+0.037646708 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T14:52:16.821 INFO:journalctl@ceph.osd.3.smithi053.stdout:Apr 25 14:52:16 smithi053 podman[86565]: 2024-04-25 14:52:16.647470296 +0000 UTC m=+0.134161386 container create b2e08064c457438372e12be1b5847704186b21aa5acac2b3e2a2e8329124af06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T14:52:16.960 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:16 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:16.514+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:17.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:16 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:16.996+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:17.283 INFO:journalctl@ceph.osd.3.smithi053.stdout:Apr 25 14:52:16 smithi053 podman[86565]: 2024-04-25 14:52:16.917978245 +0000 UTC m=+0.404669378 container init b2e08064c457438372e12be1b5847704186b21aa5acac2b3e2a2e8329124af06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T14:52:17.283 INFO:journalctl@ceph.osd.3.smithi053.stdout:Apr 25 14:52:16 smithi053 podman[86565]: 2024-04-25 14:52:16.929762915 +0000 UTC m=+0.416454028 container start b2e08064c457438372e12be1b5847704186b21aa5acac2b3e2a2e8329124af06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T14:52:17.284 INFO:journalctl@ceph.osd.3.smithi053.stdout:Apr 25 14:52:17 smithi053 podman[86565]: 2024-04-25 14:52:17.024877685 +0000 UTC m=+0.511568778 container attach b2e08064c457438372e12be1b5847704186b21aa5acac2b3e2a2e8329124af06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD) 2024-04-25T14:52:17.454 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:16 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5[44266]: 2024-04-25T14:52:16.960+0000 7f2f23eda700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:45.896685+0000 front 2024-04-25T14:51:45.896774+0000 (oldest deadline 2024-04-25T14:52:11.196601+0000) 2024-04-25T14:52:17.457 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:17 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:17.107+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:17.775 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:17 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:17.569+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:48.965340+0000 front 2024-04-25T14:51:48.964926+0000 (oldest deadline 2024-04-25T14:52:14.864930+0000) 2024-04-25T14:52:17.775 INFO:journalctl@ceph.osd.3.smithi053.stdout:Apr 25 14:52:17 smithi053 podman[86591]: 2024-04-25 14:52:17.460149841 +0000 UTC m=+0.041376909 container died b2e08064c457438372e12be1b5847704186b21aa5acac2b3e2a2e8329124af06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-25T14:52:17.961 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:17 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:17.506+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:18.033 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:18 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:18.020+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:18.252 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@osd.3.service' 2024-04-25T14:52:18.363 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:52:18.363 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-25T14:52:18.364 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-25T14:52:18.364 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@osd.4 2024-04-25T14:52:18.454 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:18 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:18.142+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:18.455 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:17 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5[44266]: 2024-04-25T14:52:17.976+0000 7f2f23eda700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:45.896685+0000 front 2024-04-25T14:51:45.896774+0000 (oldest deadline 2024-04-25T14:52:11.196601+0000) 2024-04-25T14:52:18.658 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 25 14:52:18 smithi150 systemd[1]: Stopping Ceph osd.4 for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:52:18.659 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:18 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:18.511+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:18.961 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 25 14:52:18 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-4[43891]: 2024-04-25T14:52:18.658+0000 7f6b928e5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:52:18.961 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 25 14:52:18 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-4[43891]: 2024-04-25T14:52:18.658+0000 7f6b928e5700 -1 osd.4 106 *** Got signal Terminated *** 2024-04-25T14:52:18.961 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 25 14:52:18 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-4[43891]: 2024-04-25T14:52:18.658+0000 7f6b928e5700 -1 osd.4 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:52:19.020 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:18 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:18.538+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:48.965340+0000 front 2024-04-25T14:51:48.964926+0000 (oldest deadline 2024-04-25T14:52:14.864930+0000) 2024-04-25T14:52:19.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:19 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:19.019+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:19.454 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:19 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:19.135+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:19.454 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:18 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5[44266]: 2024-04-25T14:52:18.973+0000 7f2f23eda700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:45.896685+0000 front 2024-04-25T14:51:45.896774+0000 (oldest deadline 2024-04-25T14:52:11.196601+0000) 2024-04-25T14:52:19.783 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:19 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:19.509+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:48.965340+0000 front 2024-04-25T14:51:48.964926+0000 (oldest deadline 2024-04-25T14:52:14.864930+0000) 2024-04-25T14:52:19.961 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:19 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:19.580+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:54.119723+0000 front 2024-04-25T14:51:54.119777+0000 (oldest deadline 2024-04-25T14:52:18.819789+0000) 2024-04-25T14:52:19.961 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:19 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:19.473+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:19.962 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:19 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:19.473+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:52:20.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:20 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:20.005+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:20.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:20 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:20.005+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:52:20.454 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:20 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:20.160+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:20.455 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:19 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5[44266]: 2024-04-25T14:52:19.985+0000 7f2f23eda700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:45.896685+0000 front 2024-04-25T14:51:45.896774+0000 (oldest deadline 2024-04-25T14:52:11.196601+0000) 2024-04-25T14:52:20.455 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:19 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5[44266]: 2024-04-25T14:52:19.985+0000 7f2f23eda700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:58.198202+0000 front 2024-04-25T14:51:58.198308+0000 (oldest deadline 2024-04-25T14:52:19.298334+0000) 2024-04-25T14:52:20.783 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:20 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:20.489+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:48.965340+0000 front 2024-04-25T14:51:48.964926+0000 (oldest deadline 2024-04-25T14:52:14.864930+0000) 2024-04-25T14:52:20.961 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:20 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:20.574+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:54.119723+0000 front 2024-04-25T14:51:54.119777+0000 (oldest deadline 2024-04-25T14:52:18.819789+0000) 2024-04-25T14:52:20.961 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:20 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:20.497+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:20.962 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:20 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:20.497+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:52:21.204 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:21 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:21.125+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:21.204 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:20 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5[44266]: 2024-04-25T14:52:20.936+0000 7f2f23eda700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:45.896685+0000 front 2024-04-25T14:51:45.896774+0000 (oldest deadline 2024-04-25T14:52:11.196601+0000) 2024-04-25T14:52:21.204 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:20 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5[44266]: 2024-04-25T14:52:20.936+0000 7f2f23eda700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:58.198202+0000 front 2024-04-25T14:51:58.198308+0000 (oldest deadline 2024-04-25T14:52:19.298334+0000) 2024-04-25T14:52:21.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:21 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:20.999+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:21.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:21 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:20.999+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:52:21.461 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 25 14:52:21 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-4[43891]: 2024-04-25T14:52:21.048+0000 7f6b8aed1700 -1 osd.4 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.951700+0000 front 2024-04-25T14:51:57.951680+0000 (oldest deadline 2024-04-25T14:52:20.851844+0000) 2024-04-25T14:52:21.784 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:21 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:21.512+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:48.965340+0000 front 2024-04-25T14:51:48.964926+0000 (oldest deadline 2024-04-25T14:52:14.864930+0000) 2024-04-25T14:52:21.784 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:21 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:21.512+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.566861+0000 front 2024-04-25T14:51:56.567029+0000 (oldest deadline 2024-04-25T14:52:20.666494+0000) 2024-04-25T14:52:21.961 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:21 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:21.529+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:54.119723+0000 front 2024-04-25T14:51:54.119777+0000 (oldest deadline 2024-04-25T14:52:18.819789+0000) 2024-04-25T14:52:21.961 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:21 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:21.540+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:21.961 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:21 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:21.540+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:52:22.204 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:21 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5[44266]: 2024-04-25T14:52:21.927+0000 7f2f23eda700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:45.896685+0000 front 2024-04-25T14:51:45.896774+0000 (oldest deadline 2024-04-25T14:52:11.196601+0000) 2024-04-25T14:52:22.204 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:21 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5[44266]: 2024-04-25T14:52:21.927+0000 7f2f23eda700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:58.198202+0000 front 2024-04-25T14:51:58.198308+0000 (oldest deadline 2024-04-25T14:52:19.298334+0000) 2024-04-25T14:52:22.205 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:22 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:22.127+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:22.205 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:22 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:22.127+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:52:22.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:22 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:22.029+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:22.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:22 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:22.029+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:52:22.461 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 25 14:52:22 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-4[43891]: 2024-04-25T14:52:22.013+0000 7f6b8aed1700 -1 osd.4 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.951700+0000 front 2024-04-25T14:51:57.951680+0000 (oldest deadline 2024-04-25T14:52:20.851844+0000) 2024-04-25T14:52:22.783 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:22 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:22.529+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:48.965340+0000 front 2024-04-25T14:51:48.964926+0000 (oldest deadline 2024-04-25T14:52:14.864930+0000) 2024-04-25T14:52:22.784 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:22 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:22.529+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.566861+0000 front 2024-04-25T14:51:56.567029+0000 (oldest deadline 2024-04-25T14:52:20.666494+0000) 2024-04-25T14:52:22.954 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:22 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5[44266]: 2024-04-25T14:52:22.916+0000 7f2f23eda700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:45.896685+0000 front 2024-04-25T14:51:45.896774+0000 (oldest deadline 2024-04-25T14:52:11.196601+0000) 2024-04-25T14:52:22.954 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:22 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5[44266]: 2024-04-25T14:52:22.916+0000 7f2f23eda700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:58.198202+0000 front 2024-04-25T14:51:58.198308+0000 (oldest deadline 2024-04-25T14:52:19.298334+0000) 2024-04-25T14:52:22.955 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:22 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:22.698+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.275170+0000 front 2024-04-25T14:51:56.275046+0000 (oldest deadline 2024-04-25T14:52:22.174939+0000) 2024-04-25T14:52:22.960 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:22 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:22.499+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:54.119723+0000 front 2024-04-25T14:51:54.119777+0000 (oldest deadline 2024-04-25T14:52:18.819789+0000) 2024-04-25T14:52:22.961 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:22 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:22.503+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:22.961 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:22 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:22.503+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:52:23.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:22 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:22.990+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:23.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:22 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:22.990+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:52:23.454 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:23 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:23.170+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:23.455 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:23 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:23.170+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:52:23.461 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 25 14:52:22 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-4[43891]: 2024-04-25T14:52:22.985+0000 7f6b8aed1700 -1 osd.4 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.951700+0000 front 2024-04-25T14:51:57.951680+0000 (oldest deadline 2024-04-25T14:52:20.851844+0000) 2024-04-25T14:52:23.926 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:23 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:23.650+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.275170+0000 front 2024-04-25T14:51:56.275046+0000 (oldest deadline 2024-04-25T14:52:22.174939+0000) 2024-04-25T14:52:23.961 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:23 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:23.538+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:54.119723+0000 front 2024-04-25T14:51:54.119777+0000 (oldest deadline 2024-04-25T14:52:18.819789+0000) 2024-04-25T14:52:23.961 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:23 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:23.478+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:23.962 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:23 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:23.478+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:52:24.013 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:23 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:23.540+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:48.965340+0000 front 2024-04-25T14:51:48.964926+0000 (oldest deadline 2024-04-25T14:52:14.864930+0000) 2024-04-25T14:52:24.013 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:23 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:23.540+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.566861+0000 front 2024-04-25T14:51:56.567029+0000 (oldest deadline 2024-04-25T14:52:20.666494+0000) 2024-04-25T14:52:24.205 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:23 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5[44266]: 2024-04-25T14:52:23.926+0000 7f2f23eda700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:45.896685+0000 front 2024-04-25T14:51:45.896774+0000 (oldest deadline 2024-04-25T14:52:11.196601+0000) 2024-04-25T14:52:24.206 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:23 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5[44266]: 2024-04-25T14:52:23.926+0000 7f2f23eda700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:58.198202+0000 front 2024-04-25T14:51:58.198308+0000 (oldest deadline 2024-04-25T14:52:19.298334+0000) 2024-04-25T14:52:24.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:24 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:24.012+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:24.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:24 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:24.012+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:52:24.461 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 25 14:52:24 smithi150 podman[68802]: 2024-04-25 14:52:24.200175829 +0000 UTC m=+5.646681824 container died f89a6f1a1d24a20975ec475e3d65bee61df0657adda35c4558d1f6414d709074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-4, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-25T14:52:24.461 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:24 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:24.432+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:24.461 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:24 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:24.432+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:52:24.681 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:24 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:24.206+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:24.681 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:24 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:24.206+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:52:24.834 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:24 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:24.531+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:54.119723+0000 front 2024-04-25T14:51:54.119777+0000 (oldest deadline 2024-04-25T14:52:18.819789+0000) 2024-04-25T14:52:24.834 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 25 14:52:24 smithi150 podman[68802]: 2024-04-25 14:52:24.55832225 +0000 UTC m=+6.004828251 container cleanup f89a6f1a1d24a20975ec475e3d65bee61df0657adda35c4558d1f6414d709074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-4, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-25T14:52:24.835 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 25 14:52:24 smithi150 bash[68802]: ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-4 2024-04-25T14:52:24.954 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:24 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5[44266]: 2024-04-25T14:52:24.923+0000 7f2f23eda700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:45.896685+0000 front 2024-04-25T14:51:45.896774+0000 (oldest deadline 2024-04-25T14:52:11.196601+0000) 2024-04-25T14:52:24.954 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:24 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5[44266]: 2024-04-25T14:52:24.923+0000 7f2f23eda700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:58.198202+0000 front 2024-04-25T14:51:58.198308+0000 (oldest deadline 2024-04-25T14:52:19.298334+0000) 2024-04-25T14:52:24.955 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:24 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:24.680+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.275170+0000 front 2024-04-25T14:51:56.275046+0000 (oldest deadline 2024-04-25T14:52:22.174939+0000) 2024-04-25T14:52:24.976 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:24 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:24.556+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:48.965340+0000 front 2024-04-25T14:51:48.964926+0000 (oldest deadline 2024-04-25T14:52:14.864930+0000) 2024-04-25T14:52:24.976 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:24 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:24.556+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.566861+0000 front 2024-04-25T14:51:56.567029+0000 (oldest deadline 2024-04-25T14:52:20.666494+0000) 2024-04-25T14:52:25.160 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 25 14:52:24 smithi150 podman[68819]: 2024-04-25 14:52:24.834538064 +0000 UTC m=+0.637490379 container remove f89a6f1a1d24a20975ec475e3d65bee61df0657adda35c4558d1f6414d709074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T14:52:25.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:24 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:24.975+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:25.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:24 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:24.975+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:52:25.454 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:25 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:25.186+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:25.454 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:25 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:25.186+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:52:25.461 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:25 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:25.388+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:25.461 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:25 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:25.388+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:52:25.461 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 25 14:52:25 smithi150 podman[68912]: 2024-04-25 14:52:25.160470937 +0000 UTC m=+0.103740621 container create cdc7af7c4902fc7ce40fa4feffdb76d6af284b4a70bf0ab7f4a38528f1b19780 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True) 2024-04-25T14:52:25.461 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 25 14:52:25 smithi150 podman[68912]: 2024-04-25 14:52:25.074500627 +0000 UTC m=+0.017770278 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T14:52:25.462 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 25 14:52:25 smithi150 podman[68912]: 2024-04-25 14:52:25.396324502 +0000 UTC m=+0.339594151 container init cdc7af7c4902fc7ce40fa4feffdb76d6af284b4a70bf0ab7f4a38528f1b19780 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2) 2024-04-25T14:52:25.462 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 25 14:52:25 smithi150 podman[68912]: 2024-04-25 14:52:25.400094053 +0000 UTC m=+0.343363692 container start cdc7af7c4902fc7ce40fa4feffdb76d6af284b4a70bf0ab7f4a38528f1b19780 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416) 2024-04-25T14:52:25.960 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:25 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:25.571+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:48.965340+0000 front 2024-04-25T14:51:48.964926+0000 (oldest deadline 2024-04-25T14:52:14.864930+0000) 2024-04-25T14:52:25.961 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:25 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:25.571+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.566861+0000 front 2024-04-25T14:51:56.567029+0000 (oldest deadline 2024-04-25T14:52:20.666494+0000) 2024-04-25T14:52:25.961 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:25 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:25.527+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:54.119723+0000 front 2024-04-25T14:51:54.119777+0000 (oldest deadline 2024-04-25T14:52:18.819789+0000) 2024-04-25T14:52:25.961 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 25 14:52:25 smithi150 podman[68912]: 2024-04-25 14:52:25.475045332 +0000 UTC m=+0.418314981 container attach cdc7af7c4902fc7ce40fa4feffdb76d6af284b4a70bf0ab7f4a38528f1b19780 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-25T14:52:25.962 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 25 14:52:25 smithi150 podman[68912]: 2024-04-25 14:52:25.692925776 +0000 UTC m=+0.636195416 container died cdc7af7c4902fc7ce40fa4feffdb76d6af284b4a70bf0ab7f4a38528f1b19780 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T14:52:26.153 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:25 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:25.724+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.275170+0000 front 2024-04-25T14:51:56.275046+0000 (oldest deadline 2024-04-25T14:52:22.174939+0000) 2024-04-25T14:52:26.153 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:25 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5[44266]: 2024-04-25T14:52:25.910+0000 7f2f23eda700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:45.896685+0000 front 2024-04-25T14:51:45.896774+0000 (oldest deadline 2024-04-25T14:52:11.196601+0000) 2024-04-25T14:52:26.154 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:25 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5[44266]: 2024-04-25T14:52:25.910+0000 7f2f23eda700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:58.198202+0000 front 2024-04-25T14:51:58.198308+0000 (oldest deadline 2024-04-25T14:52:19.298334+0000) 2024-04-25T14:52:26.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:25 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:25.959+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:26.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:25 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:25.959+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:52:26.316 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@osd.4.service' 2024-04-25T14:52:26.358 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:52:26.359 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-25T14:52:26.359 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-25T14:52:26.359 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@osd.5 2024-04-25T14:52:26.410 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:26 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:26.152+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:26.411 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:26 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:26.152+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:52:26.669 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:26 smithi183 systemd[1]: Stopping Ceph osd.5 for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:52:26.711 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:26 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:26.525+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:54.119723+0000 front 2024-04-25T14:51:54.119777+0000 (oldest deadline 2024-04-25T14:52:18.819789+0000) 2024-04-25T14:52:26.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:26 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:26.375+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:26.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:26 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:26.375+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:52:26.925 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:26 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:26.767+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.275170+0000 front 2024-04-25T14:51:56.275046+0000 (oldest deadline 2024-04-25T14:52:22.174939+0000) 2024-04-25T14:52:26.926 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:26 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5[44266]: 2024-04-25T14:52:26.668+0000 7f2f2b8ee700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:52:26.926 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:26 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5[44266]: 2024-04-25T14:52:26.668+0000 7f2f2b8ee700 -1 osd.5 106 *** Got signal Terminated *** 2024-04-25T14:52:26.926 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:26 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5[44266]: 2024-04-25T14:52:26.668+0000 7f2f2b8ee700 -1 osd.5 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:52:26.928 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:26 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:26.540+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:48.965340+0000 front 2024-04-25T14:51:48.964926+0000 (oldest deadline 2024-04-25T14:52:14.864930+0000) 2024-04-25T14:52:26.928 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:26 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:26.540+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.566861+0000 front 2024-04-25T14:51:56.567029+0000 (oldest deadline 2024-04-25T14:52:20.666494+0000) 2024-04-25T14:52:27.204 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:27 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:27.160+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:27.204 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:27 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:27.160+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:52:27.205 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:26 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5[44266]: 2024-04-25T14:52:26.925+0000 7f2f23eda700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:45.896685+0000 front 2024-04-25T14:51:45.896774+0000 (oldest deadline 2024-04-25T14:52:11.196601+0000) 2024-04-25T14:52:27.205 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:26 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5[44266]: 2024-04-25T14:52:26.925+0000 7f2f23eda700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:58.198202+0000 front 2024-04-25T14:51:58.198308+0000 (oldest deadline 2024-04-25T14:52:19.298334+0000) 2024-04-25T14:52:27.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:26 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:26.927+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:27.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:26 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:26.927+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:52:27.711 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:27 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:27.521+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:54.119723+0000 front 2024-04-25T14:51:54.119777+0000 (oldest deadline 2024-04-25T14:52:18.819789+0000) 2024-04-25T14:52:27.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:27 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:27.411+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:27.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:27 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:27.411+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:52:27.712 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:27 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:27.411+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:52:27.963 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:27 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:27.568+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:48.965340+0000 front 2024-04-25T14:51:48.964926+0000 (oldest deadline 2024-04-25T14:52:14.864930+0000) 2024-04-25T14:52:27.963 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:27 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:27.568+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.566861+0000 front 2024-04-25T14:51:56.567029+0000 (oldest deadline 2024-04-25T14:52:20.666494+0000) 2024-04-25T14:52:28.115 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:27 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:27.770+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.275170+0000 front 2024-04-25T14:51:56.275046+0000 (oldest deadline 2024-04-25T14:52:22.174939+0000) 2024-04-25T14:52:28.116 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:27 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5[44266]: 2024-04-25T14:52:27.907+0000 7f2f23eda700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:45.896685+0000 front 2024-04-25T14:51:45.896774+0000 (oldest deadline 2024-04-25T14:52:11.196601+0000) 2024-04-25T14:52:28.116 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:27 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5[44266]: 2024-04-25T14:52:27.907+0000 7f2f23eda700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:58.198202+0000 front 2024-04-25T14:51:58.198308+0000 (oldest deadline 2024-04-25T14:52:19.298334+0000) 2024-04-25T14:52:28.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:27 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:27.962+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:28.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:27 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:27.962+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:52:28.454 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:28 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:28.115+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:28.455 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:28 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:28.115+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:52:28.455 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:28 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:28.115+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:52:28.711 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:28 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:28.515+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:54.119723+0000 front 2024-04-25T14:51:54.119777+0000 (oldest deadline 2024-04-25T14:52:18.819789+0000) 2024-04-25T14:52:28.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:28 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:28.447+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:28.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:28 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:28.447+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:52:28.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:28 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:28.447+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:52:28.958 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:28 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:28.564+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:48.965340+0000 front 2024-04-25T14:51:48.964926+0000 (oldest deadline 2024-04-25T14:52:14.864930+0000) 2024-04-25T14:52:28.958 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:28 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:28.564+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.566861+0000 front 2024-04-25T14:51:56.567029+0000 (oldest deadline 2024-04-25T14:52:20.666494+0000) 2024-04-25T14:52:29.134 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:28 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:28.809+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.275170+0000 front 2024-04-25T14:51:56.275046+0000 (oldest deadline 2024-04-25T14:52:22.174939+0000) 2024-04-25T14:52:29.135 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:28 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5[44266]: 2024-04-25T14:52:28.915+0000 7f2f23eda700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:45.896685+0000 front 2024-04-25T14:51:45.896774+0000 (oldest deadline 2024-04-25T14:52:11.196601+0000) 2024-04-25T14:52:29.135 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:28 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5[44266]: 2024-04-25T14:52:28.915+0000 7f2f23eda700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:58.198202+0000 front 2024-04-25T14:51:58.198308+0000 (oldest deadline 2024-04-25T14:52:19.298334+0000) 2024-04-25T14:52:29.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:28 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:28.957+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:29.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:28 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:28.957+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:52:29.454 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:29 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:29.134+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:29.454 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:29 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:29.134+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:52:29.455 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:29 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:29.134+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:52:29.711 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:29 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:29.481+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:54.119723+0000 front 2024-04-25T14:51:54.119777+0000 (oldest deadline 2024-04-25T14:52:18.819789+0000) 2024-04-25T14:52:29.711 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:29 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:29.481+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:03.520880+0000 front 2024-04-25T14:52:03.520808+0000 (oldest deadline 2024-04-25T14:52:29.420795+0000) 2024-04-25T14:52:29.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:29 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:29.405+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:29.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:29 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:29.405+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:52:29.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:29 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:29.405+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:52:29.783 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:29 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:29.517+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:48.965340+0000 front 2024-04-25T14:51:48.964926+0000 (oldest deadline 2024-04-25T14:52:14.864930+0000) 2024-04-25T14:52:29.783 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:29 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:29.517+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.566861+0000 front 2024-04-25T14:51:56.567029+0000 (oldest deadline 2024-04-25T14:52:20.666494+0000) 2024-04-25T14:52:29.783 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:29 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:29.517+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.968881+0000 front 2024-04-25T14:52:05.968824+0000 (oldest deadline 2024-04-25T14:52:28.869067+0000) 2024-04-25T14:52:30.095 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:29 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:29.808+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.275170+0000 front 2024-04-25T14:51:56.275046+0000 (oldest deadline 2024-04-25T14:52:22.174939+0000) 2024-04-25T14:52:30.095 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:29 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:29.808+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.675809+0000 front 2024-04-25T14:52:05.675880+0000 (oldest deadline 2024-04-25T14:52:29.775996+0000) 2024-04-25T14:52:30.095 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:29 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5[44266]: 2024-04-25T14:52:29.882+0000 7f2f23eda700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:45.896685+0000 front 2024-04-25T14:51:45.896774+0000 (oldest deadline 2024-04-25T14:52:11.196601+0000) 2024-04-25T14:52:30.096 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:29 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5[44266]: 2024-04-25T14:52:29.882+0000 7f2f23eda700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:58.198202+0000 front 2024-04-25T14:51:58.198308+0000 (oldest deadline 2024-04-25T14:52:19.298334+0000) 2024-04-25T14:52:30.096 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:29 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5[44266]: 2024-04-25T14:52:29.882+0000 7f2f23eda700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.000452+0000 front 2024-04-25T14:52:05.000417+0000 (oldest deadline 2024-04-25T14:52:29.100612+0000) 2024-04-25T14:52:30.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:29 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:29.959+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:30.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:29 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:29.959+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:52:30.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:29 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:29.959+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:04.536360+0000 front 2024-04-25T14:52:04.536471+0000 (oldest deadline 2024-04-25T14:52:29.236751+0000) 2024-04-25T14:52:30.454 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:30 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:30.094+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:30.454 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:30 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:30.094+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:52:30.454 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:30 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:30.094+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:52:30.711 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:30 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:30.492+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:54.119723+0000 front 2024-04-25T14:51:54.119777+0000 (oldest deadline 2024-04-25T14:52:18.819789+0000) 2024-04-25T14:52:30.711 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:30 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:30.492+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:03.520880+0000 front 2024-04-25T14:52:03.520808+0000 (oldest deadline 2024-04-25T14:52:29.420795+0000) 2024-04-25T14:52:30.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:30 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:30.380+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:30.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:30 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:30.380+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:52:30.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:30 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:30.380+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:52:30.932 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:30 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:30.556+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:48.965340+0000 front 2024-04-25T14:51:48.964926+0000 (oldest deadline 2024-04-25T14:52:14.864930+0000) 2024-04-25T14:52:30.933 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:30 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:30.556+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.566861+0000 front 2024-04-25T14:51:56.567029+0000 (oldest deadline 2024-04-25T14:52:20.666494+0000) 2024-04-25T14:52:30.933 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:30 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:30.556+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.968881+0000 front 2024-04-25T14:52:05.968824+0000 (oldest deadline 2024-04-25T14:52:28.869067+0000) 2024-04-25T14:52:31.118 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:30 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:30.759+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.275170+0000 front 2024-04-25T14:51:56.275046+0000 (oldest deadline 2024-04-25T14:52:22.174939+0000) 2024-04-25T14:52:31.118 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:30 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:30.759+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.675809+0000 front 2024-04-25T14:52:05.675880+0000 (oldest deadline 2024-04-25T14:52:29.775996+0000) 2024-04-25T14:52:31.119 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:30 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5[44266]: 2024-04-25T14:52:30.895+0000 7f2f23eda700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:45.896685+0000 front 2024-04-25T14:51:45.896774+0000 (oldest deadline 2024-04-25T14:52:11.196601+0000) 2024-04-25T14:52:31.119 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:30 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5[44266]: 2024-04-25T14:52:30.895+0000 7f2f23eda700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:58.198202+0000 front 2024-04-25T14:51:58.198308+0000 (oldest deadline 2024-04-25T14:52:19.298334+0000) 2024-04-25T14:52:31.119 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:30 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5[44266]: 2024-04-25T14:52:30.895+0000 7f2f23eda700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.000452+0000 front 2024-04-25T14:52:05.000417+0000 (oldest deadline 2024-04-25T14:52:29.100612+0000) 2024-04-25T14:52:31.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:30 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:30.931+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:31.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:30 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:30.931+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:52:31.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:30 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:30.931+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:04.536360+0000 front 2024-04-25T14:52:04.536471+0000 (oldest deadline 2024-04-25T14:52:29.236751+0000) 2024-04-25T14:52:31.386 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:31 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:31.118+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:31.387 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:31 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:31.118+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:52:31.387 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:31 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:31.118+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:52:31.711 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:31 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:31.482+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:54.119723+0000 front 2024-04-25T14:51:54.119777+0000 (oldest deadline 2024-04-25T14:52:18.819789+0000) 2024-04-25T14:52:31.711 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:31 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:31.482+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:03.520880+0000 front 2024-04-25T14:52:03.520808+0000 (oldest deadline 2024-04-25T14:52:29.420795+0000) 2024-04-25T14:52:31.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:31 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:31.409+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:31.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:31 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:31.409+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:52:31.712 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:31 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:31.409+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:52:31.939 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:31 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:31.584+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:48.965340+0000 front 2024-04-25T14:51:48.964926+0000 (oldest deadline 2024-04-25T14:52:14.864930+0000) 2024-04-25T14:52:31.939 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:31 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:31.584+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.566861+0000 front 2024-04-25T14:51:56.567029+0000 (oldest deadline 2024-04-25T14:52:20.666494+0000) 2024-04-25T14:52:31.939 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:31 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:31.584+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.968881+0000 front 2024-04-25T14:52:05.968824+0000 (oldest deadline 2024-04-25T14:52:28.869067+0000) 2024-04-25T14:52:32.035 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:31 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:31.740+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.275170+0000 front 2024-04-25T14:51:56.275046+0000 (oldest deadline 2024-04-25T14:52:22.174939+0000) 2024-04-25T14:52:32.035 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:31 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:31.740+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.675809+0000 front 2024-04-25T14:52:05.675880+0000 (oldest deadline 2024-04-25T14:52:29.775996+0000) 2024-04-25T14:52:32.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:31 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:31.938+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:32.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:31 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:31.938+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:52:32.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:31 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:31.938+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:04.536360+0000 front 2024-04-25T14:52:04.536471+0000 (oldest deadline 2024-04-25T14:52:29.236751+0000) 2024-04-25T14:52:32.318 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:32 smithi183 podman[68741]: 2024-04-25 14:52:32.035158984 +0000 UTC m=+5.478417858 container died 9498b3c73c9eb93a504c36fcee9e16b1787f7845413b71d6ddd58e5ec52aa8db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T14:52:32.319 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:32 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:32.167+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:32.319 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:32 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:32.167+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:52:32.319 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:32 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:32.167+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:52:32.678 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:32 smithi183 podman[68741]: 2024-04-25 14:52:32.410258639 +0000 UTC m=+5.853517507 container cleanup 9498b3c73c9eb93a504c36fcee9e16b1787f7845413b71d6ddd58e5ec52aa8db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5) 2024-04-25T14:52:32.678 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:32 smithi183 bash[68741]: ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5 2024-04-25T14:52:32.711 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:32 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:32.491+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:54.119723+0000 front 2024-04-25T14:51:54.119777+0000 (oldest deadline 2024-04-25T14:52:18.819789+0000) 2024-04-25T14:52:32.711 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:32 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:32.491+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:03.520880+0000 front 2024-04-25T14:52:03.520808+0000 (oldest deadline 2024-04-25T14:52:29.420795+0000) 2024-04-25T14:52:32.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:32 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:32.448+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:32.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:32 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:32.448+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:52:32.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:32 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:32.448+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:52:32.933 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:32 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:32.634+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:48.965340+0000 front 2024-04-25T14:51:48.964926+0000 (oldest deadline 2024-04-25T14:52:14.864930+0000) 2024-04-25T14:52:32.933 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:32 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:32.634+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.566861+0000 front 2024-04-25T14:51:56.567029+0000 (oldest deadline 2024-04-25T14:52:20.666494+0000) 2024-04-25T14:52:32.933 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:32 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:32.634+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.968881+0000 front 2024-04-25T14:52:05.968824+0000 (oldest deadline 2024-04-25T14:52:28.869067+0000) 2024-04-25T14:52:32.954 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:32 smithi183 podman[68769]: 2024-04-25 14:52:32.678140873 +0000 UTC m=+0.643532828 container remove 9498b3c73c9eb93a504c36fcee9e16b1787f7845413b71d6ddd58e5ec52aa8db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-25T14:52:32.954 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:32 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:32.726+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.275170+0000 front 2024-04-25T14:51:56.275046+0000 (oldest deadline 2024-04-25T14:52:22.174939+0000) 2024-04-25T14:52:32.954 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:32 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:32.726+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.675809+0000 front 2024-04-25T14:52:05.675880+0000 (oldest deadline 2024-04-25T14:52:29.775996+0000) 2024-04-25T14:52:33.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:32 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:32.932+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:33.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:32 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:32.932+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:52:33.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:32 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:32.932+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:04.536360+0000 front 2024-04-25T14:52:04.536471+0000 (oldest deadline 2024-04-25T14:52:29.236751+0000) 2024-04-25T14:52:33.298 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:33 smithi183 podman[68864]: 2024-04-25 14:52:32.935125523 +0000 UTC m=+0.017738377 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T14:52:33.298 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:33 smithi183 podman[68864]: 2024-04-25 14:52:33.062356393 +0000 UTC m=+0.144969233 container create 3d22e223340208f1b3333ad2052e73fa26f0e5113a89e145a04a7d717f6f93a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5-deactivate, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:52:33.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:33 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:33.118+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:33.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:33 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:33.118+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:52:33.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:33 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:33.118+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:52:33.596 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:33 smithi183 podman[68864]: 2024-04-25 14:52:33.298045983 +0000 UTC m=+0.380658824 container init 3d22e223340208f1b3333ad2052e73fa26f0e5113a89e145a04a7d717f6f93a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:52:33.597 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:33 smithi183 podman[68864]: 2024-04-25 14:52:33.301523641 +0000 UTC m=+0.384136482 container start 3d22e223340208f1b3333ad2052e73fa26f0e5113a89e145a04a7d717f6f93a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-25T14:52:33.597 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:33 smithi183 podman[68864]: 2024-04-25 14:52:33.389851177 +0000 UTC m=+0.472464020 container attach 3d22e223340208f1b3333ad2052e73fa26f0e5113a89e145a04a7d717f6f93a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-25T14:52:33.711 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:33 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:33.505+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:54.119723+0000 front 2024-04-25T14:51:54.119777+0000 (oldest deadline 2024-04-25T14:52:18.819789+0000) 2024-04-25T14:52:33.711 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:33 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:33.505+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:03.520880+0000 front 2024-04-25T14:52:03.520808+0000 (oldest deadline 2024-04-25T14:52:29.420795+0000) 2024-04-25T14:52:33.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:33 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:33.445+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:33.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:33 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:33.445+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:52:33.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:33 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:33.445+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:52:33.884 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:33 smithi183 podman[68864]: 2024-04-25 14:52:33.596660122 +0000 UTC m=+0.679272962 container died 3d22e223340208f1b3333ad2052e73fa26f0e5113a89e145a04a7d717f6f93a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-25T14:52:33.884 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:33 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:33.775+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.275170+0000 front 2024-04-25T14:51:56.275046+0000 (oldest deadline 2024-04-25T14:52:22.174939+0000) 2024-04-25T14:52:33.884 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:33 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:33.775+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.675809+0000 front 2024-04-25T14:52:05.675880+0000 (oldest deadline 2024-04-25T14:52:29.775996+0000) 2024-04-25T14:52:33.923 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:33 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:33.606+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:48.965340+0000 front 2024-04-25T14:51:48.964926+0000 (oldest deadline 2024-04-25T14:52:14.864930+0000) 2024-04-25T14:52:33.923 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:33 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:33.606+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.566861+0000 front 2024-04-25T14:51:56.567029+0000 (oldest deadline 2024-04-25T14:52:20.666494+0000) 2024-04-25T14:52:33.923 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:33 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:33.606+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.968881+0000 front 2024-04-25T14:52:05.968824+0000 (oldest deadline 2024-04-25T14:52:28.869067+0000) 2024-04-25T14:52:34.142 INFO:journalctl@ceph.osd.5.smithi183.stdout:Apr 25 14:52:34 smithi183 podman[68890]: 2024-04-25 14:52:34.126015696 +0000 UTC m=+0.523498438 container remove 3d22e223340208f1b3333ad2052e73fa26f0e5113a89e145a04a7d717f6f93a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-25T14:52:34.143 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:34 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:34.100+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:34.143 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:34 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:34.100+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:52:34.143 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:34 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:34.100+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:52:34.147 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@osd.5.service' 2024-04-25T14:52:34.184 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:52:34.184 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-25T14:52:34.184 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-25T14:52:34.184 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@osd.6 2024-04-25T14:52:34.266 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:33 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:33.922+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:34.266 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:33 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:33.922+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:52:34.267 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:33 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:33.922+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:04.536360+0000 front 2024-04-25T14:52:04.536471+0000 (oldest deadline 2024-04-25T14:52:29.236751+0000) 2024-04-25T14:52:34.533 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:34 smithi053 systemd[1]: Stopping Ceph osd.6 for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:52:34.711 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:34 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:34.485+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:54.119723+0000 front 2024-04-25T14:51:54.119777+0000 (oldest deadline 2024-04-25T14:52:18.819789+0000) 2024-04-25T14:52:34.711 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:34 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:34.485+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:03.520880+0000 front 2024-04-25T14:52:03.520808+0000 (oldest deadline 2024-04-25T14:52:29.420795+0000) 2024-04-25T14:52:34.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:34 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:34.427+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:34.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:34 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:34.427+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:52:34.712 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:34 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:34.427+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:52:34.901 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:34 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:34.570+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:48.965340+0000 front 2024-04-25T14:51:48.964926+0000 (oldest deadline 2024-04-25T14:52:14.864930+0000) 2024-04-25T14:52:34.901 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:34 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:34.570+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.566861+0000 front 2024-04-25T14:51:56.567029+0000 (oldest deadline 2024-04-25T14:52:20.666494+0000) 2024-04-25T14:52:34.901 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:34 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:34.570+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.968881+0000 front 2024-04-25T14:52:05.968824+0000 (oldest deadline 2024-04-25T14:52:28.869067+0000) 2024-04-25T14:52:34.901 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:34 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:34.770+0000 7f268aa00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:52:34.901 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:34 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:34.770+0000 7f268aa00700 -1 osd.6 106 *** Got signal Terminated *** 2024-04-25T14:52:34.902 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:34 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:34.770+0000 7f268aa00700 -1 osd.6 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:52:35.120 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:34 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:34.814+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.275170+0000 front 2024-04-25T14:51:56.275046+0000 (oldest deadline 2024-04-25T14:52:22.174939+0000) 2024-04-25T14:52:35.120 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:34 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:34.814+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.675809+0000 front 2024-04-25T14:52:05.675880+0000 (oldest deadline 2024-04-25T14:52:29.775996+0000) 2024-04-25T14:52:35.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:34 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:34.899+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:35.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:34 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:34.899+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:52:35.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:34 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:34.899+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:04.536360+0000 front 2024-04-25T14:52:04.536471+0000 (oldest deadline 2024-04-25T14:52:29.236751+0000) 2024-04-25T14:52:35.454 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:35 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:35.120+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:35.455 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:35 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:35.120+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:52:35.455 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:35 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:35.120+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:52:35.711 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:35 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:35.495+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:54.119723+0000 front 2024-04-25T14:51:54.119777+0000 (oldest deadline 2024-04-25T14:52:18.819789+0000) 2024-04-25T14:52:35.711 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:35 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:35.495+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:03.520880+0000 front 2024-04-25T14:52:03.520808+0000 (oldest deadline 2024-04-25T14:52:29.420795+0000) 2024-04-25T14:52:35.711 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:35 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:35.495+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:10.522059+0000 front 2024-04-25T14:52:10.522155+0000 (oldest deadline 2024-04-25T14:52:35.221853+0000) 2024-04-25T14:52:35.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:35 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:35.385+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:35.712 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:35 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:35.385+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:52:35.712 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:35 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:35.385+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:52:35.855 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:35 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:35.545+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:48.965340+0000 front 2024-04-25T14:51:48.964926+0000 (oldest deadline 2024-04-25T14:52:14.864930+0000) 2024-04-25T14:52:35.856 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:35 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:35.545+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.566861+0000 front 2024-04-25T14:51:56.567029+0000 (oldest deadline 2024-04-25T14:52:20.666494+0000) 2024-04-25T14:52:35.856 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:35 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:35.545+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.968881+0000 front 2024-04-25T14:52:05.968824+0000 (oldest deadline 2024-04-25T14:52:28.869067+0000) 2024-04-25T14:52:35.856 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:35 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:35.545+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:13.471890+0000 front 2024-04-25T14:52:13.471532+0000 (oldest deadline 2024-04-25T14:52:35.171408+0000) 2024-04-25T14:52:36.158 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:35 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:35.771+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.275170+0000 front 2024-04-25T14:51:56.275046+0000 (oldest deadline 2024-04-25T14:52:22.174939+0000) 2024-04-25T14:52:36.159 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:35 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:35.771+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.675809+0000 front 2024-04-25T14:52:05.675880+0000 (oldest deadline 2024-04-25T14:52:29.775996+0000) 2024-04-25T14:52:36.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:35 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:35.854+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:36.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:35 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:35.854+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:52:36.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:35 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:35.854+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:04.536360+0000 front 2024-04-25T14:52:04.536471+0000 (oldest deadline 2024-04-25T14:52:29.236751+0000) 2024-04-25T14:52:36.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:35 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:35.854+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:13.339516+0000 front 2024-04-25T14:52:13.339648+0000 (oldest deadline 2024-04-25T14:52:35.039780+0000) 2024-04-25T14:52:36.382 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:36 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:36.379+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:36.382 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:36 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:36.379+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:52:36.454 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:36 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:36.158+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:36.454 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:36 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:36.158+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:52:36.454 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:36 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:36.158+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:52:36.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:36 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:36.379+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:52:36.711 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:36 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:36.479+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:54.119723+0000 front 2024-04-25T14:51:54.119777+0000 (oldest deadline 2024-04-25T14:52:18.819789+0000) 2024-04-25T14:52:36.711 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:36 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:36.479+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:03.520880+0000 front 2024-04-25T14:52:03.520808+0000 (oldest deadline 2024-04-25T14:52:29.420795+0000) 2024-04-25T14:52:36.711 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:36 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:36.479+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:10.522059+0000 front 2024-04-25T14:52:10.522155+0000 (oldest deadline 2024-04-25T14:52:35.221853+0000) 2024-04-25T14:52:36.894 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:36 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:36.536+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:48.965340+0000 front 2024-04-25T14:51:48.964926+0000 (oldest deadline 2024-04-25T14:52:14.864930+0000) 2024-04-25T14:52:36.894 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:36 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:36.536+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.566861+0000 front 2024-04-25T14:51:56.567029+0000 (oldest deadline 2024-04-25T14:52:20.666494+0000) 2024-04-25T14:52:36.894 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:36 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:36.536+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.968881+0000 front 2024-04-25T14:52:05.968824+0000 (oldest deadline 2024-04-25T14:52:28.869067+0000) 2024-04-25T14:52:36.894 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:36 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:36.536+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:13.471890+0000 front 2024-04-25T14:52:13.471532+0000 (oldest deadline 2024-04-25T14:52:35.171408+0000) 2024-04-25T14:52:37.167 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:36 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:36.785+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.275170+0000 front 2024-04-25T14:51:56.275046+0000 (oldest deadline 2024-04-25T14:52:22.174939+0000) 2024-04-25T14:52:37.168 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:36 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:36.785+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.675809+0000 front 2024-04-25T14:52:05.675880+0000 (oldest deadline 2024-04-25T14:52:29.775996+0000) 2024-04-25T14:52:37.168 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:36 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:36.785+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:12.077368+0000 front 2024-04-25T14:52:12.077727+0000 (oldest deadline 2024-04-25T14:52:36.777146+0000) 2024-04-25T14:52:37.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:36 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:36.890+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:37.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:36 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:36.890+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:52:37.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:36 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:36.890+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:04.536360+0000 front 2024-04-25T14:52:04.536471+0000 (oldest deadline 2024-04-25T14:52:29.236751+0000) 2024-04-25T14:52:37.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:36 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:36.890+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:13.339516+0000 front 2024-04-25T14:52:13.339648+0000 (oldest deadline 2024-04-25T14:52:35.039780+0000) 2024-04-25T14:52:37.454 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:37 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:37.167+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:37.454 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:37 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:37.167+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:52:37.455 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:37 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:37.167+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:52:37.711 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:37 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:37.512+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:54.119723+0000 front 2024-04-25T14:51:54.119777+0000 (oldest deadline 2024-04-25T14:52:18.819789+0000) 2024-04-25T14:52:37.711 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:37 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:37.512+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:03.520880+0000 front 2024-04-25T14:52:03.520808+0000 (oldest deadline 2024-04-25T14:52:29.420795+0000) 2024-04-25T14:52:37.712 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:37 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:37.512+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:10.522059+0000 front 2024-04-25T14:52:10.522155+0000 (oldest deadline 2024-04-25T14:52:35.221853+0000) 2024-04-25T14:52:37.712 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:37 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:37.415+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:37.712 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:37 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:37.415+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:52:37.712 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:37 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:37.415+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:52:37.712 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:37 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:37.415+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:11.873721+0000 front 2024-04-25T14:52:11.874007+0000 (oldest deadline 2024-04-25T14:52:37.073925+0000) 2024-04-25T14:52:37.896 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:37 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:37.533+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:48.965340+0000 front 2024-04-25T14:51:48.964926+0000 (oldest deadline 2024-04-25T14:52:14.864930+0000) 2024-04-25T14:52:37.896 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:37 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:37.533+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.566861+0000 front 2024-04-25T14:51:56.567029+0000 (oldest deadline 2024-04-25T14:52:20.666494+0000) 2024-04-25T14:52:37.896 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:37 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:37.533+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.968881+0000 front 2024-04-25T14:52:05.968824+0000 (oldest deadline 2024-04-25T14:52:28.869067+0000) 2024-04-25T14:52:37.896 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:37 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:37.533+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:13.471890+0000 front 2024-04-25T14:52:13.471532+0000 (oldest deadline 2024-04-25T14:52:35.171408+0000) 2024-04-25T14:52:38.154 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:37 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:37.835+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.275170+0000 front 2024-04-25T14:51:56.275046+0000 (oldest deadline 2024-04-25T14:52:22.174939+0000) 2024-04-25T14:52:38.154 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:37 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:37.835+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.675809+0000 front 2024-04-25T14:52:05.675880+0000 (oldest deadline 2024-04-25T14:52:29.775996+0000) 2024-04-25T14:52:38.154 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:37 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:37.835+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:12.077368+0000 front 2024-04-25T14:52:12.077727+0000 (oldest deadline 2024-04-25T14:52:36.777146+0000) 2024-04-25T14:52:38.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:37 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:37.894+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:38.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:37 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:37.894+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:52:38.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:37 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:37.894+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:04.536360+0000 front 2024-04-25T14:52:04.536471+0000 (oldest deadline 2024-04-25T14:52:29.236751+0000) 2024-04-25T14:52:38.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:37 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:37.894+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:13.339516+0000 front 2024-04-25T14:52:13.339648+0000 (oldest deadline 2024-04-25T14:52:35.039780+0000) 2024-04-25T14:52:38.454 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:38 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:38.153+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:38.454 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:38 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:38.153+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:52:38.455 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:38 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:38.153+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:52:38.711 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:38 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:38.548+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:54.119723+0000 front 2024-04-25T14:51:54.119777+0000 (oldest deadline 2024-04-25T14:52:18.819789+0000) 2024-04-25T14:52:38.711 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:38 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:38.548+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:03.520880+0000 front 2024-04-25T14:52:03.520808+0000 (oldest deadline 2024-04-25T14:52:29.420795+0000) 2024-04-25T14:52:38.711 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:38 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:38.548+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:10.522059+0000 front 2024-04-25T14:52:10.522155+0000 (oldest deadline 2024-04-25T14:52:35.221853+0000) 2024-04-25T14:52:38.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:38 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:38.393+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:38.712 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:38 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:38.393+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:52:38.712 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:38 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:38.393+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:52:38.712 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:38 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:38.393+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:11.873721+0000 front 2024-04-25T14:52:11.874007+0000 (oldest deadline 2024-04-25T14:52:37.073925+0000) 2024-04-25T14:52:38.884 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:38 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:38.582+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:48.965340+0000 front 2024-04-25T14:51:48.964926+0000 (oldest deadline 2024-04-25T14:52:14.864930+0000) 2024-04-25T14:52:38.884 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:38 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:38.582+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.566861+0000 front 2024-04-25T14:51:56.567029+0000 (oldest deadline 2024-04-25T14:52:20.666494+0000) 2024-04-25T14:52:38.884 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:38 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:38.582+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.968881+0000 front 2024-04-25T14:52:05.968824+0000 (oldest deadline 2024-04-25T14:52:28.869067+0000) 2024-04-25T14:52:38.884 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:38 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:38.582+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:13.471890+0000 front 2024-04-25T14:52:13.471532+0000 (oldest deadline 2024-04-25T14:52:35.171408+0000) 2024-04-25T14:52:39.185 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:38 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:38.807+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.275170+0000 front 2024-04-25T14:51:56.275046+0000 (oldest deadline 2024-04-25T14:52:22.174939+0000) 2024-04-25T14:52:39.185 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:38 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:38.807+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.675809+0000 front 2024-04-25T14:52:05.675880+0000 (oldest deadline 2024-04-25T14:52:29.775996+0000) 2024-04-25T14:52:39.185 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:38 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:38.807+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:12.077368+0000 front 2024-04-25T14:52:12.077727+0000 (oldest deadline 2024-04-25T14:52:36.777146+0000) 2024-04-25T14:52:39.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:38 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:38.883+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:39.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:38 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:38.883+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:52:39.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:38 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:38.883+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:04.536360+0000 front 2024-04-25T14:52:04.536471+0000 (oldest deadline 2024-04-25T14:52:29.236751+0000) 2024-04-25T14:52:39.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:38 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:38.883+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:13.339516+0000 front 2024-04-25T14:52:13.339648+0000 (oldest deadline 2024-04-25T14:52:35.039780+0000) 2024-04-25T14:52:39.454 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:39 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:39.184+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:39.454 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:39 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:39.184+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:52:39.455 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:39 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:39.184+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:52:39.711 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:39 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:39.544+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:54.119723+0000 front 2024-04-25T14:51:54.119777+0000 (oldest deadline 2024-04-25T14:52:18.819789+0000) 2024-04-25T14:52:39.711 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:39 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:39.544+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:03.520880+0000 front 2024-04-25T14:52:03.520808+0000 (oldest deadline 2024-04-25T14:52:29.420795+0000) 2024-04-25T14:52:39.711 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:39 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:39.544+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:10.522059+0000 front 2024-04-25T14:52:10.522155+0000 (oldest deadline 2024-04-25T14:52:35.221853+0000) 2024-04-25T14:52:39.712 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:39 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:39.347+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:39.712 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:39 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:39.347+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:52:39.712 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:39 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:39.347+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:52:39.712 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:39 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:39.347+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:11.873721+0000 front 2024-04-25T14:52:11.874007+0000 (oldest deadline 2024-04-25T14:52:37.073925+0000) 2024-04-25T14:52:39.899 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:39 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:39.582+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:48.965340+0000 front 2024-04-25T14:51:48.964926+0000 (oldest deadline 2024-04-25T14:52:14.864930+0000) 2024-04-25T14:52:39.899 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:39 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:39.582+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.566861+0000 front 2024-04-25T14:51:56.567029+0000 (oldest deadline 2024-04-25T14:52:20.666494+0000) 2024-04-25T14:52:39.899 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:39 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:39.582+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.968881+0000 front 2024-04-25T14:52:05.968824+0000 (oldest deadline 2024-04-25T14:52:28.869067+0000) 2024-04-25T14:52:39.899 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:39 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6[55594]: 2024-04-25T14:52:39.582+0000 7f2686200700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:13.471890+0000 front 2024-04-25T14:52:13.471532+0000 (oldest deadline 2024-04-25T14:52:35.171408+0000) 2024-04-25T14:52:40.204 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:39 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:39.823+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.275170+0000 front 2024-04-25T14:51:56.275046+0000 (oldest deadline 2024-04-25T14:52:22.174939+0000) 2024-04-25T14:52:40.204 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:39 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:39.823+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.675809+0000 front 2024-04-25T14:52:05.675880+0000 (oldest deadline 2024-04-25T14:52:29.775996+0000) 2024-04-25T14:52:40.205 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:39 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:39.823+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:12.077368+0000 front 2024-04-25T14:52:12.077727+0000 (oldest deadline 2024-04-25T14:52:36.777146+0000) 2024-04-25T14:52:40.241 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:39 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:39.898+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:40.241 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:39 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:39.898+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:52:40.242 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:39 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:39.898+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:04.536360+0000 front 2024-04-25T14:52:04.536471+0000 (oldest deadline 2024-04-25T14:52:29.236751+0000) 2024-04-25T14:52:40.242 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:39 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:39.898+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:13.339516+0000 front 2024-04-25T14:52:13.339648+0000 (oldest deadline 2024-04-25T14:52:35.039780+0000) 2024-04-25T14:52:40.533 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:40 smithi053 podman[86670]: 2024-04-25 14:52:40.24098077 +0000 UTC m=+5.599088210 container died 9fde94d2e826a4c051f0e773d24e66673ae66c215c5abc4962e3f792cab11b7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-25T14:52:40.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:40 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:40.225+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:40.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:40 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:40.225+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:52:40.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:40 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:40.225+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:52:40.711 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:40 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:40.572+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:54.119723+0000 front 2024-04-25T14:51:54.119777+0000 (oldest deadline 2024-04-25T14:52:18.819789+0000) 2024-04-25T14:52:40.711 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:40 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:40.572+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:03.520880+0000 front 2024-04-25T14:52:03.520808+0000 (oldest deadline 2024-04-25T14:52:29.420795+0000) 2024-04-25T14:52:40.711 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:40 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:40.572+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:10.522059+0000 front 2024-04-25T14:52:10.522155+0000 (oldest deadline 2024-04-25T14:52:35.221853+0000) 2024-04-25T14:52:40.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:40 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:40.329+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:40.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:40 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:40.329+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:52:40.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:40 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:40.329+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:52:40.712 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:40 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:40.329+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:11.873721+0000 front 2024-04-25T14:52:11.874007+0000 (oldest deadline 2024-04-25T14:52:37.073925+0000) 2024-04-25T14:52:40.982 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:40 smithi053 podman[86670]: 2024-04-25 14:52:40.797304903 +0000 UTC m=+6.155412336 container cleanup 9fde94d2e826a4c051f0e773d24e66673ae66c215c5abc4962e3f792cab11b7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-25T14:52:40.982 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:40 smithi053 bash[86670]: ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6 2024-04-25T14:52:40.983 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:40 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:40.856+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:40.983 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:40 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:40.857+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:52:40.983 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:40 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:40.857+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:04.536360+0000 front 2024-04-25T14:52:04.536471+0000 (oldest deadline 2024-04-25T14:52:29.236751+0000) 2024-04-25T14:52:40.983 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:40 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:40.857+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:13.339516+0000 front 2024-04-25T14:52:13.339648+0000 (oldest deadline 2024-04-25T14:52:35.039780+0000) 2024-04-25T14:52:41.204 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:40 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:40.803+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.275170+0000 front 2024-04-25T14:51:56.275046+0000 (oldest deadline 2024-04-25T14:52:22.174939+0000) 2024-04-25T14:52:41.204 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:40 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:40.803+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.675809+0000 front 2024-04-25T14:52:05.675880+0000 (oldest deadline 2024-04-25T14:52:29.775996+0000) 2024-04-25T14:52:41.204 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:40 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:40.803+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:12.077368+0000 front 2024-04-25T14:52:12.077727+0000 (oldest deadline 2024-04-25T14:52:36.777146+0000) 2024-04-25T14:52:41.283 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:40 smithi053 podman[86690]: 2024-04-25 14:52:40.981419858 +0000 UTC m=+0.725491876 container remove 9fde94d2e826a4c051f0e773d24e66673ae66c215c5abc4962e3f792cab11b7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:52:41.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:41 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:41.226+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:41.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:41 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:41.226+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:52:41.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:41 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:41.226+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:52:41.711 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:41 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:41.527+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:54.119723+0000 front 2024-04-25T14:51:54.119777+0000 (oldest deadline 2024-04-25T14:52:18.819789+0000) 2024-04-25T14:52:41.711 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:41 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:41.527+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:03.520880+0000 front 2024-04-25T14:52:03.520808+0000 (oldest deadline 2024-04-25T14:52:29.420795+0000) 2024-04-25T14:52:41.711 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:41 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:41.527+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:10.522059+0000 front 2024-04-25T14:52:10.522155+0000 (oldest deadline 2024-04-25T14:52:35.221853+0000) 2024-04-25T14:52:41.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:41 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:41.285+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:41.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:41 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:41.285+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:52:41.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:41 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:41.285+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:52:41.712 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:41 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:41.285+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:11.873721+0000 front 2024-04-25T14:52:11.874007+0000 (oldest deadline 2024-04-25T14:52:37.073925+0000) 2024-04-25T14:52:41.975 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:41 smithi053 podman[86787]: 2024-04-25 14:52:41.635406101 +0000 UTC m=+0.034194707 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T14:52:41.975 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:41 smithi053 podman[86787]: 2024-04-25 14:52:41.774164204 +0000 UTC m=+0.172952849 container create 343909d5365529d7141400bb1a44e66bd06fb2dc350b47d52f89e7194c091aea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:52:41.976 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:41 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:41.882+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:41.976 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:41 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:41.882+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:52:41.976 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:41 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:41.882+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:04.536360+0000 front 2024-04-25T14:52:04.536471+0000 (oldest deadline 2024-04-25T14:52:29.236751+0000) 2024-04-25T14:52:41.976 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:41 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:41.882+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:13.339516+0000 front 2024-04-25T14:52:13.339648+0000 (oldest deadline 2024-04-25T14:52:35.039780+0000) 2024-04-25T14:52:42.204 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:41 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:41.841+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.275170+0000 front 2024-04-25T14:51:56.275046+0000 (oldest deadline 2024-04-25T14:52:22.174939+0000) 2024-04-25T14:52:42.204 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:41 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:41.841+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.675809+0000 front 2024-04-25T14:52:05.675880+0000 (oldest deadline 2024-04-25T14:52:29.775996+0000) 2024-04-25T14:52:42.205 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:41 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:41.841+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:12.077368+0000 front 2024-04-25T14:52:12.077727+0000 (oldest deadline 2024-04-25T14:52:36.777146+0000) 2024-04-25T14:52:42.283 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:42 smithi053 podman[86787]: 2024-04-25 14:52:42.078397302 +0000 UTC m=+0.477185973 container init 343909d5365529d7141400bb1a44e66bd06fb2dc350b47d52f89e7194c091aea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-25T14:52:42.283 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:42 smithi053 podman[86787]: 2024-04-25 14:52:42.090094143 +0000 UTC m=+0.488882768 container start 343909d5365529d7141400bb1a44e66bd06fb2dc350b47d52f89e7194c091aea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:52:42.509 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:42 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:42.247+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:42.510 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:42 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:42.247+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:52:42.510 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:42 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:42.247+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:52:42.510 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:42 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:42.247+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:11.873721+0000 front 2024-04-25T14:52:11.874007+0000 (oldest deadline 2024-04-25T14:52:37.073925+0000) 2024-04-25T14:52:42.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:42 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:42.272+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:42.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:42 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:42.272+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:52:42.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:42 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:42.272+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:52:42.783 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:42 smithi053 podman[86787]: 2024-04-25 14:52:42.318262353 +0000 UTC m=+0.717050988 container attach 343909d5365529d7141400bb1a44e66bd06fb2dc350b47d52f89e7194c091aea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-25T14:52:42.783 INFO:journalctl@ceph.osd.6.smithi053.stdout:Apr 25 14:52:42 smithi053 podman[86787]: 2024-04-25 14:52:42.561236908 +0000 UTC m=+0.960025554 container died 343909d5365529d7141400bb1a44e66bd06fb2dc350b47d52f89e7194c091aea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-6-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T14:52:42.961 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:42 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:42.509+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:54.119723+0000 front 2024-04-25T14:51:54.119777+0000 (oldest deadline 2024-04-25T14:52:18.819789+0000) 2024-04-25T14:52:42.961 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:42 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:42.509+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:03.520880+0000 front 2024-04-25T14:52:03.520808+0000 (oldest deadline 2024-04-25T14:52:29.420795+0000) 2024-04-25T14:52:42.961 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:42 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:42.509+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:10.522059+0000 front 2024-04-25T14:52:10.522155+0000 (oldest deadline 2024-04-25T14:52:35.221853+0000) 2024-04-25T14:52:43.204 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:42 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:42.877+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.275170+0000 front 2024-04-25T14:51:56.275046+0000 (oldest deadline 2024-04-25T14:52:22.174939+0000) 2024-04-25T14:52:43.204 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:42 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:42.877+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.675809+0000 front 2024-04-25T14:52:05.675880+0000 (oldest deadline 2024-04-25T14:52:29.775996+0000) 2024-04-25T14:52:43.204 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:42 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:42.877+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:12.077368+0000 front 2024-04-25T14:52:12.077727+0000 (oldest deadline 2024-04-25T14:52:36.777146+0000) 2024-04-25T14:52:43.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:42 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:42.892+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:43.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:42 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:42.892+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:52:43.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:42 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:42.892+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:04.536360+0000 front 2024-04-25T14:52:04.536471+0000 (oldest deadline 2024-04-25T14:52:29.236751+0000) 2024-04-25T14:52:43.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:42 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:42.892+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:13.339516+0000 front 2024-04-25T14:52:13.339648+0000 (oldest deadline 2024-04-25T14:52:35.039780+0000) 2024-04-25T14:52:43.436 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@osd.6.service' 2024-04-25T14:52:43.507 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:43 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:43.222+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:43.507 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:43 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:43.222+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:52:43.507 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:43 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:43.222+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:52:43.507 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:43 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:43.222+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:11.873721+0000 front 2024-04-25T14:52:11.874007+0000 (oldest deadline 2024-04-25T14:52:37.073925+0000) 2024-04-25T14:52:43.559 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:52:43.559 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-25T14:52:43.559 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-25T14:52:43.559 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@osd.7 2024-04-25T14:52:43.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:43 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:43.253+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:43.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:43 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:43.253+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:52:43.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:43 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:43.253+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:52:43.852 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:43 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:43.506+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:54.119723+0000 front 2024-04-25T14:51:54.119777+0000 (oldest deadline 2024-04-25T14:52:18.819789+0000) 2024-04-25T14:52:43.852 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:43 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:43.506+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:03.520880+0000 front 2024-04-25T14:52:03.520808+0000 (oldest deadline 2024-04-25T14:52:29.420795+0000) 2024-04-25T14:52:43.852 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:43 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:43.506+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:10.522059+0000 front 2024-04-25T14:52:10.522155+0000 (oldest deadline 2024-04-25T14:52:35.221853+0000) 2024-04-25T14:52:43.852 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:43 smithi150 systemd[1]: Stopping Ceph osd.7 for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:52:44.202 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:43 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:43.851+0000 7fc6b481c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:52:44.202 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:43 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:43.851+0000 7fc6b481c700 -1 osd.7 106 *** Got signal Terminated *** 2024-04-25T14:52:44.202 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:43 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:43.851+0000 7fc6b481c700 -1 osd.7 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:52:44.204 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:43 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:43.847+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.275170+0000 front 2024-04-25T14:51:56.275046+0000 (oldest deadline 2024-04-25T14:52:22.174939+0000) 2024-04-25T14:52:44.204 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:43 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:43.847+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.675809+0000 front 2024-04-25T14:52:05.675880+0000 (oldest deadline 2024-04-25T14:52:29.775996+0000) 2024-04-25T14:52:44.204 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:43 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:43.847+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:12.077368+0000 front 2024-04-25T14:52:12.077727+0000 (oldest deadline 2024-04-25T14:52:36.777146+0000) 2024-04-25T14:52:44.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:43 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:43.846+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:44.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:43 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:43.846+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:52:44.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:43 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:43.846+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:04.536360+0000 front 2024-04-25T14:52:04.536471+0000 (oldest deadline 2024-04-25T14:52:29.236751+0000) 2024-04-25T14:52:44.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:43 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:43.846+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:13.339516+0000 front 2024-04-25T14:52:13.339648+0000 (oldest deadline 2024-04-25T14:52:35.039780+0000) 2024-04-25T14:52:44.461 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:44 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:44.201+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:44.461 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:44 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:44.201+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:52:44.461 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:44 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:44.201+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:52:44.461 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:44 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:44.201+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:11.873721+0000 front 2024-04-25T14:52:11.874007+0000 (oldest deadline 2024-04-25T14:52:37.073925+0000) 2024-04-25T14:52:44.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:44 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:44.297+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:44.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:44 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:44.297+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:52:44.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:44 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:44.297+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:52:44.961 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:44 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:44.477+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:54.119723+0000 front 2024-04-25T14:51:54.119777+0000 (oldest deadline 2024-04-25T14:52:18.819789+0000) 2024-04-25T14:52:44.961 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:44 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:44.477+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:03.520880+0000 front 2024-04-25T14:52:03.520808+0000 (oldest deadline 2024-04-25T14:52:29.420795+0000) 2024-04-25T14:52:44.961 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:44 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:44.477+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:10.522059+0000 front 2024-04-25T14:52:10.522155+0000 (oldest deadline 2024-04-25T14:52:35.221853+0000) 2024-04-25T14:52:45.204 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:44 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:44.856+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.275170+0000 front 2024-04-25T14:51:56.275046+0000 (oldest deadline 2024-04-25T14:52:22.174939+0000) 2024-04-25T14:52:45.204 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:44 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:44.856+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.675809+0000 front 2024-04-25T14:52:05.675880+0000 (oldest deadline 2024-04-25T14:52:29.775996+0000) 2024-04-25T14:52:45.205 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:44 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:44.856+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:12.077368+0000 front 2024-04-25T14:52:12.077727+0000 (oldest deadline 2024-04-25T14:52:36.777146+0000) 2024-04-25T14:52:45.205 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:44 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:44.856+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:23.079358+0000 front 2024-04-25T14:52:23.079263+0000 (oldest deadline 2024-04-25T14:52:44.779289+0000) 2024-04-25T14:52:45.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:44 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:44.863+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:45.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:44 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:44.863+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:52:45.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:44 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:44.863+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:04.536360+0000 front 2024-04-25T14:52:04.536471+0000 (oldest deadline 2024-04-25T14:52:29.236751+0000) 2024-04-25T14:52:45.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:44 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:44.863+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:13.339516+0000 front 2024-04-25T14:52:13.339648+0000 (oldest deadline 2024-04-25T14:52:35.039780+0000) 2024-04-25T14:52:45.461 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:45 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:45.184+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:45.461 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:45 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:45.184+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:52:45.461 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:45 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:45.184+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:52:45.462 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:45 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:45.184+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:11.873721+0000 front 2024-04-25T14:52:11.874007+0000 (oldest deadline 2024-04-25T14:52:37.073925+0000) 2024-04-25T14:52:45.462 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:45 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:45.184+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:19.374670+0000 front 2024-04-25T14:52:19.374695+0000 (oldest deadline 2024-04-25T14:52:44.674874+0000) 2024-04-25T14:52:45.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:45 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:45.308+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:45.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:45 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:45.308+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:52:45.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:45 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:45.308+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:52:45.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:45 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:45.308+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:22.079400+0000 front 2024-04-25T14:52:22.079475+0000 (oldest deadline 2024-04-25T14:52:44.379400+0000) 2024-04-25T14:52:45.961 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:45 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:45.486+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:54.119723+0000 front 2024-04-25T14:51:54.119777+0000 (oldest deadline 2024-04-25T14:52:18.819789+0000) 2024-04-25T14:52:45.961 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:45 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:45.486+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:03.520880+0000 front 2024-04-25T14:52:03.520808+0000 (oldest deadline 2024-04-25T14:52:29.420795+0000) 2024-04-25T14:52:45.961 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:45 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:45.486+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:10.522059+0000 front 2024-04-25T14:52:10.522155+0000 (oldest deadline 2024-04-25T14:52:35.221853+0000) 2024-04-25T14:52:46.204 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:45 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:45.898+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.275170+0000 front 2024-04-25T14:51:56.275046+0000 (oldest deadline 2024-04-25T14:52:22.174939+0000) 2024-04-25T14:52:46.204 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:45 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:45.898+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.675809+0000 front 2024-04-25T14:52:05.675880+0000 (oldest deadline 2024-04-25T14:52:29.775996+0000) 2024-04-25T14:52:46.205 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:45 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:45.898+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:12.077368+0000 front 2024-04-25T14:52:12.077727+0000 (oldest deadline 2024-04-25T14:52:36.777146+0000) 2024-04-25T14:52:46.205 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:45 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:45.898+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:23.079358+0000 front 2024-04-25T14:52:23.079263+0000 (oldest deadline 2024-04-25T14:52:44.779289+0000) 2024-04-25T14:52:46.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:45 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:45.830+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:46.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:45 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:45.830+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:52:46.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:45 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:45.830+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:04.536360+0000 front 2024-04-25T14:52:04.536471+0000 (oldest deadline 2024-04-25T14:52:29.236751+0000) 2024-04-25T14:52:46.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:45 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:45.830+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:13.339516+0000 front 2024-04-25T14:52:13.339648+0000 (oldest deadline 2024-04-25T14:52:35.039780+0000) 2024-04-25T14:52:46.467 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:46 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:46.215+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:46.467 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:46 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:46.215+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:52:46.467 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:46 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:46.215+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:52:46.467 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:46 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:46.215+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:11.873721+0000 front 2024-04-25T14:52:11.874007+0000 (oldest deadline 2024-04-25T14:52:37.073925+0000) 2024-04-25T14:52:46.467 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:46 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:46.215+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:19.374670+0000 front 2024-04-25T14:52:19.374695+0000 (oldest deadline 2024-04-25T14:52:44.674874+0000) 2024-04-25T14:52:46.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:46 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:46.266+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:46.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:46 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:46.266+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:52:46.706 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:46 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:46.266+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:52:46.706 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:46 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:46.266+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:22.079400+0000 front 2024-04-25T14:52:22.079475+0000 (oldest deadline 2024-04-25T14:52:44.379400+0000) 2024-04-25T14:52:46.961 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:46 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:46.466+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:54.119723+0000 front 2024-04-25T14:51:54.119777+0000 (oldest deadline 2024-04-25T14:52:18.819789+0000) 2024-04-25T14:52:46.961 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:46 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:46.466+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:03.520880+0000 front 2024-04-25T14:52:03.520808+0000 (oldest deadline 2024-04-25T14:52:29.420795+0000) 2024-04-25T14:52:46.961 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:46 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:46.466+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:10.522059+0000 front 2024-04-25T14:52:10.522155+0000 (oldest deadline 2024-04-25T14:52:35.221853+0000) 2024-04-25T14:52:46.961 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:46 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:46.466+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:22.823060+0000 front 2024-04-25T14:52:22.823021+0000 (oldest deadline 2024-04-25T14:52:46.323160+0000) 2024-04-25T14:52:47.204 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:46 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:46.935+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.275170+0000 front 2024-04-25T14:51:56.275046+0000 (oldest deadline 2024-04-25T14:52:22.174939+0000) 2024-04-25T14:52:47.204 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:46 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:46.935+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.675809+0000 front 2024-04-25T14:52:05.675880+0000 (oldest deadline 2024-04-25T14:52:29.775996+0000) 2024-04-25T14:52:47.205 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:46 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:46.935+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:12.077368+0000 front 2024-04-25T14:52:12.077727+0000 (oldest deadline 2024-04-25T14:52:36.777146+0000) 2024-04-25T14:52:47.205 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:46 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:46.935+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:23.079358+0000 front 2024-04-25T14:52:23.079263+0000 (oldest deadline 2024-04-25T14:52:44.779289+0000) 2024-04-25T14:52:47.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:46 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:46.827+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:47.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:46 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:46.827+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:52:47.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:46 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:46.827+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:04.536360+0000 front 2024-04-25T14:52:04.536471+0000 (oldest deadline 2024-04-25T14:52:29.236751+0000) 2024-04-25T14:52:47.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:46 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:46.827+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:13.339516+0000 front 2024-04-25T14:52:13.339648+0000 (oldest deadline 2024-04-25T14:52:35.039780+0000) 2024-04-25T14:52:47.471 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:47 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:47.217+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:47.471 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:47 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:47.217+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:52:47.471 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:47 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:47.217+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:52:47.471 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:47 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:47.217+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:11.873721+0000 front 2024-04-25T14:52:11.874007+0000 (oldest deadline 2024-04-25T14:52:37.073925+0000) 2024-04-25T14:52:47.472 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:47 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:47.217+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:19.374670+0000 front 2024-04-25T14:52:19.374695+0000 (oldest deadline 2024-04-25T14:52:44.674874+0000) 2024-04-25T14:52:47.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:47 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:47.295+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:47.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:47 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:47.295+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:52:47.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:47 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:47.295+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:52:47.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:47 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:47.295+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:22.079400+0000 front 2024-04-25T14:52:22.079475+0000 (oldest deadline 2024-04-25T14:52:44.379400+0000) 2024-04-25T14:52:47.961 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:47 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:47.470+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:54.119723+0000 front 2024-04-25T14:51:54.119777+0000 (oldest deadline 2024-04-25T14:52:18.819789+0000) 2024-04-25T14:52:47.961 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:47 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:47.470+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:03.520880+0000 front 2024-04-25T14:52:03.520808+0000 (oldest deadline 2024-04-25T14:52:29.420795+0000) 2024-04-25T14:52:47.961 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:47 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:47.470+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:10.522059+0000 front 2024-04-25T14:52:10.522155+0000 (oldest deadline 2024-04-25T14:52:35.221853+0000) 2024-04-25T14:52:47.961 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:47 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:47.470+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:22.823060+0000 front 2024-04-25T14:52:22.823021+0000 (oldest deadline 2024-04-25T14:52:46.323160+0000) 2024-04-25T14:52:48.204 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:47 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:47.892+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.275170+0000 front 2024-04-25T14:51:56.275046+0000 (oldest deadline 2024-04-25T14:52:22.174939+0000) 2024-04-25T14:52:48.204 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:47 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:47.892+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.675809+0000 front 2024-04-25T14:52:05.675880+0000 (oldest deadline 2024-04-25T14:52:29.775996+0000) 2024-04-25T14:52:48.205 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:47 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:47.892+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:12.077368+0000 front 2024-04-25T14:52:12.077727+0000 (oldest deadline 2024-04-25T14:52:36.777146+0000) 2024-04-25T14:52:48.205 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:47 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:47.892+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:23.079358+0000 front 2024-04-25T14:52:23.079263+0000 (oldest deadline 2024-04-25T14:52:44.779289+0000) 2024-04-25T14:52:48.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:47 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:47.850+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:48.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:47 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:47.850+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:52:48.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:47 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:47.850+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:04.536360+0000 front 2024-04-25T14:52:04.536471+0000 (oldest deadline 2024-04-25T14:52:29.236751+0000) 2024-04-25T14:52:48.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:47 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:47.850+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:13.339516+0000 front 2024-04-25T14:52:13.339648+0000 (oldest deadline 2024-04-25T14:52:35.039780+0000) 2024-04-25T14:52:48.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:48 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:48.261+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:48.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:48 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:48.261+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:52:48.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:48 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:48.261+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:52:48.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:48 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:48.261+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:22.079400+0000 front 2024-04-25T14:52:22.079475+0000 (oldest deadline 2024-04-25T14:52:44.379400+0000) 2024-04-25T14:52:48.711 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:48 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:48.456+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:54.119723+0000 front 2024-04-25T14:51:54.119777+0000 (oldest deadline 2024-04-25T14:52:18.819789+0000) 2024-04-25T14:52:48.711 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:48 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:48.456+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:03.520880+0000 front 2024-04-25T14:52:03.520808+0000 (oldest deadline 2024-04-25T14:52:29.420795+0000) 2024-04-25T14:52:48.711 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:48 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:48.456+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:10.522059+0000 front 2024-04-25T14:52:10.522155+0000 (oldest deadline 2024-04-25T14:52:35.221853+0000) 2024-04-25T14:52:48.711 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:48 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7[50472]: 2024-04-25T14:52:48.456+0000 7fc6b0e10700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:22.823060+0000 front 2024-04-25T14:52:22.823021+0000 (oldest deadline 2024-04-25T14:52:46.323160+0000) 2024-04-25T14:52:48.712 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:48 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:48.221+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:48.712 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:48 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:48.221+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:52:48.712 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:48 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:48.221+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:52:48.712 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:48 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:48.221+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:11.873721+0000 front 2024-04-25T14:52:11.874007+0000 (oldest deadline 2024-04-25T14:52:37.073925+0000) 2024-04-25T14:52:48.712 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:48 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:48.221+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:19.374670+0000 front 2024-04-25T14:52:19.374695+0000 (oldest deadline 2024-04-25T14:52:44.674874+0000) 2024-04-25T14:52:49.204 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:48 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:48.879+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.275170+0000 front 2024-04-25T14:51:56.275046+0000 (oldest deadline 2024-04-25T14:52:22.174939+0000) 2024-04-25T14:52:49.204 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:48 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:48.879+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.675809+0000 front 2024-04-25T14:52:05.675880+0000 (oldest deadline 2024-04-25T14:52:29.775996+0000) 2024-04-25T14:52:49.205 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:48 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:48.879+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:12.077368+0000 front 2024-04-25T14:52:12.077727+0000 (oldest deadline 2024-04-25T14:52:36.777146+0000) 2024-04-25T14:52:49.205 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:48 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:48.879+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:23.079358+0000 front 2024-04-25T14:52:23.079263+0000 (oldest deadline 2024-04-25T14:52:44.779289+0000) 2024-04-25T14:52:49.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:48 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:48.863+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:49.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:48 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:48.863+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:52:49.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:48 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:48.863+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:04.536360+0000 front 2024-04-25T14:52:04.536471+0000 (oldest deadline 2024-04-25T14:52:29.236751+0000) 2024-04-25T14:52:49.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:48 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:48.863+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:13.339516+0000 front 2024-04-25T14:52:13.339648+0000 (oldest deadline 2024-04-25T14:52:35.039780+0000) 2024-04-25T14:52:49.611 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:49 smithi150 podman[69018]: 2024-04-25 14:52:49.332748669 +0000 UTC m=+5.576776916 container died 873cf903b4e57072b3d3a1516bb90d209febe08e33e71afd4c1316b134d6fe43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS) 2024-04-25T14:52:49.611 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:49 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:49.256+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:49.611 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:49 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:49.256+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:52:49.611 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:49 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:49.256+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:52:49.612 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:49 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:49.256+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:11.873721+0000 front 2024-04-25T14:52:11.874007+0000 (oldest deadline 2024-04-25T14:52:37.073925+0000) 2024-04-25T14:52:49.612 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:49 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:49.256+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:19.374670+0000 front 2024-04-25T14:52:19.374695+0000 (oldest deadline 2024-04-25T14:52:44.674874+0000) 2024-04-25T14:52:49.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:49 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:49.246+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:49.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:49 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:49.246+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:52:49.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:49 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:49.246+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:52:49.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:49 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:49.246+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:22.079400+0000 front 2024-04-25T14:52:22.079475+0000 (oldest deadline 2024-04-25T14:52:44.379400+0000) 2024-04-25T14:52:49.911 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:49 smithi150 podman[69018]: 2024-04-25 14:52:49.718499351 +0000 UTC m=+5.962527606 container cleanup 873cf903b4e57072b3d3a1516bb90d209febe08e33e71afd4c1316b134d6fe43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-25T14:52:49.911 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:49 smithi150 bash[69018]: ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7 2024-04-25T14:52:50.204 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:49 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:49.846+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.275170+0000 front 2024-04-25T14:51:56.275046+0000 (oldest deadline 2024-04-25T14:52:22.174939+0000) 2024-04-25T14:52:50.205 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:49 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:49.846+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.675809+0000 front 2024-04-25T14:52:05.675880+0000 (oldest deadline 2024-04-25T14:52:29.775996+0000) 2024-04-25T14:52:50.205 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:49 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:49.846+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:12.077368+0000 front 2024-04-25T14:52:12.077727+0000 (oldest deadline 2024-04-25T14:52:36.777146+0000) 2024-04-25T14:52:50.205 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:49 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:49.846+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:23.079358+0000 front 2024-04-25T14:52:23.079263+0000 (oldest deadline 2024-04-25T14:52:44.779289+0000) 2024-04-25T14:52:50.210 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:49 smithi150 podman[69034]: 2024-04-25 14:52:49.911302548 +0000 UTC m=+0.578407225 container remove 873cf903b4e57072b3d3a1516bb90d209febe08e33e71afd4c1316b134d6fe43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:52:50.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:49 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:49.866+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:50.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:49 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:49.866+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:52:50.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:49 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:49.866+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:04.536360+0000 front 2024-04-25T14:52:04.536471+0000 (oldest deadline 2024-04-25T14:52:29.236751+0000) 2024-04-25T14:52:50.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:49 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:49.866+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:13.339516+0000 front 2024-04-25T14:52:13.339648+0000 (oldest deadline 2024-04-25T14:52:35.039780+0000) 2024-04-25T14:52:50.522 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:50 smithi150 podman[69125]: 2024-04-25 14:52:50.114565994 +0000 UTC m=+0.018267987 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T14:52:50.523 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:50 smithi150 podman[69125]: 2024-04-25 14:52:50.228676111 +0000 UTC m=+0.132378076 container create 45d2626eefccc024a85b5a555c88521f707933de509184e5920084fd7c610da7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T14:52:50.523 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:50 smithi150 podman[69125]: 2024-04-25 14:52:50.439268111 +0000 UTC m=+0.342970077 container init 45d2626eefccc024a85b5a555c88521f707933de509184e5920084fd7c610da7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T14:52:50.523 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:50 smithi150 podman[69125]: 2024-04-25 14:52:50.442445159 +0000 UTC m=+0.346147128 container start 45d2626eefccc024a85b5a555c88521f707933de509184e5920084fd7c610da7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:52:50.523 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:50 smithi150 podman[69125]: 2024-04-25 14:52:50.522549209 +0000 UTC m=+0.426251193 container attach 45d2626eefccc024a85b5a555c88521f707933de509184e5920084fd7c610da7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T14:52:50.523 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:50 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:50.264+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:50.523 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:50 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:50.264+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:52:50.524 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:50 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:50.264+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:52:50.524 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:50 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:50.264+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:11.873721+0000 front 2024-04-25T14:52:11.874007+0000 (oldest deadline 2024-04-25T14:52:37.073925+0000) 2024-04-25T14:52:50.524 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:50 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:50.264+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:19.374670+0000 front 2024-04-25T14:52:19.374695+0000 (oldest deadline 2024-04-25T14:52:44.674874+0000) 2024-04-25T14:52:50.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:50 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:50.234+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:50.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:50 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:50.234+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:52:50.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:50 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:50.234+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:52:50.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:50 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:50.234+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:22.079400+0000 front 2024-04-25T14:52:22.079475+0000 (oldest deadline 2024-04-25T14:52:44.379400+0000) 2024-04-25T14:52:50.961 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:50 smithi150 podman[69125]: 2024-04-25 14:52:50.734337194 +0000 UTC m=+0.638039178 container died 45d2626eefccc024a85b5a555c88521f707933de509184e5920084fd7c610da7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0) 2024-04-25T14:52:51.068 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:50 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:50.817+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:51.068 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:50 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:50.817+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:52:51.068 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:50 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:50.817+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:04.536360+0000 front 2024-04-25T14:52:04.536471+0000 (oldest deadline 2024-04-25T14:52:29.236751+0000) 2024-04-25T14:52:51.068 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:50 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:50.817+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:13.339516+0000 front 2024-04-25T14:52:13.339648+0000 (oldest deadline 2024-04-25T14:52:35.039780+0000) 2024-04-25T14:52:51.204 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:50 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:50.864+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.275170+0000 front 2024-04-25T14:51:56.275046+0000 (oldest deadline 2024-04-25T14:52:22.174939+0000) 2024-04-25T14:52:51.205 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:50 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:50.864+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.675809+0000 front 2024-04-25T14:52:05.675880+0000 (oldest deadline 2024-04-25T14:52:29.775996+0000) 2024-04-25T14:52:51.205 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:50 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:50.864+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:12.077368+0000 front 2024-04-25T14:52:12.077727+0000 (oldest deadline 2024-04-25T14:52:36.777146+0000) 2024-04-25T14:52:51.205 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:50 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:50.864+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:23.079358+0000 front 2024-04-25T14:52:23.079263+0000 (oldest deadline 2024-04-25T14:52:44.779289+0000) 2024-04-25T14:52:51.254 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@osd.7.service' 2024-04-25T14:52:51.277 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:51 smithi150 podman[69151]: 2024-04-25 14:52:51.22609912 +0000 UTC m=+0.486329431 container remove 45d2626eefccc024a85b5a555c88521f707933de509184e5920084fd7c610da7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T14:52:51.277 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:51 smithi150 systemd[1]: ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@osd.7.service: Deactivated successfully. 2024-04-25T14:52:51.277 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:51 smithi150 systemd[1]: Stopped Ceph osd.7 for fd30e6ce-030f-11ef-bc93-c7b262605968. 2024-04-25T14:52:51.277 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 14:52:51 smithi150 systemd[1]: ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@osd.7.service: Consumed 7.885s CPU time. 2024-04-25T14:52:51.294 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:52:51.294 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-25T14:52:51.295 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-25T14:52:51.295 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@osd.8 2024-04-25T14:52:51.578 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:51 smithi183 systemd[1]: Stopping Ceph osd.8 for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:52:51.578 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:51 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:51.208+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:51.579 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:51 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:51.208+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:52:51.579 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:51 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:51.208+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:52:51.579 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:51 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:51.208+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:22.079400+0000 front 2024-04-25T14:52:22.079475+0000 (oldest deadline 2024-04-25T14:52:44.379400+0000) 2024-04-25T14:52:51.670 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:51 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:51.296+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:51.670 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:51 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:51.296+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:52:51.670 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:51 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:51.296+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:52:51.670 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:51 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:51.296+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:11.873721+0000 front 2024-04-25T14:52:11.874007+0000 (oldest deadline 2024-04-25T14:52:37.073925+0000) 2024-04-25T14:52:51.670 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:51 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:51.296+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:19.374670+0000 front 2024-04-25T14:52:19.374695+0000 (oldest deadline 2024-04-25T14:52:44.674874+0000) 2024-04-25T14:52:51.866 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:51 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:51.576+0000 7f2ec2ff8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:52:51.866 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:51 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:51.576+0000 7f2ec2ff8700 -1 osd.8 106 *** Got signal Terminated *** 2024-04-25T14:52:51.866 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:51 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:51.576+0000 7f2ec2ff8700 -1 osd.8 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:52:51.867 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:51 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:51.864+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.275170+0000 front 2024-04-25T14:51:56.275046+0000 (oldest deadline 2024-04-25T14:52:22.174939+0000) 2024-04-25T14:52:51.867 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:51 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:51.864+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.675809+0000 front 2024-04-25T14:52:05.675880+0000 (oldest deadline 2024-04-25T14:52:29.775996+0000) 2024-04-25T14:52:51.867 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:51 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:51.864+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:12.077368+0000 front 2024-04-25T14:52:12.077727+0000 (oldest deadline 2024-04-25T14:52:36.777146+0000) 2024-04-25T14:52:51.867 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:51 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:51.864+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:23.079358+0000 front 2024-04-25T14:52:23.079263+0000 (oldest deadline 2024-04-25T14:52:44.779289+0000) 2024-04-25T14:52:52.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:51 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:51.835+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:52.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:51 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:51.835+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:52:52.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:51 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:51.835+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:04.536360+0000 front 2024-04-25T14:52:04.536471+0000 (oldest deadline 2024-04-25T14:52:29.236751+0000) 2024-04-25T14:52:52.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:51 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:51.835+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:13.339516+0000 front 2024-04-25T14:52:13.339648+0000 (oldest deadline 2024-04-25T14:52:35.039780+0000) 2024-04-25T14:52:52.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:52 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:52.204+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:52.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:52 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:52.204+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:52:52.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:52 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:52.204+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:52:52.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:52 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:52.204+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:22.079400+0000 front 2024-04-25T14:52:22.079475+0000 (oldest deadline 2024-04-25T14:52:44.379400+0000) 2024-04-25T14:52:52.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:52 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:52.258+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:52.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:52 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:52.258+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:52:52.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:52 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:52.258+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:52:52.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:52 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:52.258+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:11.873721+0000 front 2024-04-25T14:52:11.874007+0000 (oldest deadline 2024-04-25T14:52:37.073925+0000) 2024-04-25T14:52:52.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:52 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:52.258+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:19.374670+0000 front 2024-04-25T14:52:19.374695+0000 (oldest deadline 2024-04-25T14:52:44.674874+0000) 2024-04-25T14:52:53.204 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:52 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:52.849+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.275170+0000 front 2024-04-25T14:51:56.275046+0000 (oldest deadline 2024-04-25T14:52:22.174939+0000) 2024-04-25T14:52:53.204 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:52 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:52.849+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.675809+0000 front 2024-04-25T14:52:05.675880+0000 (oldest deadline 2024-04-25T14:52:29.775996+0000) 2024-04-25T14:52:53.205 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:52 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:52.849+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:12.077368+0000 front 2024-04-25T14:52:12.077727+0000 (oldest deadline 2024-04-25T14:52:36.777146+0000) 2024-04-25T14:52:53.205 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:52 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:52.849+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:23.079358+0000 front 2024-04-25T14:52:23.079263+0000 (oldest deadline 2024-04-25T14:52:44.779289+0000) 2024-04-25T14:52:53.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:52 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:52.868+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:53.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:52 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:52.868+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:52:53.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:52 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:52.868+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:04.536360+0000 front 2024-04-25T14:52:04.536471+0000 (oldest deadline 2024-04-25T14:52:29.236751+0000) 2024-04-25T14:52:53.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:52 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:52.868+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:13.339516+0000 front 2024-04-25T14:52:13.339648+0000 (oldest deadline 2024-04-25T14:52:35.039780+0000) 2024-04-25T14:52:53.613 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:53 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:53.268+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:53.613 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:53 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:53.268+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:52:53.613 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:53 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:53.268+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:52:53.613 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:53 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:53.268+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:11.873721+0000 front 2024-04-25T14:52:11.874007+0000 (oldest deadline 2024-04-25T14:52:37.073925+0000) 2024-04-25T14:52:53.613 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:53 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:53.268+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:19.374670+0000 front 2024-04-25T14:52:19.374695+0000 (oldest deadline 2024-04-25T14:52:44.674874+0000) 2024-04-25T14:52:53.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:53 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:53.252+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:53.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:53 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:53.252+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:52:53.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:53 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:53.252+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:52:53.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:53 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:53.252+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:22.079400+0000 front 2024-04-25T14:52:22.079475+0000 (oldest deadline 2024-04-25T14:52:44.379400+0000) 2024-04-25T14:52:53.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:53 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:53.252+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:27.181013+0000 front 2024-04-25T14:52:27.181032+0000 (oldest deadline 2024-04-25T14:52:52.481109+0000) 2024-04-25T14:52:54.204 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:53 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:53.898+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.275170+0000 front 2024-04-25T14:51:56.275046+0000 (oldest deadline 2024-04-25T14:52:22.174939+0000) 2024-04-25T14:52:54.204 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:53 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:53.898+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.675809+0000 front 2024-04-25T14:52:05.675880+0000 (oldest deadline 2024-04-25T14:52:29.775996+0000) 2024-04-25T14:52:54.205 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:53 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:53.898+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:12.077368+0000 front 2024-04-25T14:52:12.077727+0000 (oldest deadline 2024-04-25T14:52:36.777146+0000) 2024-04-25T14:52:54.205 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:53 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:53.898+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:23.079358+0000 front 2024-04-25T14:52:23.079263+0000 (oldest deadline 2024-04-25T14:52:44.779289+0000) 2024-04-25T14:52:54.205 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:53 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:53.898+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:29.380789+0000 front 2024-04-25T14:52:29.380771+0000 (oldest deadline 2024-04-25T14:52:52.880876+0000) 2024-04-25T14:52:54.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:53 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:53.827+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:54.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:53 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:53.827+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:52:54.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:53 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:53.827+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:04.536360+0000 front 2024-04-25T14:52:04.536471+0000 (oldest deadline 2024-04-25T14:52:29.236751+0000) 2024-04-25T14:52:54.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:53 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:53.827+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:13.339516+0000 front 2024-04-25T14:52:13.339648+0000 (oldest deadline 2024-04-25T14:52:35.039780+0000) 2024-04-25T14:52:54.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:53 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:53.827+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:30.645293+0000 front 2024-04-25T14:52:30.645365+0000 (oldest deadline 2024-04-25T14:52:52.945094+0000) 2024-04-25T14:52:54.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:54 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:54.219+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:54.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:54 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:54.219+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:52:54.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:54 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:54.219+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:52:54.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:54 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:54.219+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:22.079400+0000 front 2024-04-25T14:52:22.079475+0000 (oldest deadline 2024-04-25T14:52:44.379400+0000) 2024-04-25T14:52:54.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:54 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:54.219+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:27.181013+0000 front 2024-04-25T14:52:27.181032+0000 (oldest deadline 2024-04-25T14:52:52.481109+0000) 2024-04-25T14:52:54.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:54 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:54.287+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:54.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:54 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:54.287+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:52:54.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:54 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:54.287+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:52:54.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:54 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:54.287+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:11.873721+0000 front 2024-04-25T14:52:11.874007+0000 (oldest deadline 2024-04-25T14:52:37.073925+0000) 2024-04-25T14:52:54.712 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:54 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:54.287+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:19.374670+0000 front 2024-04-25T14:52:19.374695+0000 (oldest deadline 2024-04-25T14:52:44.674874+0000) 2024-04-25T14:52:55.204 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:54 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:54.888+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.275170+0000 front 2024-04-25T14:51:56.275046+0000 (oldest deadline 2024-04-25T14:52:22.174939+0000) 2024-04-25T14:52:55.205 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:54 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:54.888+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.675809+0000 front 2024-04-25T14:52:05.675880+0000 (oldest deadline 2024-04-25T14:52:29.775996+0000) 2024-04-25T14:52:55.205 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:54 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:54.888+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:12.077368+0000 front 2024-04-25T14:52:12.077727+0000 (oldest deadline 2024-04-25T14:52:36.777146+0000) 2024-04-25T14:52:55.205 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:54 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:54.888+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:23.079358+0000 front 2024-04-25T14:52:23.079263+0000 (oldest deadline 2024-04-25T14:52:44.779289+0000) 2024-04-25T14:52:55.205 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:54 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:54.888+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:29.380789+0000 front 2024-04-25T14:52:29.380771+0000 (oldest deadline 2024-04-25T14:52:52.880876+0000) 2024-04-25T14:52:55.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:54 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:54.818+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:55.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:54 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:54.818+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:52:55.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:54 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:54.818+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:04.536360+0000 front 2024-04-25T14:52:04.536471+0000 (oldest deadline 2024-04-25T14:52:29.236751+0000) 2024-04-25T14:52:55.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:54 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:54.818+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:13.339516+0000 front 2024-04-25T14:52:13.339648+0000 (oldest deadline 2024-04-25T14:52:35.039780+0000) 2024-04-25T14:52:55.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:54 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:54.818+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:30.645293+0000 front 2024-04-25T14:52:30.645365+0000 (oldest deadline 2024-04-25T14:52:52.945094+0000) 2024-04-25T14:52:55.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:55 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:55.262+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:55.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:55 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:55.262+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:52:55.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:55 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:55.262+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:52:55.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:55 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:55.262+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:22.079400+0000 front 2024-04-25T14:52:22.079475+0000 (oldest deadline 2024-04-25T14:52:44.379400+0000) 2024-04-25T14:52:55.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:55 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:55.262+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:27.181013+0000 front 2024-04-25T14:52:27.181032+0000 (oldest deadline 2024-04-25T14:52:52.481109+0000) 2024-04-25T14:52:55.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:55 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:55.305+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:55.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:55 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:55.305+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:52:55.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:55 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:55.305+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:52:55.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:55 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:55.305+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:11.873721+0000 front 2024-04-25T14:52:11.874007+0000 (oldest deadline 2024-04-25T14:52:37.073925+0000) 2024-04-25T14:52:55.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:55 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:55.305+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:19.374670+0000 front 2024-04-25T14:52:19.374695+0000 (oldest deadline 2024-04-25T14:52:44.674874+0000) 2024-04-25T14:52:55.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:55 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:55.305+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:29.876186+0000 front 2024-04-25T14:52:29.876209+0000 (oldest deadline 2024-04-25T14:52:55.176063+0000) 2024-04-25T14:52:56.204 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:55 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:55.923+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:56.275170+0000 front 2024-04-25T14:51:56.275046+0000 (oldest deadline 2024-04-25T14:52:22.174939+0000) 2024-04-25T14:52:56.205 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:55 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:55.923+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.675809+0000 front 2024-04-25T14:52:05.675880+0000 (oldest deadline 2024-04-25T14:52:29.775996+0000) 2024-04-25T14:52:56.205 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:55 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:55.923+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:12.077368+0000 front 2024-04-25T14:52:12.077727+0000 (oldest deadline 2024-04-25T14:52:36.777146+0000) 2024-04-25T14:52:56.205 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:55 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:55.923+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:23.079358+0000 front 2024-04-25T14:52:23.079263+0000 (oldest deadline 2024-04-25T14:52:44.779289+0000) 2024-04-25T14:52:56.205 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:55 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8[50981]: 2024-04-25T14:52:55.923+0000 7f2ebb5e4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:29.380789+0000 front 2024-04-25T14:52:29.380771+0000 (oldest deadline 2024-04-25T14:52:52.880876+0000) 2024-04-25T14:52:56.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:55 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:55.796+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:56.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:55 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:55.796+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:52:56.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:55 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:55.796+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:04.536360+0000 front 2024-04-25T14:52:04.536471+0000 (oldest deadline 2024-04-25T14:52:29.236751+0000) 2024-04-25T14:52:56.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:55 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:55.796+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:13.339516+0000 front 2024-04-25T14:52:13.339648+0000 (oldest deadline 2024-04-25T14:52:35.039780+0000) 2024-04-25T14:52:56.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:55 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:55.796+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:30.645293+0000 front 2024-04-25T14:52:30.645365+0000 (oldest deadline 2024-04-25T14:52:52.945094+0000) 2024-04-25T14:52:56.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:56 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:56.306+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:56.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:56 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:56.306+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:52:56.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:56 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:56.306+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:52:56.706 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:56 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:56.306+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:22.079400+0000 front 2024-04-25T14:52:22.079475+0000 (oldest deadline 2024-04-25T14:52:44.379400+0000) 2024-04-25T14:52:56.706 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:56 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:56.306+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:27.181013+0000 front 2024-04-25T14:52:27.181032+0000 (oldest deadline 2024-04-25T14:52:52.481109+0000) 2024-04-25T14:52:56.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:56 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:56.319+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:56.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:56 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:56.319+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:52:56.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:56 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:56.319+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:52:56.712 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:56 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:56.319+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:11.873721+0000 front 2024-04-25T14:52:11.874007+0000 (oldest deadline 2024-04-25T14:52:37.073925+0000) 2024-04-25T14:52:56.712 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:56 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:56.319+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:19.374670+0000 front 2024-04-25T14:52:19.374695+0000 (oldest deadline 2024-04-25T14:52:44.674874+0000) 2024-04-25T14:52:56.712 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:56 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:56.319+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:29.876186+0000 front 2024-04-25T14:52:29.876209+0000 (oldest deadline 2024-04-25T14:52:55.176063+0000) 2024-04-25T14:52:57.178 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:56 smithi183 podman[68968]: 2024-04-25 14:52:56.920215795 +0000 UTC m=+5.438311296 container died caf9cd7c0459922478018f7bff7e75ebe762e2e8bba3fa689128f57d73e65c76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:52:57.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:56 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:56.823+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:57.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:56 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:56.823+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:52:57.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:56 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:56.823+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:04.536360+0000 front 2024-04-25T14:52:04.536471+0000 (oldest deadline 2024-04-25T14:52:29.236751+0000) 2024-04-25T14:52:57.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:56 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:56.823+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:13.339516+0000 front 2024-04-25T14:52:13.339648+0000 (oldest deadline 2024-04-25T14:52:35.039780+0000) 2024-04-25T14:52:57.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:56 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:56.823+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:30.645293+0000 front 2024-04-25T14:52:30.645365+0000 (oldest deadline 2024-04-25T14:52:52.945094+0000) 2024-04-25T14:52:57.454 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:57 smithi183 podman[68968]: 2024-04-25 14:52:57.260235518 +0000 UTC m=+5.778331058 container cleanup caf9cd7c0459922478018f7bff7e75ebe762e2e8bba3fa689128f57d73e65c76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-25T14:52:57.455 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:57 smithi183 bash[68968]: ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8 2024-04-25T14:52:57.455 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:57 smithi183 podman[68985]: 2024-04-25 14:52:57.428246993 +0000 UTC m=+0.510862664 container remove caf9cd7c0459922478018f7bff7e75ebe762e2e8bba3fa689128f57d73e65c76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:52:57.455 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:57 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:57.352+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:57.455 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:57 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:57.352+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:52:57.455 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:57 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:57.352+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:52:57.455 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:57 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:57.352+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:22.079400+0000 front 2024-04-25T14:52:22.079475+0000 (oldest deadline 2024-04-25T14:52:44.379400+0000) 2024-04-25T14:52:57.456 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:57 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:57.352+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:27.181013+0000 front 2024-04-25T14:52:27.181032+0000 (oldest deadline 2024-04-25T14:52:52.481109+0000) 2024-04-25T14:52:57.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:57 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:57.362+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:57.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:57 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:57.362+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:52:57.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:57 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:57.362+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:52:57.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:57 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:57.362+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:11.873721+0000 front 2024-04-25T14:52:11.874007+0000 (oldest deadline 2024-04-25T14:52:37.073925+0000) 2024-04-25T14:52:57.712 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:57 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:57.362+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:19.374670+0000 front 2024-04-25T14:52:19.374695+0000 (oldest deadline 2024-04-25T14:52:44.674874+0000) 2024-04-25T14:52:57.712 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:57 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:57.362+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:29.876186+0000 front 2024-04-25T14:52:29.876209+0000 (oldest deadline 2024-04-25T14:52:55.176063+0000) 2024-04-25T14:52:58.097 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:57 smithi183 podman[69075]: 2024-04-25 14:52:57.812236574 +0000 UTC m=+0.102476221 container create 5a70bc201f0fc0480247e6904fc9c8c818f68fd1bc5a748427f00ec066ced3b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-25T14:52:58.098 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:57 smithi183 podman[69075]: 2024-04-25 14:52:57.727965859 +0000 UTC m=+0.018205522 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T14:52:58.098 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:58 smithi183 podman[69075]: 2024-04-25 14:52:58.022729638 +0000 UTC m=+0.312969286 container init 5a70bc201f0fc0480247e6904fc9c8c818f68fd1bc5a748427f00ec066ced3b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-25T14:52:58.098 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:58 smithi183 podman[69075]: 2024-04-25 14:52:58.025746944 +0000 UTC m=+0.315986594 container start 5a70bc201f0fc0480247e6904fc9c8c818f68fd1bc5a748427f00ec066ced3b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T14:52:58.098 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:58 smithi183 podman[69075]: 2024-04-25 14:52:58.097549655 +0000 UTC m=+0.387789309 container attach 5a70bc201f0fc0480247e6904fc9c8c818f68fd1bc5a748427f00ec066ced3b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:52:58.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:57 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:57.860+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:58.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:57 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:57.860+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:52:58.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:57 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:57.860+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:04.536360+0000 front 2024-04-25T14:52:04.536471+0000 (oldest deadline 2024-04-25T14:52:29.236751+0000) 2024-04-25T14:52:58.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:57 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:57.860+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:13.339516+0000 front 2024-04-25T14:52:13.339648+0000 (oldest deadline 2024-04-25T14:52:35.039780+0000) 2024-04-25T14:52:58.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:57 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:57.860+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:30.645293+0000 front 2024-04-25T14:52:30.645365+0000 (oldest deadline 2024-04-25T14:52:52.945094+0000) 2024-04-25T14:52:58.367 INFO:journalctl@ceph.osd.8.smithi183.stdout:Apr 25 14:52:58 smithi183 podman[69075]: 2024-04-25 14:52:58.319563676 +0000 UTC m=+0.609803325 container died 5a70bc201f0fc0480247e6904fc9c8c818f68fd1bc5a748427f00ec066ced3b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True) 2024-04-25T14:52:58.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:58 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:58.365+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:58.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:58 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:58.365+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:52:58.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:58 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:58.365+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:52:58.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:58 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:58.365+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:22.079400+0000 front 2024-04-25T14:52:22.079475+0000 (oldest deadline 2024-04-25T14:52:44.379400+0000) 2024-04-25T14:52:58.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:58 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:58.365+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:27.181013+0000 front 2024-04-25T14:52:27.181032+0000 (oldest deadline 2024-04-25T14:52:52.481109+0000) 2024-04-25T14:52:58.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:58 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:58.340+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:58.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:58 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:58.340+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:52:58.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:58 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:58.340+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:52:58.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:58 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:58.340+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:11.873721+0000 front 2024-04-25T14:52:11.874007+0000 (oldest deadline 2024-04-25T14:52:37.073925+0000) 2024-04-25T14:52:58.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:58 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:58.340+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:19.374670+0000 front 2024-04-25T14:52:19.374695+0000 (oldest deadline 2024-04-25T14:52:44.674874+0000) 2024-04-25T14:52:58.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:58 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:58.340+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:29.876186+0000 front 2024-04-25T14:52:29.876209+0000 (oldest deadline 2024-04-25T14:52:55.176063+0000) 2024-04-25T14:52:58.886 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@osd.8.service' 2024-04-25T14:52:58.921 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:52:58.922 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-25T14:52:58.922 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-25T14:52:58.922 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@osd.9 2024-04-25T14:52:59.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:58 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:58.875+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:52:59.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:58 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:58.875+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:52:59.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:58 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:58.875+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:04.536360+0000 front 2024-04-25T14:52:04.536471+0000 (oldest deadline 2024-04-25T14:52:29.236751+0000) 2024-04-25T14:52:59.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:58 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:58.875+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:13.339516+0000 front 2024-04-25T14:52:13.339648+0000 (oldest deadline 2024-04-25T14:52:35.039780+0000) 2024-04-25T14:52:59.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:58 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:58.875+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:30.645293+0000 front 2024-04-25T14:52:30.645365+0000 (oldest deadline 2024-04-25T14:52:52.945094+0000) 2024-04-25T14:52:59.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:59 smithi053 systemd[1]: Stopping Ceph osd.9 for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:52:59.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:59 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:59.379+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:52:59.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:59 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:59.379+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:52:59.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:59 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:59.379+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:52:59.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:59 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:59.379+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:22.079400+0000 front 2024-04-25T14:52:22.079475+0000 (oldest deadline 2024-04-25T14:52:44.379400+0000) 2024-04-25T14:52:59.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:52:59 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:52:59.379+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:27.181013+0000 front 2024-04-25T14:52:27.181032+0000 (oldest deadline 2024-04-25T14:52:52.481109+0000) 2024-04-25T14:52:59.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:59 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:59.308+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:52:59.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:59 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:59.308+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:52:59.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:59 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:59.308+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:52:59.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:59 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:59.308+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:11.873721+0000 front 2024-04-25T14:52:11.874007+0000 (oldest deadline 2024-04-25T14:52:37.073925+0000) 2024-04-25T14:52:59.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:59 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:59.308+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:19.374670+0000 front 2024-04-25T14:52:19.374695+0000 (oldest deadline 2024-04-25T14:52:44.674874+0000) 2024-04-25T14:52:59.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:52:59 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:52:59.308+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:29.876186+0000 front 2024-04-25T14:52:29.876209+0000 (oldest deadline 2024-04-25T14:52:55.176063+0000) 2024-04-25T14:52:59.783 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:59 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:59.505+0000 7fcdd1800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:52:59.783 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:59 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:59.505+0000 7fcdd1800700 -1 osd.9 106 *** Got signal Terminated *** 2024-04-25T14:52:59.783 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:59 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:59.505+0000 7fcdd1800700 -1 osd.9 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:53:00.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:59 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:59.848+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:53:00.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:59 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:59.848+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:53:00.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:59 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:59.848+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:04.536360+0000 front 2024-04-25T14:52:04.536471+0000 (oldest deadline 2024-04-25T14:52:29.236751+0000) 2024-04-25T14:53:00.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:59 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:59.848+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:13.339516+0000 front 2024-04-25T14:52:13.339648+0000 (oldest deadline 2024-04-25T14:52:35.039780+0000) 2024-04-25T14:53:00.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:52:59 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:52:59.848+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:30.645293+0000 front 2024-04-25T14:52:30.645365+0000 (oldest deadline 2024-04-25T14:52:52.945094+0000) 2024-04-25T14:53:00.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:00 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:00.344+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:53:00.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:00 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:00.344+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:53:00.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:00 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:00.344+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:53:00.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:00 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:00.344+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:22.079400+0000 front 2024-04-25T14:52:22.079475+0000 (oldest deadline 2024-04-25T14:52:44.379400+0000) 2024-04-25T14:53:00.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:00 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:00.344+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:27.181013+0000 front 2024-04-25T14:52:27.181032+0000 (oldest deadline 2024-04-25T14:52:52.481109+0000) 2024-04-25T14:53:00.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:00 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:00.309+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:53:00.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:00 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:00.309+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:53:00.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:00 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:00.309+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:53:00.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:00 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:00.309+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:11.873721+0000 front 2024-04-25T14:52:11.874007+0000 (oldest deadline 2024-04-25T14:52:37.073925+0000) 2024-04-25T14:53:00.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:00 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:00.309+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:19.374670+0000 front 2024-04-25T14:52:19.374695+0000 (oldest deadline 2024-04-25T14:52:44.674874+0000) 2024-04-25T14:53:00.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:00 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:00.309+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:29.876186+0000 front 2024-04-25T14:52:29.876209+0000 (oldest deadline 2024-04-25T14:52:55.176063+0000) 2024-04-25T14:53:01.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:53:00 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:53:00.885+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:53:01.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:53:00 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:53:00.885+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:53:01.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:53:00 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:53:00.885+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:04.536360+0000 front 2024-04-25T14:52:04.536471+0000 (oldest deadline 2024-04-25T14:52:29.236751+0000) 2024-04-25T14:53:01.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:53:00 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:53:00.885+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:13.339516+0000 front 2024-04-25T14:52:13.339648+0000 (oldest deadline 2024-04-25T14:52:35.039780+0000) 2024-04-25T14:53:01.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:53:00 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:53:00.885+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:30.645293+0000 front 2024-04-25T14:52:30.645365+0000 (oldest deadline 2024-04-25T14:52:52.945094+0000) 2024-04-25T14:53:01.454 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:01 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:01.311+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:53:01.455 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:01 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:01.311+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:53:01.455 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:01 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:01.311+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:53:01.455 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:01 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:01.311+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:22.079400+0000 front 2024-04-25T14:52:22.079475+0000 (oldest deadline 2024-04-25T14:52:44.379400+0000) 2024-04-25T14:53:01.455 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:01 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:01.311+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:27.181013+0000 front 2024-04-25T14:52:27.181032+0000 (oldest deadline 2024-04-25T14:52:52.481109+0000) 2024-04-25T14:53:01.455 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:01 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:01.311+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2024-04-25T14:52:34.782054+0000 front 2024-04-25T14:52:34.782189+0000 (oldest deadline 2024-04-25T14:53:00.681787+0000) 2024-04-25T14:53:01.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:01 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:01.305+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:53:01.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:01 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:01.305+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:53:01.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:01 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:01.305+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:53:01.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:01 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:01.305+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:11.873721+0000 front 2024-04-25T14:52:11.874007+0000 (oldest deadline 2024-04-25T14:52:37.073925+0000) 2024-04-25T14:53:01.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:01 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:01.305+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:19.374670+0000 front 2024-04-25T14:52:19.374695+0000 (oldest deadline 2024-04-25T14:52:44.674874+0000) 2024-04-25T14:53:01.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:01 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:01.305+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:29.876186+0000 front 2024-04-25T14:52:29.876209+0000 (oldest deadline 2024-04-25T14:52:55.176063+0000) 2024-04-25T14:53:02.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:53:01 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:53:01.924+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:53:02.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:53:01 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:53:01.924+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:53:02.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:53:01 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:53:01.924+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:04.536360+0000 front 2024-04-25T14:52:04.536471+0000 (oldest deadline 2024-04-25T14:52:29.236751+0000) 2024-04-25T14:53:02.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:53:01 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:53:01.924+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:13.339516+0000 front 2024-04-25T14:52:13.339648+0000 (oldest deadline 2024-04-25T14:52:35.039780+0000) 2024-04-25T14:53:02.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:53:01 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:53:01.924+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:30.645293+0000 front 2024-04-25T14:52:30.645365+0000 (oldest deadline 2024-04-25T14:52:52.945094+0000) 2024-04-25T14:53:02.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:02 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:02.292+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:53:02.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:02 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:02.292+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:53:02.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:02 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:02.292+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:53:02.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:02 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:02.292+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:22.079400+0000 front 2024-04-25T14:52:22.079475+0000 (oldest deadline 2024-04-25T14:52:44.379400+0000) 2024-04-25T14:53:02.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:02 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:02.292+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:27.181013+0000 front 2024-04-25T14:52:27.181032+0000 (oldest deadline 2024-04-25T14:52:52.481109+0000) 2024-04-25T14:53:02.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:02 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:02.292+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2024-04-25T14:52:34.782054+0000 front 2024-04-25T14:52:34.782189+0000 (oldest deadline 2024-04-25T14:53:00.681787+0000) 2024-04-25T14:53:02.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:02 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:02.333+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:53:02.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:02 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:02.333+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:53:02.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:02 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:02.333+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:53:02.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:02 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:02.333+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:11.873721+0000 front 2024-04-25T14:52:11.874007+0000 (oldest deadline 2024-04-25T14:52:37.073925+0000) 2024-04-25T14:53:02.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:02 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:02.333+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:19.374670+0000 front 2024-04-25T14:52:19.374695+0000 (oldest deadline 2024-04-25T14:52:44.674874+0000) 2024-04-25T14:53:02.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:02 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:02.333+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:29.876186+0000 front 2024-04-25T14:52:29.876209+0000 (oldest deadline 2024-04-25T14:52:55.176063+0000) 2024-04-25T14:53:02.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:02 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:02.333+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2024-04-25T14:52:38.577631+0000 front 2024-04-25T14:52:38.577760+0000 (oldest deadline 2024-04-25T14:53:02.077439+0000) 2024-04-25T14:53:03.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:53:02 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:53:02.890+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:53:03.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:53:02 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:53:02.890+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:53:03.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:53:02 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:53:02.890+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:04.536360+0000 front 2024-04-25T14:52:04.536471+0000 (oldest deadline 2024-04-25T14:52:29.236751+0000) 2024-04-25T14:53:03.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:53:02 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:53:02.890+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:13.339516+0000 front 2024-04-25T14:52:13.339648+0000 (oldest deadline 2024-04-25T14:52:35.039780+0000) 2024-04-25T14:53:03.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:53:02 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:53:02.890+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:30.645293+0000 front 2024-04-25T14:52:30.645365+0000 (oldest deadline 2024-04-25T14:52:52.945094+0000) 2024-04-25T14:53:03.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:53:02 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:53:02.890+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2024-04-25T14:52:35.147881+0000 front 2024-04-25T14:52:35.147656+0000 (oldest deadline 2024-04-25T14:53:02.148062+0000) 2024-04-25T14:53:03.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:03 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:03.296+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:53:03.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:03 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:03.296+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:53:03.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:03 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:03.296+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:53:03.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:03 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:03.296+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:22.079400+0000 front 2024-04-25T14:52:22.079475+0000 (oldest deadline 2024-04-25T14:52:44.379400+0000) 2024-04-25T14:53:03.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:03 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:03.296+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:27.181013+0000 front 2024-04-25T14:52:27.181032+0000 (oldest deadline 2024-04-25T14:52:52.481109+0000) 2024-04-25T14:53:03.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:03 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:03.296+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2024-04-25T14:52:34.782054+0000 front 2024-04-25T14:52:34.782189+0000 (oldest deadline 2024-04-25T14:53:00.681787+0000) 2024-04-25T14:53:03.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:03 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:03.372+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:53:03.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:03 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:03.372+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:53:03.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:03 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:03.372+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:53:03.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:03 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:03.372+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:11.873721+0000 front 2024-04-25T14:52:11.874007+0000 (oldest deadline 2024-04-25T14:52:37.073925+0000) 2024-04-25T14:53:03.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:03 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:03.372+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:19.374670+0000 front 2024-04-25T14:52:19.374695+0000 (oldest deadline 2024-04-25T14:52:44.674874+0000) 2024-04-25T14:53:03.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:03 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:03.372+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:29.876186+0000 front 2024-04-25T14:52:29.876209+0000 (oldest deadline 2024-04-25T14:52:55.176063+0000) 2024-04-25T14:53:03.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:03 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:03.372+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2024-04-25T14:52:38.577631+0000 front 2024-04-25T14:52:38.577760+0000 (oldest deadline 2024-04-25T14:53:02.077439+0000) 2024-04-25T14:53:04.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:53:03 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:53:03.851+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:44.729251+0000 front 2024-04-25T14:51:44.729445+0000 (oldest deadline 2024-04-25T14:52:10.629620+0000) 2024-04-25T14:53:04.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:53:03 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:53:03.851+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:55.134155+0000 front 2024-04-25T14:51:55.134087+0000 (oldest deadline 2024-04-25T14:52:19.234319+0000) 2024-04-25T14:53:04.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:53:03 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:53:03.851+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:04.536360+0000 front 2024-04-25T14:52:04.536471+0000 (oldest deadline 2024-04-25T14:52:29.236751+0000) 2024-04-25T14:53:04.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:53:03 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:53:03.851+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:13.339516+0000 front 2024-04-25T14:52:13.339648+0000 (oldest deadline 2024-04-25T14:52:35.039780+0000) 2024-04-25T14:53:04.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:53:03 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:53:03.851+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:30.645293+0000 front 2024-04-25T14:52:30.645365+0000 (oldest deadline 2024-04-25T14:52:52.945094+0000) 2024-04-25T14:53:04.284 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:53:03 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9[62439]: 2024-04-25T14:53:03.851+0000 7fcdcd000700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2024-04-25T14:52:35.147881+0000 front 2024-04-25T14:52:35.147656+0000 (oldest deadline 2024-04-25T14:53:02.148062+0000) 2024-04-25T14:53:04.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:04 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:04.261+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:53:04.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:04 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:04.261+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:53:04.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:04 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:04.261+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:53:04.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:04 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:04.261+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:22.079400+0000 front 2024-04-25T14:52:22.079475+0000 (oldest deadline 2024-04-25T14:52:44.379400+0000) 2024-04-25T14:53:04.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:04 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:04.261+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:27.181013+0000 front 2024-04-25T14:52:27.181032+0000 (oldest deadline 2024-04-25T14:52:52.481109+0000) 2024-04-25T14:53:04.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:04 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:04.261+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2024-04-25T14:52:34.782054+0000 front 2024-04-25T14:52:34.782189+0000 (oldest deadline 2024-04-25T14:53:00.681787+0000) 2024-04-25T14:53:04.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:04 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:04.365+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:53:04.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:04 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:04.365+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:53:04.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:04 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:04.365+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:53:04.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:04 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:04.365+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:11.873721+0000 front 2024-04-25T14:52:11.874007+0000 (oldest deadline 2024-04-25T14:52:37.073925+0000) 2024-04-25T14:53:04.712 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:04 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:04.365+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:19.374670+0000 front 2024-04-25T14:52:19.374695+0000 (oldest deadline 2024-04-25T14:52:44.674874+0000) 2024-04-25T14:53:04.712 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:04 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:04.365+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:29.876186+0000 front 2024-04-25T14:52:29.876209+0000 (oldest deadline 2024-04-25T14:52:55.176063+0000) 2024-04-25T14:53:04.712 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:04 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:04.365+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2024-04-25T14:52:38.577631+0000 front 2024-04-25T14:52:38.577760+0000 (oldest deadline 2024-04-25T14:53:02.077439+0000) 2024-04-25T14:53:05.464 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:53:05 smithi053 podman[86893]: 2024-04-25 14:53:05.035760635 +0000 UTC m=+5.670907555 container died 6d5d402ef1e22a4456ae31be8f29c6c966581840087e32d387790d9513f987cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-25T14:53:05.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:05 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:05.245+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:53:05.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:05 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:05.245+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:53:05.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:05 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:05.245+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:53:05.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:05 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:05.245+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:22.079400+0000 front 2024-04-25T14:52:22.079475+0000 (oldest deadline 2024-04-25T14:52:44.379400+0000) 2024-04-25T14:53:05.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:05 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:05.245+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:27.181013+0000 front 2024-04-25T14:52:27.181032+0000 (oldest deadline 2024-04-25T14:52:52.481109+0000) 2024-04-25T14:53:05.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:05 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:05.245+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2024-04-25T14:52:34.782054+0000 front 2024-04-25T14:52:34.782189+0000 (oldest deadline 2024-04-25T14:53:00.681787+0000) 2024-04-25T14:53:05.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:05 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:05.383+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:53:05.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:05 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:05.383+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:53:05.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:05 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:05.383+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:53:05.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:05 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:05.383+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:11.873721+0000 front 2024-04-25T14:52:11.874007+0000 (oldest deadline 2024-04-25T14:52:37.073925+0000) 2024-04-25T14:53:05.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:05 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:05.383+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:19.374670+0000 front 2024-04-25T14:52:19.374695+0000 (oldest deadline 2024-04-25T14:52:44.674874+0000) 2024-04-25T14:53:05.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:05 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:05.383+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:29.876186+0000 front 2024-04-25T14:52:29.876209+0000 (oldest deadline 2024-04-25T14:52:55.176063+0000) 2024-04-25T14:53:05.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:05 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:05.383+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2024-04-25T14:52:38.577631+0000 front 2024-04-25T14:52:38.577760+0000 (oldest deadline 2024-04-25T14:53:02.077439+0000) 2024-04-25T14:53:05.783 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:53:05 smithi053 podman[86893]: 2024-04-25 14:53:05.556352356 +0000 UTC m=+6.191499300 container cleanup 6d5d402ef1e22a4456ae31be8f29c6c966581840087e32d387790d9513f987cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True) 2024-04-25T14:53:05.783 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:53:05 smithi053 bash[86893]: ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9 2024-04-25T14:53:06.282 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:53:05 smithi053 podman[86911]: 2024-04-25 14:53:05.88267022 +0000 UTC m=+0.834079533 container remove 6d5d402ef1e22a4456ae31be8f29c6c966581840087e32d387790d9513f987cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T14:53:06.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:06 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:06.275+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:53:06.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:06 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:06.275+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:53:06.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:06 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:06.275+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:53:06.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:06 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:06.275+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:22.079400+0000 front 2024-04-25T14:52:22.079475+0000 (oldest deadline 2024-04-25T14:52:44.379400+0000) 2024-04-25T14:53:06.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:06 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:06.275+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:27.181013+0000 front 2024-04-25T14:52:27.181032+0000 (oldest deadline 2024-04-25T14:52:52.481109+0000) 2024-04-25T14:53:06.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:06 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:06.275+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2024-04-25T14:52:34.782054+0000 front 2024-04-25T14:52:34.782189+0000 (oldest deadline 2024-04-25T14:53:00.681787+0000) 2024-04-25T14:53:06.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:06 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:06.369+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:53:06.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:06 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:06.369+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:53:06.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:06 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:06.369+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:53:06.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:06 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:06.369+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:11.873721+0000 front 2024-04-25T14:52:11.874007+0000 (oldest deadline 2024-04-25T14:52:37.073925+0000) 2024-04-25T14:53:06.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:06 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:06.369+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:19.374670+0000 front 2024-04-25T14:52:19.374695+0000 (oldest deadline 2024-04-25T14:52:44.674874+0000) 2024-04-25T14:53:06.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:06 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:06.369+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:29.876186+0000 front 2024-04-25T14:52:29.876209+0000 (oldest deadline 2024-04-25T14:52:55.176063+0000) 2024-04-25T14:53:06.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:06 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:06.369+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2024-04-25T14:52:38.577631+0000 front 2024-04-25T14:52:38.577760+0000 (oldest deadline 2024-04-25T14:53:02.077439+0000) 2024-04-25T14:53:07.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:53:06 smithi053 podman[86999]: 2024-04-25 14:53:06.764354142 +0000 UTC m=+0.043412594 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T14:53:07.283 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:53:06 smithi053 podman[86999]: 2024-04-25 14:53:06.883381416 +0000 UTC m=+0.162439890 container create bcc36cfa57a4481c76d0c0ec63940f5f1fddd2110b2933dcbac146e8009000c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-25T14:53:07.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:07 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:07.262+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:53:07.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:07 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:07.262+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:53:07.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:07 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:07.262+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:53:07.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:07 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:07.262+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:22.079400+0000 front 2024-04-25T14:52:22.079475+0000 (oldest deadline 2024-04-25T14:52:44.379400+0000) 2024-04-25T14:53:07.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:07 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:07.262+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:27.181013+0000 front 2024-04-25T14:52:27.181032+0000 (oldest deadline 2024-04-25T14:52:52.481109+0000) 2024-04-25T14:53:07.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:07 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:07.262+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2024-04-25T14:52:34.782054+0000 front 2024-04-25T14:52:34.782189+0000 (oldest deadline 2024-04-25T14:53:00.681787+0000) 2024-04-25T14:53:07.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:07 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:07.410+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:53:07.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:07 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:07.410+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:53:07.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:07 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:07.410+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:53:07.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:07 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:07.410+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:11.873721+0000 front 2024-04-25T14:52:11.874007+0000 (oldest deadline 2024-04-25T14:52:37.073925+0000) 2024-04-25T14:53:07.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:07 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:07.410+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:19.374670+0000 front 2024-04-25T14:52:19.374695+0000 (oldest deadline 2024-04-25T14:52:44.674874+0000) 2024-04-25T14:53:07.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:07 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:07.410+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:29.876186+0000 front 2024-04-25T14:52:29.876209+0000 (oldest deadline 2024-04-25T14:52:55.176063+0000) 2024-04-25T14:53:07.712 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:07 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:07.410+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2024-04-25T14:52:38.577631+0000 front 2024-04-25T14:52:38.577760+0000 (oldest deadline 2024-04-25T14:53:02.077439+0000) 2024-04-25T14:53:07.742 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:53:07 smithi053 podman[86999]: 2024-04-25 14:53:07.311383003 +0000 UTC m=+0.590441454 container init bcc36cfa57a4481c76d0c0ec63940f5f1fddd2110b2933dcbac146e8009000c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-25T14:53:07.742 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:53:07 smithi053 podman[86999]: 2024-04-25 14:53:07.322741685 +0000 UTC m=+0.601800144 container start bcc36cfa57a4481c76d0c0ec63940f5f1fddd2110b2933dcbac146e8009000c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:53:07.742 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:53:07 smithi053 podman[86999]: 2024-04-25 14:53:07.3939067 +0000 UTC m=+0.672965160 container attach bcc36cfa57a4481c76d0c0ec63940f5f1fddd2110b2933dcbac146e8009000c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0) 2024-04-25T14:53:08.033 INFO:journalctl@ceph.osd.9.smithi053.stdout:Apr 25 14:53:07 smithi053 podman[86999]: 2024-04-25 14:53:07.794279866 +0000 UTC m=+1.073338389 container died bcc36cfa57a4481c76d0c0ec63940f5f1fddd2110b2933dcbac146e8009000c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-25T14:53:08.515 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@osd.9.service' 2024-04-25T14:53:08.619 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:53:08.619 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-25T14:53:08.620 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-25T14:53:08.620 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@osd.10 2024-04-25T14:53:08.647 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:08 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:08.361+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:53:08.648 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:08 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:08.361+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:53:08.648 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:08 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:08.361+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:53:08.648 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:08 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:08.361+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:11.873721+0000 front 2024-04-25T14:52:11.874007+0000 (oldest deadline 2024-04-25T14:52:37.073925+0000) 2024-04-25T14:53:08.648 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:08 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:08.361+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:19.374670+0000 front 2024-04-25T14:52:19.374695+0000 (oldest deadline 2024-04-25T14:52:44.674874+0000) 2024-04-25T14:53:08.648 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:08 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:08.361+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:29.876186+0000 front 2024-04-25T14:52:29.876209+0000 (oldest deadline 2024-04-25T14:52:55.176063+0000) 2024-04-25T14:53:08.648 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:08 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:08.361+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2024-04-25T14:52:38.577631+0000 front 2024-04-25T14:52:38.577760+0000 (oldest deadline 2024-04-25T14:53:02.077439+0000) 2024-04-25T14:53:08.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:08 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:08.239+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:53:08.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:08 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:08.239+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:53:08.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:08 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:08.239+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:53:08.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:08 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:08.239+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:22.079400+0000 front 2024-04-25T14:52:22.079475+0000 (oldest deadline 2024-04-25T14:52:44.379400+0000) 2024-04-25T14:53:08.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:08 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:08.239+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:27.181013+0000 front 2024-04-25T14:52:27.181032+0000 (oldest deadline 2024-04-25T14:52:52.481109+0000) 2024-04-25T14:53:08.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:08 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:08.239+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2024-04-25T14:52:34.782054+0000 front 2024-04-25T14:52:34.782189+0000 (oldest deadline 2024-04-25T14:53:00.681787+0000) 2024-04-25T14:53:08.929 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:08 smithi150 systemd[1]: Stopping Ceph osd.10 for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:53:09.211 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:08 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:08.928+0000 7fbd73290700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:53:09.211 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:08 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:08.928+0000 7fbd73290700 -1 osd.10 106 *** Got signal Terminated *** 2024-04-25T14:53:09.211 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:08 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:08.928+0000 7fbd73290700 -1 osd.10 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:53:09.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:09 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:09.206+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:53:09.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:09 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:09.206+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:53:09.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:09 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:09.206+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:53:09.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:09 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:09.206+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:22.079400+0000 front 2024-04-25T14:52:22.079475+0000 (oldest deadline 2024-04-25T14:52:44.379400+0000) 2024-04-25T14:53:09.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:09 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:09.206+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:27.181013+0000 front 2024-04-25T14:52:27.181032+0000 (oldest deadline 2024-04-25T14:52:52.481109+0000) 2024-04-25T14:53:09.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:09 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:09.206+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2024-04-25T14:52:34.782054+0000 front 2024-04-25T14:52:34.782189+0000 (oldest deadline 2024-04-25T14:53:00.681787+0000) 2024-04-25T14:53:09.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:09 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:09.370+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:53:09.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:09 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:09.370+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:53:09.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:09 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:09.370+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:53:09.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:09 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:09.370+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:11.873721+0000 front 2024-04-25T14:52:11.874007+0000 (oldest deadline 2024-04-25T14:52:37.073925+0000) 2024-04-25T14:53:09.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:09 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:09.370+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:19.374670+0000 front 2024-04-25T14:52:19.374695+0000 (oldest deadline 2024-04-25T14:52:44.674874+0000) 2024-04-25T14:53:09.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:09 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:09.370+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:29.876186+0000 front 2024-04-25T14:52:29.876209+0000 (oldest deadline 2024-04-25T14:52:55.176063+0000) 2024-04-25T14:53:09.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:09 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:09.370+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2024-04-25T14:52:38.577631+0000 front 2024-04-25T14:52:38.577760+0000 (oldest deadline 2024-04-25T14:53:02.077439+0000) 2024-04-25T14:53:10.454 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:10 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:10.184+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:53:10.455 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:10 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:10.184+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:53:10.455 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:10 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:10.184+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:53:10.455 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:10 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:10.184+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:22.079400+0000 front 2024-04-25T14:52:22.079475+0000 (oldest deadline 2024-04-25T14:52:44.379400+0000) 2024-04-25T14:53:10.455 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:10 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:10.184+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:27.181013+0000 front 2024-04-25T14:52:27.181032+0000 (oldest deadline 2024-04-25T14:52:52.481109+0000) 2024-04-25T14:53:10.455 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:10 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:10.184+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2024-04-25T14:52:34.782054+0000 front 2024-04-25T14:52:34.782189+0000 (oldest deadline 2024-04-25T14:53:00.681787+0000) 2024-04-25T14:53:10.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:10 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:10.378+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:53:10.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:10 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:10.378+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:53:10.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:10 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:10.378+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:53:10.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:10 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:10.378+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:11.873721+0000 front 2024-04-25T14:52:11.874007+0000 (oldest deadline 2024-04-25T14:52:37.073925+0000) 2024-04-25T14:53:10.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:10 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:10.378+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:19.374670+0000 front 2024-04-25T14:52:19.374695+0000 (oldest deadline 2024-04-25T14:52:44.674874+0000) 2024-04-25T14:53:10.712 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:10 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:10.378+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:29.876186+0000 front 2024-04-25T14:52:29.876209+0000 (oldest deadline 2024-04-25T14:52:55.176063+0000) 2024-04-25T14:53:10.712 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:10 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:10.378+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2024-04-25T14:52:38.577631+0000 front 2024-04-25T14:52:38.577760+0000 (oldest deadline 2024-04-25T14:53:02.077439+0000) 2024-04-25T14:53:11.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:11 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:11.228+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:53:11.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:11 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:11.228+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:53:11.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:11 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:11.228+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:53:11.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:11 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:11.228+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:22.079400+0000 front 2024-04-25T14:52:22.079475+0000 (oldest deadline 2024-04-25T14:52:44.379400+0000) 2024-04-25T14:53:11.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:11 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:11.228+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:27.181013+0000 front 2024-04-25T14:52:27.181032+0000 (oldest deadline 2024-04-25T14:52:52.481109+0000) 2024-04-25T14:53:11.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:11 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:11.228+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2024-04-25T14:52:34.782054+0000 front 2024-04-25T14:52:34.782189+0000 (oldest deadline 2024-04-25T14:53:00.681787+0000) 2024-04-25T14:53:11.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:11 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:11.228+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-04-25T14:52:45.982780+0000 front 2024-04-25T14:52:45.982720+0000 (oldest deadline 2024-04-25T14:53:10.682715+0000) 2024-04-25T14:53:11.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:11 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:11.356+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:53:11.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:11 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:11.356+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:53:11.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:11 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:11.356+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:53:11.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:11 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:11.356+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:11.873721+0000 front 2024-04-25T14:52:11.874007+0000 (oldest deadline 2024-04-25T14:52:37.073925+0000) 2024-04-25T14:53:11.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:11 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:11.356+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:19.374670+0000 front 2024-04-25T14:52:19.374695+0000 (oldest deadline 2024-04-25T14:52:44.674874+0000) 2024-04-25T14:53:11.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:11 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:11.356+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:29.876186+0000 front 2024-04-25T14:52:29.876209+0000 (oldest deadline 2024-04-25T14:52:55.176063+0000) 2024-04-25T14:53:11.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:11 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:11.356+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2024-04-25T14:52:38.577631+0000 front 2024-04-25T14:52:38.577760+0000 (oldest deadline 2024-04-25T14:53:02.077439+0000) 2024-04-25T14:53:12.507 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:12 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:12.210+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:53:12.507 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:12 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:12.210+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:53:12.508 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:12 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:12.210+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:53:12.508 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:12 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:12.210+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:22.079400+0000 front 2024-04-25T14:52:22.079475+0000 (oldest deadline 2024-04-25T14:52:44.379400+0000) 2024-04-25T14:53:12.508 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:12 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:12.210+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:27.181013+0000 front 2024-04-25T14:52:27.181032+0000 (oldest deadline 2024-04-25T14:52:52.481109+0000) 2024-04-25T14:53:12.508 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:12 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:12.210+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2024-04-25T14:52:34.782054+0000 front 2024-04-25T14:52:34.782189+0000 (oldest deadline 2024-04-25T14:53:00.681787+0000) 2024-04-25T14:53:12.508 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:12 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:12.210+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-04-25T14:52:45.982780+0000 front 2024-04-25T14:52:45.982720+0000 (oldest deadline 2024-04-25T14:53:10.682715+0000) 2024-04-25T14:53:12.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:12 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:12.366+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:53:12.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:12 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:12.366+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:53:12.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:12 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:12.366+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:53:12.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:12 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:12.366+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:11.873721+0000 front 2024-04-25T14:52:11.874007+0000 (oldest deadline 2024-04-25T14:52:37.073925+0000) 2024-04-25T14:53:12.712 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:12 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:12.366+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:19.374670+0000 front 2024-04-25T14:52:19.374695+0000 (oldest deadline 2024-04-25T14:52:44.674874+0000) 2024-04-25T14:53:12.712 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:12 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:12.366+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:29.876186+0000 front 2024-04-25T14:52:29.876209+0000 (oldest deadline 2024-04-25T14:52:55.176063+0000) 2024-04-25T14:53:12.712 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:12 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:12.366+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2024-04-25T14:52:38.577631+0000 front 2024-04-25T14:52:38.577760+0000 (oldest deadline 2024-04-25T14:53:02.077439+0000) 2024-04-25T14:53:13.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:13 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:13.233+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:53:13.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:13 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:13.233+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:53:13.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:13 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:13.233+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:53:13.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:13 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:13.233+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:22.079400+0000 front 2024-04-25T14:52:22.079475+0000 (oldest deadline 2024-04-25T14:52:44.379400+0000) 2024-04-25T14:53:13.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:13 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:13.233+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:27.181013+0000 front 2024-04-25T14:52:27.181032+0000 (oldest deadline 2024-04-25T14:52:52.481109+0000) 2024-04-25T14:53:13.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:13 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:13.233+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2024-04-25T14:52:34.782054+0000 front 2024-04-25T14:52:34.782189+0000 (oldest deadline 2024-04-25T14:53:00.681787+0000) 2024-04-25T14:53:13.706 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:13 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:13.233+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-04-25T14:52:45.982780+0000 front 2024-04-25T14:52:45.982720+0000 (oldest deadline 2024-04-25T14:53:10.682715+0000) 2024-04-25T14:53:13.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:13 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:13.337+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:47.771055+0000 front 2024-04-25T14:51:47.771179+0000 (oldest deadline 2024-04-25T14:52:13.670906+0000) 2024-04-25T14:53:13.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:13 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:13.337+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:53.671237+0000 front 2024-04-25T14:51:53.671340+0000 (oldest deadline 2024-04-25T14:52:18.971444+0000) 2024-04-25T14:53:13.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:13 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:13.337+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:01.272701+0000 front 2024-04-25T14:52:01.272613+0000 (oldest deadline 2024-04-25T14:52:27.172495+0000) 2024-04-25T14:53:13.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:13 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:13.337+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6814 osd.3 since back 2024-04-25T14:52:11.873721+0000 front 2024-04-25T14:52:11.874007+0000 (oldest deadline 2024-04-25T14:52:37.073925+0000) 2024-04-25T14:53:13.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:13 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:13.337+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:19.374670+0000 front 2024-04-25T14:52:19.374695+0000 (oldest deadline 2024-04-25T14:52:44.674874+0000) 2024-04-25T14:53:13.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:13 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:13.337+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:29.876186+0000 front 2024-04-25T14:52:29.876209+0000 (oldest deadline 2024-04-25T14:52:55.176063+0000) 2024-04-25T14:53:13.711 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:13 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10[57237]: 2024-04-25T14:53:13.337+0000 7fbd6f884700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2024-04-25T14:52:38.577631+0000 front 2024-04-25T14:52:38.577760+0000 (oldest deadline 2024-04-25T14:53:02.077439+0000) 2024-04-25T14:53:14.454 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:14 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:14.199+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:53:14.455 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:14 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:14.199+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:53:14.455 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:14 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:14.199+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:53:14.455 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:14 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:14.199+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:22.079400+0000 front 2024-04-25T14:52:22.079475+0000 (oldest deadline 2024-04-25T14:52:44.379400+0000) 2024-04-25T14:53:14.455 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:14 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:14.199+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:27.181013+0000 front 2024-04-25T14:52:27.181032+0000 (oldest deadline 2024-04-25T14:52:52.481109+0000) 2024-04-25T14:53:14.455 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:14 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:14.199+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2024-04-25T14:52:34.782054+0000 front 2024-04-25T14:52:34.782189+0000 (oldest deadline 2024-04-25T14:53:00.681787+0000) 2024-04-25T14:53:14.455 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:14 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:14.199+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-04-25T14:52:45.982780+0000 front 2024-04-25T14:52:45.982720+0000 (oldest deadline 2024-04-25T14:53:10.682715+0000) 2024-04-25T14:53:14.570 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:14 smithi150 podman[69242]: 2024-04-25 14:53:14.308293964 +0000 UTC m=+5.499972317 container died 23336e8514976117276c641a020f40412599bc8be1f5c311504e065039d1ca17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416) 2024-04-25T14:53:14.920 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:14 smithi150 podman[69242]: 2024-04-25 14:53:14.661569758 +0000 UTC m=+5.853248152 container cleanup 23336e8514976117276c641a020f40412599bc8be1f5c311504e065039d1ca17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-25T14:53:14.921 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:14 smithi150 bash[69242]: ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10 2024-04-25T14:53:15.211 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:14 smithi150 podman[69259]: 2024-04-25 14:53:14.920825747 +0000 UTC m=+0.612807815 container remove 23336e8514976117276c641a020f40412599bc8be1f5c311504e065039d1ca17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416) 2024-04-25T14:53:15.566 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:15 smithi150 podman[69347]: 2024-04-25 14:53:15.180570984 +0000 UTC m=+0.018119271 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T14:53:15.566 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:15 smithi150 podman[69347]: 2024-04-25 14:53:15.322014507 +0000 UTC m=+0.159562825 container create ae3d30d2be344aab9611e367137c7c6cde9b05ad2d03a9130536dd31ca7037ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-25T14:53:15.566 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:15 smithi150 podman[69347]: 2024-04-25 14:53:15.565937975 +0000 UTC m=+0.403486248 container init ae3d30d2be344aab9611e367137c7c6cde9b05ad2d03a9130536dd31ca7037ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-25T14:53:15.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:15 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:15.234+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:53:15.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:15 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:15.234+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:53:15.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:15 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:15.234+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:53:15.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:15 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:15.234+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:22.079400+0000 front 2024-04-25T14:52:22.079475+0000 (oldest deadline 2024-04-25T14:52:44.379400+0000) 2024-04-25T14:53:15.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:15 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:15.234+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:27.181013+0000 front 2024-04-25T14:52:27.181032+0000 (oldest deadline 2024-04-25T14:52:52.481109+0000) 2024-04-25T14:53:15.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:15 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:15.234+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2024-04-25T14:52:34.782054+0000 front 2024-04-25T14:52:34.782189+0000 (oldest deadline 2024-04-25T14:53:00.681787+0000) 2024-04-25T14:53:15.706 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:15 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:15.234+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-04-25T14:52:45.982780+0000 front 2024-04-25T14:52:45.982720+0000 (oldest deadline 2024-04-25T14:53:10.682715+0000) 2024-04-25T14:53:15.863 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:15 smithi150 podman[69347]: 2024-04-25 14:53:15.569356184 +0000 UTC m=+0.406904463 container start ae3d30d2be344aab9611e367137c7c6cde9b05ad2d03a9130536dd31ca7037ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416) 2024-04-25T14:53:15.863 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:15 smithi150 podman[69347]: 2024-04-25 14:53:15.632481842 +0000 UTC m=+0.470030131 container attach ae3d30d2be344aab9611e367137c7c6cde9b05ad2d03a9130536dd31ca7037ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T14:53:16.201 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 14:53:15 smithi150 podman[69347]: 2024-04-25 14:53:15.864364251 +0000 UTC m=+0.701912547 container died ae3d30d2be344aab9611e367137c7c6cde9b05ad2d03a9130536dd31ca7037ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T14:53:16.487 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@osd.10.service' 2024-04-25T14:53:16.564 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:53:16.564 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-25T14:53:16.564 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-25T14:53:16.564 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@osd.11 2024-04-25T14:53:16.591 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:16 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:16.255+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:53:16.591 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:16 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:16.255+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:53:16.592 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:16 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:16.255+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:53:16.592 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:16 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:16.255+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:22.079400+0000 front 2024-04-25T14:52:22.079475+0000 (oldest deadline 2024-04-25T14:52:44.379400+0000) 2024-04-25T14:53:16.592 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:16 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:16.255+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:27.181013+0000 front 2024-04-25T14:52:27.181032+0000 (oldest deadline 2024-04-25T14:52:52.481109+0000) 2024-04-25T14:53:16.592 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:16 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:16.255+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2024-04-25T14:52:34.782054+0000 front 2024-04-25T14:52:34.782189+0000 (oldest deadline 2024-04-25T14:53:00.681787+0000) 2024-04-25T14:53:16.592 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:16 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:16.255+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-04-25T14:52:45.982780+0000 front 2024-04-25T14:52:45.982720+0000 (oldest deadline 2024-04-25T14:53:10.682715+0000) 2024-04-25T14:53:16.860 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:16 smithi183 systemd[1]: Stopping Ceph osd.11 for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:53:17.204 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:16 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:16.860+0000 7f26c4bb4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:53:17.204 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:16 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:16.860+0000 7f26c4bb4700 -1 osd.11 106 *** Got signal Terminated *** 2024-04-25T14:53:17.204 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:16 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:16.860+0000 7f26c4bb4700 -1 osd.11 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:53:17.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:17 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:17.283+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:53:17.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:17 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:17.283+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:53:17.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:17 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:17.283+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:53:17.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:17 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:17.283+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:22.079400+0000 front 2024-04-25T14:52:22.079475+0000 (oldest deadline 2024-04-25T14:52:44.379400+0000) 2024-04-25T14:53:17.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:17 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:17.283+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:27.181013+0000 front 2024-04-25T14:52:27.181032+0000 (oldest deadline 2024-04-25T14:52:52.481109+0000) 2024-04-25T14:53:17.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:17 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:17.283+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2024-04-25T14:52:34.782054+0000 front 2024-04-25T14:52:34.782189+0000 (oldest deadline 2024-04-25T14:53:00.681787+0000) 2024-04-25T14:53:17.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:17 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:17.283+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-04-25T14:52:45.982780+0000 front 2024-04-25T14:52:45.982720+0000 (oldest deadline 2024-04-25T14:53:10.682715+0000) 2024-04-25T14:53:18.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:18 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:18.277+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:53:18.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:18 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:18.277+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:53:18.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:18 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:18.277+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:53:18.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:18 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:18.277+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:22.079400+0000 front 2024-04-25T14:52:22.079475+0000 (oldest deadline 2024-04-25T14:52:44.379400+0000) 2024-04-25T14:53:18.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:18 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:18.277+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:27.181013+0000 front 2024-04-25T14:52:27.181032+0000 (oldest deadline 2024-04-25T14:52:52.481109+0000) 2024-04-25T14:53:18.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:18 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:18.277+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2024-04-25T14:52:34.782054+0000 front 2024-04-25T14:52:34.782189+0000 (oldest deadline 2024-04-25T14:53:00.681787+0000) 2024-04-25T14:53:18.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:18 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:18.277+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-04-25T14:52:45.982780+0000 front 2024-04-25T14:52:45.982720+0000 (oldest deadline 2024-04-25T14:53:10.682715+0000) 2024-04-25T14:53:19.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:19 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:19.326+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:53:19.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:19 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:19.326+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:53:19.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:19 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:19.326+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:53:19.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:19 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:19.326+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:22.079400+0000 front 2024-04-25T14:52:22.079475+0000 (oldest deadline 2024-04-25T14:52:44.379400+0000) 2024-04-25T14:53:19.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:19 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:19.326+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:27.181013+0000 front 2024-04-25T14:52:27.181032+0000 (oldest deadline 2024-04-25T14:52:52.481109+0000) 2024-04-25T14:53:19.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:19 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:19.326+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2024-04-25T14:52:34.782054+0000 front 2024-04-25T14:52:34.782189+0000 (oldest deadline 2024-04-25T14:53:00.681787+0000) 2024-04-25T14:53:19.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:19 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:19.326+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-04-25T14:52:45.982780+0000 front 2024-04-25T14:52:45.982720+0000 (oldest deadline 2024-04-25T14:53:10.682715+0000) 2024-04-25T14:53:19.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:19 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:19.326+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6820 osd.8 since back 2024-04-25T14:52:55.784536+0000 front 2024-04-25T14:52:55.784568+0000 (oldest deadline 2024-04-25T14:53:18.684585+0000) 2024-04-25T14:53:20.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:20 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:20.373+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:53:20.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:20 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:20.373+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:53:20.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:20 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:20.373+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:53:20.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:20 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:20.373+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:22.079400+0000 front 2024-04-25T14:52:22.079475+0000 (oldest deadline 2024-04-25T14:52:44.379400+0000) 2024-04-25T14:53:20.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:20 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:20.373+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:27.181013+0000 front 2024-04-25T14:52:27.181032+0000 (oldest deadline 2024-04-25T14:52:52.481109+0000) 2024-04-25T14:53:20.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:20 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:20.373+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2024-04-25T14:52:34.782054+0000 front 2024-04-25T14:52:34.782189+0000 (oldest deadline 2024-04-25T14:53:00.681787+0000) 2024-04-25T14:53:20.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:20 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:20.373+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-04-25T14:52:45.982780+0000 front 2024-04-25T14:52:45.982720+0000 (oldest deadline 2024-04-25T14:53:10.682715+0000) 2024-04-25T14:53:20.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:20 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:20.373+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6820 osd.8 since back 2024-04-25T14:52:55.784536+0000 front 2024-04-25T14:52:55.784568+0000 (oldest deadline 2024-04-25T14:53:18.684585+0000) 2024-04-25T14:53:21.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:21 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:21.417+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6806 osd.0 since back 2024-04-25T14:51:46.272109+0000 front 2024-04-25T14:51:46.272094+0000 (oldest deadline 2024-04-25T14:52:12.171929+0000) 2024-04-25T14:53:21.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:21 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:21.417+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T14:51:57.174781+0000 front 2024-04-25T14:51:57.174797+0000 (oldest deadline 2024-04-25T14:52:21.274630+0000) 2024-04-25T14:53:21.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:21 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:21.417+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-04-25T14:52:05.975611+0000 front 2024-04-25T14:52:05.975578+0000 (oldest deadline 2024-04-25T14:52:27.675720+0000) 2024-04-25T14:53:21.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:21 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:21.417+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T14:52:22.079400+0000 front 2024-04-25T14:52:22.079475+0000 (oldest deadline 2024-04-25T14:52:44.379400+0000) 2024-04-25T14:53:21.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:21 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:21.417+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-04-25T14:52:27.181013+0000 front 2024-04-25T14:52:27.181032+0000 (oldest deadline 2024-04-25T14:52:52.481109+0000) 2024-04-25T14:53:21.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:21 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:21.417+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.53:6822 osd.6 since back 2024-04-25T14:52:34.782054+0000 front 2024-04-25T14:52:34.782189+0000 (oldest deadline 2024-04-25T14:53:00.681787+0000) 2024-04-25T14:53:21.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:21 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:21.417+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-04-25T14:52:45.982780+0000 front 2024-04-25T14:52:45.982720+0000 (oldest deadline 2024-04-25T14:53:10.682715+0000) 2024-04-25T14:53:21.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:21 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11[57590]: 2024-04-25T14:53:21.417+0000 7f26bd1a0700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.183:6820 osd.8 since back 2024-04-25T14:52:55.784536+0000 front 2024-04-25T14:52:55.784568+0000 (oldest deadline 2024-04-25T14:53:18.684585+0000) 2024-04-25T14:53:22.661 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:22 smithi183 podman[69192]: 2024-04-25 14:53:22.401035853 +0000 UTC m=+5.644466804 container died a940eabdc2d796ef41e4e0bb6186da8a4f61899a922e80cb2afa2a7d4a21b2f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True) 2024-04-25T14:53:22.919 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:22 smithi183 podman[69192]: 2024-04-25 14:53:22.734247279 +0000 UTC m=+5.977678242 container cleanup a940eabdc2d796ef41e4e0bb6186da8a4f61899a922e80cb2afa2a7d4a21b2f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-25T14:53:22.919 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:22 smithi183 bash[69192]: ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11 2024-04-25T14:53:23.204 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:22 smithi183 podman[69208]: 2024-04-25 14:53:22.919149674 +0000 UTC m=+0.512009780 container remove a940eabdc2d796ef41e4e0bb6186da8a4f61899a922e80cb2afa2a7d4a21b2f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11, org.label-schema.build-date=20240416, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:53:23.615 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:23 smithi183 podman[69301]: 2024-04-25 14:53:23.24751005 +0000 UTC m=+0.018081029 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T14:53:23.615 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:23 smithi183 podman[69301]: 2024-04-25 14:53:23.345244096 +0000 UTC m=+0.115815052 container create 670de675ada1c9fc46639bb29d66f8a6d4493ce893f66d63ba95819e979d6c0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11-deactivate, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-25T14:53:23.615 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:23 smithi183 podman[69301]: 2024-04-25 14:53:23.53998868 +0000 UTC m=+0.310559633 container init 670de675ada1c9fc46639bb29d66f8a6d4493ce893f66d63ba95819e979d6c0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-25T14:53:23.615 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:23 smithi183 podman[69301]: 2024-04-25 14:53:23.54302562 +0000 UTC m=+0.313596580 container start 670de675ada1c9fc46639bb29d66f8a6d4493ce893f66d63ba95819e979d6c0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-25T14:53:23.615 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:23 smithi183 podman[69301]: 2024-04-25 14:53:23.614838254 +0000 UTC m=+0.385409211 container attach 670de675ada1c9fc46639bb29d66f8a6d4493ce893f66d63ba95819e979d6c0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-25T14:53:23.954 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:23 smithi183 podman[69301]: 2024-04-25 14:53:23.838157888 +0000 UTC m=+0.608728849 container died 670de675ada1c9fc46639bb29d66f8a6d4493ce893f66d63ba95819e979d6c0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-25T14:53:24.417 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@osd.11.service' 2024-04-25T14:53:24.454 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:24 smithi183 podman[69327]: 2024-04-25 14:53:24.392760673 +0000 UTC m=+0.549222502 container remove 670de675ada1c9fc46639bb29d66f8a6d4493ce893f66d63ba95819e979d6c0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-25T14:53:24.454 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:24 smithi183 systemd[1]: ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@osd.11.service: Deactivated successfully. 2024-04-25T14:53:24.454 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:24 smithi183 systemd[1]: Stopped Ceph osd.11 for fd30e6ce-030f-11ef-bc93-c7b262605968. 2024-04-25T14:53:24.455 INFO:journalctl@ceph.osd.11.smithi183.stdout:Apr 25 14:53:24 smithi183 systemd[1]: ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@osd.11.service: Consumed 7.915s CPU time. 2024-04-25T14:53:24.490 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:53:24.490 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-25T14:53:24.491 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-25T14:53:24.491 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mds.a 2024-04-25T14:53:25.033 INFO:journalctl@ceph.mds.a.smithi053.stdout:Apr 25 14:53:24 smithi053 systemd[1]: Stopping Ceph mds.a for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:53:25.533 INFO:journalctl@ceph.mds.a.smithi053.stdout:Apr 25 14:53:25 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-a[69332]: 2024-04-25T14:53:25.073+0000 7f0d8fc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:53:25.534 INFO:journalctl@ceph.mds.a.smithi053.stdout:Apr 25 14:53:25 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-a[69332]: 2024-04-25T14:53:25.073+0000 7f0d8fc00700 -1 mds.a *** got signal Terminated *** 2024-04-25T14:53:26.659 INFO:journalctl@ceph.mds.a.smithi053.stdout:Apr 25 14:53:26 smithi053 podman[87120]: 2024-04-25 14:53:26.363342643 +0000 UTC m=+1.424291652 container died 39903d2c0c72d2e30a91ee177c1e850fe7d5415ddee8343ac05e0e5abd7ebe0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-a, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-25T14:53:26.958 INFO:journalctl@ceph.mds.a.smithi053.stdout:Apr 25 14:53:26 smithi053 podman[87120]: 2024-04-25 14:53:26.773424984 +0000 UTC m=+1.834374015 container cleanup 39903d2c0c72d2e30a91ee177c1e850fe7d5415ddee8343ac05e0e5abd7ebe0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416) 2024-04-25T14:53:26.959 INFO:journalctl@ceph.mds.a.smithi053.stdout:Apr 25 14:53:26 smithi053 bash[87120]: ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-a 2024-04-25T14:53:27.283 INFO:journalctl@ceph.mds.a.smithi053.stdout:Apr 25 14:53:26 smithi053 podman[87134]: 2024-04-25 14:53:26.957697313 +0000 UTC m=+0.572893103 container remove 39903d2c0c72d2e30a91ee177c1e850fe7d5415ddee8343ac05e0e5abd7ebe0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-a, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-25T14:53:27.462 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mds.a.service' 2024-04-25T14:53:27.578 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:53:27.579 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-25T14:53:27.579 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-25T14:53:27.579 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mds.d 2024-04-25T14:53:28.033 INFO:journalctl@ceph.mds.d.smithi053.stdout:Apr 25 14:53:27 smithi053 systemd[1]: Stopping Ceph mds.d for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:53:28.533 INFO:journalctl@ceph.mds.d.smithi053.stdout:Apr 25 14:53:28 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-d[68340]: 2024-04-25T14:53:28.201+0000 7fef55000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:53:28.533 INFO:journalctl@ceph.mds.d.smithi053.stdout:Apr 25 14:53:28 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-d[68340]: 2024-04-25T14:53:28.201+0000 7fef55000700 -1 mds.d *** got signal Terminated *** 2024-04-25T14:53:32.533 INFO:journalctl@ceph.mds.d.smithi053.stdout:Apr 25 14:53:32 smithi053 podman[87238]: 2024-04-25 14:53:32.2241035 +0000 UTC m=+4.155738190 container died a35a0e9c6b5135a100009a093b2a1efc541886c775b2ff5c3ffc3aee4a8f5850 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416) 2024-04-25T14:53:33.033 INFO:journalctl@ceph.mds.d.smithi053.stdout:Apr 25 14:53:32 smithi053 podman[87238]: 2024-04-25 14:53:32.768487622 +0000 UTC m=+4.700122342 container cleanup a35a0e9c6b5135a100009a093b2a1efc541886c775b2ff5c3ffc3aee4a8f5850 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-d, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:53:33.033 INFO:journalctl@ceph.mds.d.smithi053.stdout:Apr 25 14:53:32 smithi053 bash[87238]: ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-d 2024-04-25T14:53:33.533 INFO:journalctl@ceph.mds.d.smithi053.stdout:Apr 25 14:53:33 smithi053 podman[87252]: 2024-04-25 14:53:33.060023228 +0000 UTC m=+0.818916363 container remove a35a0e9c6b5135a100009a093b2a1efc541886c775b2ff5c3ffc3aee4a8f5850 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-25T14:53:33.579 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mds.d.service' 2024-04-25T14:53:33.692 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:53:33.692 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-25T14:53:33.692 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-25T14:53:33.692 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mds.g 2024-04-25T14:53:34.283 INFO:journalctl@ceph.mds.g.smithi053.stdout:Apr 25 14:53:33 smithi053 systemd[1]: Stopping Ceph mds.g for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:53:34.783 INFO:journalctl@ceph.mds.g.smithi053.stdout:Apr 25 14:53:34 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-g[68985]: 2024-04-25T14:53:34.319+0000 7f8f90c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:53:34.783 INFO:journalctl@ceph.mds.g.smithi053.stdout:Apr 25 14:53:34 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-g[68985]: 2024-04-25T14:53:34.319+0000 7f8f90c00700 -1 mds.g *** got signal Terminated *** 2024-04-25T14:53:38.601 INFO:journalctl@ceph.mds.g.smithi053.stdout:Apr 25 14:53:38 smithi053 podman[87358]: 2024-04-25 14:53:38.342130906 +0000 UTC m=+4.154989676 container died 67aef1549e186d1f5ceee6e83d3e6a40111ed214391c2bc4eec3ac75699e56e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T14:53:38.905 INFO:journalctl@ceph.mds.g.smithi053.stdout:Apr 25 14:53:38 smithi053 podman[87358]: 2024-04-25 14:53:38.745689432 +0000 UTC m=+4.558548215 container cleanup 67aef1549e186d1f5ceee6e83d3e6a40111ed214391c2bc4eec3ac75699e56e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-g, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=) 2024-04-25T14:53:38.905 INFO:journalctl@ceph.mds.g.smithi053.stdout:Apr 25 14:53:38 smithi053 bash[87358]: ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-g 2024-04-25T14:53:39.283 INFO:journalctl@ceph.mds.g.smithi053.stdout:Apr 25 14:53:38 smithi053 podman[87373]: 2024-04-25 14:53:38.904835504 +0000 UTC m=+0.550847851 container remove 67aef1549e186d1f5ceee6e83d3e6a40111ed214391c2bc4eec3ac75699e56e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-g, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-25T14:53:39.462 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mds.g.service' 2024-04-25T14:53:39.575 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:53:39.575 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-25T14:53:39.575 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-25T14:53:39.576 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mds.j 2024-04-25T14:53:40.033 INFO:journalctl@ceph.mds.j.smithi053.stdout:Apr 25 14:53:39 smithi053 systemd[1]: Stopping Ceph mds.j for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:53:40.533 INFO:journalctl@ceph.mds.j.smithi053.stdout:Apr 25 14:53:40 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-j[68004]: 2024-04-25T14:53:40.222+0000 7ff9c1e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:53:40.533 INFO:journalctl@ceph.mds.j.smithi053.stdout:Apr 25 14:53:40 smithi053 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-j[68004]: 2024-04-25T14:53:40.222+0000 7ff9c1e00700 -1 mds.j *** got signal Terminated *** 2024-04-25T14:53:44.533 INFO:journalctl@ceph.mds.j.smithi053.stdout:Apr 25 14:53:44 smithi053 podman[87478]: 2024-04-25 14:53:44.240767283 +0000 UTC m=+4.199933840 container died 5baff6f1ec70e2890ea55bb2314c42a865b9c558dcc3ec1dd494c9a135e21c3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-j, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-25T14:53:44.992 INFO:journalctl@ceph.mds.j.smithi053.stdout:Apr 25 14:53:44 smithi053 podman[87478]: 2024-04-25 14:53:44.657189911 +0000 UTC m=+4.616356498 container cleanup 5baff6f1ec70e2890ea55bb2314c42a865b9c558dcc3ec1dd494c9a135e21c3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-j, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-25T14:53:44.992 INFO:journalctl@ceph.mds.j.smithi053.stdout:Apr 25 14:53:44 smithi053 bash[87478]: ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-j 2024-04-25T14:53:45.283 INFO:journalctl@ceph.mds.j.smithi053.stdout:Apr 25 14:53:44 smithi053 podman[87504]: 2024-04-25 14:53:44.990996821 +0000 UTC m=+0.729201799 container remove 5baff6f1ec70e2890ea55bb2314c42a865b9c558dcc3ec1dd494c9a135e21c3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-j, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-25T14:53:45.465 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mds.j.service' 2024-04-25T14:53:45.582 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:53:45.582 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-25T14:53:45.583 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-25T14:53:45.583 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mds.b 2024-04-25T14:53:45.878 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 25 14:53:45 smithi150 systemd[1]: Stopping Ceph mds.b for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:53:46.211 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 25 14:53:45 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-b[62911]: 2024-04-25T14:53:45.877+0000 7f6a4a331700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:53:46.211 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 25 14:53:45 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-b[62911]: 2024-04-25T14:53:45.877+0000 7f6a4a331700 -1 mds.b *** got signal Terminated *** 2024-04-25T14:53:47.078 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 25 14:53:46 smithi150 podman[69453]: 2024-04-25 14:53:46.898445172 +0000 UTC m=+1.121927900 container died 835db8e020fa3e17336f5ae2c39735bae50f67950623744fd55403d26ad17869 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-25T14:53:47.412 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 25 14:53:47 smithi150 podman[69453]: 2024-04-25 14:53:47.236172347 +0000 UTC m=+1.459655073 container cleanup 835db8e020fa3e17336f5ae2c39735bae50f67950623744fd55403d26ad17869 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T14:53:47.412 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 25 14:53:47 smithi150 bash[69453]: ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-b 2024-04-25T14:53:47.669 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 25 14:53:47 smithi150 podman[69468]: 2024-04-25 14:53:47.412280798 +0000 UTC m=+0.513153839 container remove 835db8e020fa3e17336f5ae2c39735bae50f67950623744fd55403d26ad17869 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-b, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-25T14:53:47.674 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mds.b.service' 2024-04-25T14:53:47.709 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:53:47.709 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-25T14:53:47.709 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-25T14:53:47.709 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mds.e 2024-04-25T14:53:47.961 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 25 14:53:47 smithi150 systemd[1]: Stopping Ceph mds.e for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:53:48.461 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 25 14:53:48 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-e[62279]: 2024-04-25T14:53:48.056+0000 7fac67b26700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:53:48.461 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 25 14:53:48 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-e[62279]: 2024-04-25T14:53:48.056+0000 7fac67b26700 -1 mds.e *** got signal Terminated *** 2024-04-25T14:53:51.507 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 25 14:53:51 smithi150 podman[69584]: 2024-04-25 14:53:51.221248911 +0000 UTC m=+3.288340784 container died b270a6af510a8e330d1804de028d11a270fe4b8e18f35332c793cfd4a69e2f83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-e, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-25T14:53:51.832 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 25 14:53:51 smithi150 podman[69584]: 2024-04-25 14:53:51.598727067 +0000 UTC m=+3.665818937 container cleanup b270a6af510a8e330d1804de028d11a270fe4b8e18f35332c793cfd4a69e2f83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-25T14:53:51.833 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 25 14:53:51 smithi150 bash[69584]: ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-e 2024-04-25T14:53:51.980 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mds.e.service' 2024-04-25T14:53:52.013 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:53:52.014 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-25T14:53:52.014 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-25T14:53:52.014 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mds.h 2024-04-25T14:53:52.461 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 25 14:53:52 smithi150 systemd[1]: Stopping Ceph mds.h for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:53:52.461 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 25 14:53:52 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-h[62587]: 2024-04-25T14:53:52.326+0000 7f2395b5d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:53:52.461 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 25 14:53:52 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-h[62587]: 2024-04-25T14:53:52.326+0000 7f2395b5d700 -1 mds.h *** got signal Terminated *** 2024-04-25T14:53:55.186 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 25 14:53:54 smithi150 podman[69706]: 2024-04-25 14:53:54.898452708 +0000 UTC m=+2.672986743 container died 803c34c5a1c6314a346783f7e027fc67addeadfd9258b49078ddb2b62393b508 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-h, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T14:53:55.461 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 25 14:53:55 smithi150 podman[69706]: 2024-04-25 14:53:55.252241595 +0000 UTC m=+3.026775675 container cleanup 803c34c5a1c6314a346783f7e027fc67addeadfd9258b49078ddb2b62393b508 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-h, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T14:53:55.461 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 25 14:53:55 smithi150 bash[69706]: ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-h 2024-04-25T14:53:55.717 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mds.h.service' 2024-04-25T14:53:55.749 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:53:55.749 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-25T14:53:55.750 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-25T14:53:55.750 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mds.k 2024-04-25T14:53:56.211 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 25 14:53:55 smithi150 systemd[1]: Stopping Ceph mds.k for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:53:56.211 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 25 14:53:56 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-k[61962]: 2024-04-25T14:53:56.072+0000 7f91923bc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:53:56.211 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 25 14:53:56 smithi150 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-k[61962]: 2024-04-25T14:53:56.072+0000 7f91923bc700 -1 mds.k *** got signal Terminated *** 2024-04-25T14:54:00.381 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 25 14:54:00 smithi150 podman[69827]: 2024-04-25 14:54:00.087081053 +0000 UTC m=+4.115147714 container died 21b3427b8c012213dd9767aab26b693e6550bc90159c24f403b9132dc5046a3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-25T14:54:00.706 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 25 14:54:00 smithi150 podman[69827]: 2024-04-25 14:54:00.489117023 +0000 UTC m=+4.517183699 container cleanup 21b3427b8c012213dd9767aab26b693e6550bc90159c24f403b9132dc5046a3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-k, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True) 2024-04-25T14:54:00.707 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 25 14:54:00 smithi150 bash[69827]: ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-k 2024-04-25T14:54:00.837 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mds.k.service' 2024-04-25T14:54:00.867 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:54:00.868 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-25T14:54:00.868 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-25T14:54:00.868 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mds.c 2024-04-25T14:54:01.204 INFO:journalctl@ceph.mds.c.smithi183.stdout:Apr 25 14:54:00 smithi183 systemd[1]: Stopping Ceph mds.c for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:54:01.704 INFO:journalctl@ceph.mds.c.smithi183.stdout:Apr 25 14:54:01 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-c[62590]: 2024-04-25T14:54:01.239+0000 7f23d0fbf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:54:01.704 INFO:journalctl@ceph.mds.c.smithi183.stdout:Apr 25 14:54:01 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-c[62590]: 2024-04-25T14:54:01.239+0000 7f23d0fbf700 -1 mds.c *** got signal Terminated *** 2024-04-25T14:54:03.174 INFO:journalctl@ceph.mds.c.smithi183.stdout:Apr 25 14:54:02 smithi183 podman[69408]: 2024-04-25 14:54:02.893198742 +0000 UTC m=+1.830729105 container died bd74bae21e32796343eea224b87bdff6caf6f3a7455e728ca7ce38eff177e013 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-25T14:54:03.454 INFO:journalctl@ceph.mds.c.smithi183.stdout:Apr 25 14:54:03 smithi183 podman[69408]: 2024-04-25 14:54:03.265462615 +0000 UTC m=+2.202992982 container cleanup bd74bae21e32796343eea224b87bdff6caf6f3a7455e728ca7ce38eff177e013 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-c, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T14:54:03.455 INFO:journalctl@ceph.mds.c.smithi183.stdout:Apr 25 14:54:03 smithi183 bash[69408]: ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-c 2024-04-25T14:54:03.702 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mds.c.service' 2024-04-25T14:54:03.735 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:54:03.736 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-25T14:54:03.736 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-25T14:54:03.736 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mds.f 2024-04-25T14:54:04.204 INFO:journalctl@ceph.mds.f.smithi183.stdout:Apr 25 14:54:03 smithi183 systemd[1]: Stopping Ceph mds.f for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:54:04.204 INFO:journalctl@ceph.mds.f.smithi183.stdout:Apr 25 14:54:04 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-f[62283]: 2024-04-25T14:54:04.035+0000 7fa6f46a4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:54:04.204 INFO:journalctl@ceph.mds.f.smithi183.stdout:Apr 25 14:54:04 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-f[62283]: 2024-04-25T14:54:04.035+0000 7fa6f46a4700 -1 mds.f *** got signal Terminated *** 2024-04-25T14:54:07.161 INFO:journalctl@ceph.mds.f.smithi183.stdout:Apr 25 14:54:06 smithi183 podman[69538]: 2024-04-25 14:54:06.890297486 +0000 UTC m=+2.936652227 container died 63624d8b533c07f75ba4a08835acc8c271e949ff7a0ecab5598466198f4089ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-f, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-25T14:54:07.412 INFO:journalctl@ceph.mds.f.smithi183.stdout:Apr 25 14:54:07 smithi183 podman[69538]: 2024-04-25 14:54:07.24401976 +0000 UTC m=+3.290374499 container cleanup 63624d8b533c07f75ba4a08835acc8c271e949ff7a0ecab5598466198f4089ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-f, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T14:54:07.412 INFO:journalctl@ceph.mds.f.smithi183.stdout:Apr 25 14:54:07 smithi183 bash[69538]: ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-f 2024-04-25T14:54:07.672 INFO:journalctl@ceph.mds.f.smithi183.stdout:Apr 25 14:54:07 smithi183 podman[69552]: 2024-04-25 14:54:07.412175052 +0000 UTC m=+0.519217312 container remove 63624d8b533c07f75ba4a08835acc8c271e949ff7a0ecab5598466198f4089ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T14:54:07.676 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mds.f.service' 2024-04-25T14:54:07.707 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:54:07.707 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-25T14:54:07.708 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-25T14:54:07.708 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mds.i 2024-04-25T14:54:07.954 INFO:journalctl@ceph.mds.i.smithi183.stdout:Apr 25 14:54:07 smithi183 systemd[1]: Stopping Ceph mds.i for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:54:08.454 INFO:journalctl@ceph.mds.i.smithi183.stdout:Apr 25 14:54:08 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-i[62892]: 2024-04-25T14:54:08.005+0000 7f4e31629700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:54:08.455 INFO:journalctl@ceph.mds.i.smithi183.stdout:Apr 25 14:54:08 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-i[62892]: 2024-04-25T14:54:08.005+0000 7f4e31629700 -1 mds.i *** got signal Terminated *** 2024-04-25T14:54:11.155 INFO:journalctl@ceph.mds.i.smithi183.stdout:Apr 25 14:54:10 smithi183 podman[69657]: 2024-04-25 14:54:10.895105228 +0000 UTC m=+2.978437855 container died 90c0d1203a2f408baf231e8d0fc549627a5442b194bd80ecdf3c177681478b77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-25T14:54:11.156 INFO:journalctl@ceph.mds.i.smithi183.stdout:Apr 25 14:54:11 smithi183 podman[69657]: 2024-04-25 14:54:11.155823904 +0000 UTC m=+3.239156522 container cleanup 90c0d1203a2f408baf231e8d0fc549627a5442b194bd80ecdf3c177681478b77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-i, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-25T14:54:11.407 INFO:journalctl@ceph.mds.i.smithi183.stdout:Apr 25 14:54:11 smithi183 bash[69657]: ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-i 2024-04-25T14:54:11.582 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mds.i.service' 2024-04-25T14:54:11.613 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:54:11.613 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-25T14:54:11.614 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-25T14:54:11.614 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mds.l 2024-04-25T14:54:11.927 INFO:journalctl@ceph.mds.l.smithi183.stdout:Apr 25 14:54:11 smithi183 systemd[1]: Stopping Ceph mds.l for fd30e6ce-030f-11ef-bc93-c7b262605968... 2024-04-25T14:54:12.204 INFO:journalctl@ceph.mds.l.smithi183.stdout:Apr 25 14:54:11 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-l[61970]: 2024-04-25T14:54:11.925+0000 7f6983899700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:54:12.204 INFO:journalctl@ceph.mds.l.smithi183.stdout:Apr 25 14:54:11 smithi183 ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-l[61970]: 2024-04-25T14:54:11.925+0000 7f6983899700 -1 mds.l *** got signal Terminated *** 2024-04-25T14:54:13.994 INFO:journalctl@ceph.mds.l.smithi183.stdout:Apr 25 14:54:13 smithi183 podman[69777]: 2024-04-25 14:54:13.727905578 +0000 UTC m=+1.894011290 container died de96912deee8eefe4d8f0c22e5acf13bcc68dcbc55725187b7056270b884392d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-l, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-25T14:54:14.335 INFO:journalctl@ceph.mds.l.smithi183.stdout:Apr 25 14:54:14 smithi183 podman[69777]: 2024-04-25 14:54:14.093398814 +0000 UTC m=+2.259504555 container cleanup de96912deee8eefe4d8f0c22e5acf13bcc68dcbc55725187b7056270b884392d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-l, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T14:54:14.336 INFO:journalctl@ceph.mds.l.smithi183.stdout:Apr 25 14:54:14 smithi183 bash[69777]: ceph-fd30e6ce-030f-11ef-bc93-c7b262605968-mds-l 2024-04-25T14:54:14.477 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fd30e6ce-030f-11ef-bc93-c7b262605968@mds.l.service' 2024-04-25T14:54:14.508 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:54:14.509 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-25T14:54:14.509 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T14:54:14.839 INFO:teuthology.orchestra.run.smithi053.stdout:Deleting cluster with fsid: fd30e6ce-030f-11ef-bc93-c7b262605968 2024-04-25T14:54:19.925 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T14:54:20.116 INFO:teuthology.orchestra.run.smithi150.stdout:Deleting cluster with fsid: fd30e6ce-030f-11ef-bc93-c7b262605968 2024-04-25T14:54:22.566 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T14:54:22.755 INFO:teuthology.orchestra.run.smithi183.stdout:Deleting cluster with fsid: fd30e6ce-030f-11ef-bc93-c7b262605968 2024-04-25T14:54:25.069 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T14:54:25.158 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T14:54:25.189 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T14:54:25.214 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-25T14:54:25.217 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673495/remote/smithi053/crash 2024-04-25T14:54:25.217 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/crash -- . 2024-04-25T14:54:25.340 INFO:teuthology.orchestra.run.smithi053.stderr:tar: /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T14:54:25.340 INFO:teuthology.orchestra.run.smithi053.stderr:tar: Error is not recoverable: exiting now 2024-04-25T14:54:25.345 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673495/remote/smithi150/crash 2024-04-25T14:54:25.346 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/crash -- . 2024-04-25T14:54:25.391 INFO:teuthology.orchestra.run.smithi150.stderr:tar: /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T14:54:25.391 INFO:teuthology.orchestra.run.smithi150.stderr:tar: Error is not recoverable: exiting now 2024-04-25T14:54:25.393 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673495/remote/smithi183/crash 2024-04-25T14:54:25.394 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/crash -- . 2024-04-25T14:54:25.428 INFO:teuthology.orchestra.run.smithi183.stderr:tar: /var/lib/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T14:54:25.428 INFO:teuthology.orchestra.run.smithi183.stderr:tar: Error is not recoverable: exiting now 2024-04-25T14:54:25.429 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-25T14:54:25.430 DEBUG:teuthology.orchestra.run.smithi053:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-25T14:54:25.542 INFO:tasks.cephadm:Compressing logs... 2024-04-25T14:54:25.542 DEBUG:teuthology.orchestra.run.smithi053:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T14:54:25.546 DEBUG:teuthology.orchestra.run.smithi150:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T14:54:25.549 DEBUG:teuthology.orchestra.run.smithi183:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T14:54:25.573 INFO:teuthology.orchestra.run.smithi183.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph.log 2024-04-25T14:54:25.573 INFO:teuthology.orchestra.run.smithi183.stderr:: No such file or directory 2024-04-25T14:54:25.573 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-25T14:54:25.573 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-25T14:54:25.573 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T14:54:25.574 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-25T14:54:25.574 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-25T14:54:25.580 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-25T14:54:25.580 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-25T14:54:25.580 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-25T14:54:25.580 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mgr.z.log: 87.9% -- replaced with /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T14:54:25.580 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T14:54:25.580 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mds.i.log: 90.5% -- replaced with /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T14:54:25.581 INFO:teuthology.orchestra.run.smithi150.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-25T14:54:25.581 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph.log 2024-04-25T14:54:25.581 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T14:54:25.581 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-25T14:54:25.581 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-25T14:54:25.581 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-25T14:54:25.582 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-25T14:54:25.582 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-25T14:54:25.582 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-25T14:54:25.582 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mgr.y.log: 87.9% -- replaced with /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T14:54:25.582 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-25T14:54:25.582 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T14:54:25.582 INFO:teuthology.orchestra.run.smithi150.stderr: 90.4% -- replaced with /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T14:54:25.582 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-25T14:54:25.583 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.7% -- replaced with /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T14:54:25.583 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T14:54:25.584 INFO:teuthology.orchestra.run.smithi183.stderr: 92.0% -- replaced with /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-25T14:54:25.586 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mds.e.log: 94.4% -- replaced with /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-25T14:54:25.586 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-25T14:54:25.587 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-25T14:54:25.587 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-25T14:54:25.588 INFO:teuthology.orchestra.run.smithi183.stderr: 83.7% -- replaced with /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T14:54:25.589 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-volume.log: 92.0% -- replaced with /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-25T14:54:25.589 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-25T14:54:25.595 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T14:54:25.600 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T14:54:25.601 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T14:54:25.602 INFO:teuthology.orchestra.run.smithi053.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-25T14:54:25.602 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph.log 2024-04-25T14:54:25.603 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T14:54:25.604 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-25T14:54:25.605 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-25T14:54:25.606 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mds.k.log: 94.4% -- replaced with /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-25T14:54:25.607 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-25T14:54:25.608 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mds.j.log: 87.8% -- replaced with /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T14:54:25.608 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-25T14:54:25.609 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-25T14:54:25.609 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T14:54:25.610 INFO:teuthology.orchestra.run.smithi053.stderr: 90.3% -- replaced with /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T14:54:25.610 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-25T14:54:25.611 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-25T14:54:25.612 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-25T14:54:25.612 INFO:teuthology.orchestra.run.smithi053.stderr: 94.3% -- replaced with /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-25T14:54:25.614 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T14:54:25.616 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-osd.6.log: /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.9% -- replaced with /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T14:54:25.617 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-25T14:54:25.620 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T14:54:25.621 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T14:54:25.624 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-25T14:54:25.626 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mds.l.log: 94.5% -- replaced with /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-25T14:54:25.627 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-25T14:54:25.630 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T14:54:25.634 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mds.d.log: /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T14:54:25.635 INFO:teuthology.orchestra.run.smithi183.stderr: 93.1% -- replaced with /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-25T14:54:25.635 INFO:teuthology.orchestra.run.smithi183.stderr: 91.0% -- replaced with /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T14:54:25.636 INFO:teuthology.orchestra.run.smithi053.stderr: 94.3% -- replaced with /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-25T14:54:25.637 INFO:teuthology.orchestra.run.smithi183.stderr: 94.4% -- replaced with /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-25T14:54:25.641 INFO:teuthology.orchestra.run.smithi183.stderr: 94.0% -- replaced with /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-25T14:54:25.645 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/cephadm.log: 94.4% -- replaced with /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-25T14:54:25.645 INFO:teuthology.orchestra.run.smithi150.stderr: 94.5% -- replaced with /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-25T14:54:25.646 INFO:teuthology.orchestra.run.smithi053.stderr: 91.0% -- replaced with /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T14:54:25.653 INFO:teuthology.orchestra.run.smithi053.stderr: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T14:54:25.686 INFO:teuthology.orchestra.run.smithi053.stderr: 92.7% -- replaced with /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-25T14:54:25.787 INFO:teuthology.orchestra.run.smithi150.stderr: 93.4% -- replaced with /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-25T14:54:25.945 INFO:teuthology.orchestra.run.smithi150.stderr: 91.8% -- replaced with /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-25T14:54:25.952 INFO:teuthology.orchestra.run.smithi183.stderr: 93.0% -- replaced with /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-25T14:54:26.012 INFO:teuthology.orchestra.run.smithi053.stderr: 90.1% -- replaced with /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-25T14:54:26.029 INFO:teuthology.orchestra.run.smithi183.stderr: 91.9% -- replaced with /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-25T14:54:26.039 INFO:teuthology.orchestra.run.smithi150.stderr: 92.9% -- replaced with /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-25T14:54:26.042 INFO:teuthology.orchestra.run.smithi183.stderr: 92.8% -- replaced with /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-25T14:54:26.109 INFO:teuthology.orchestra.run.smithi150.stderr: 92.9% -- replaced with /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-25T14:54:26.142 INFO:teuthology.orchestra.run.smithi183.stderr: 92.6% -- replaced with /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-25T14:54:26.169 INFO:teuthology.orchestra.run.smithi150.stderr: 92.5% -- replaced with /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-25T14:54:26.247 INFO:teuthology.orchestra.run.smithi183.stderr: 92.8% -- replaced with /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-25T14:54:26.249 INFO:teuthology.orchestra.run.smithi183.stderr: 2024-04-25T14:54:26.249 INFO:teuthology.orchestra.run.smithi183.stderr:real 0m0.686s 2024-04-25T14:54:26.249 INFO:teuthology.orchestra.run.smithi183.stderr:user 0m2.622s 2024-04-25T14:54:26.249 INFO:teuthology.orchestra.run.smithi183.stderr:sys 0m0.118s 2024-04-25T14:54:26.282 INFO:teuthology.orchestra.run.smithi053.stderr: 92.8% -- replaced with /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-25T14:54:26.320 INFO:teuthology.orchestra.run.smithi053.stderr: 92.9% -- replaced with /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-25T14:54:26.356 INFO:teuthology.orchestra.run.smithi053.stderr: 92.7% -- replaced with /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-25T14:54:26.359 INFO:teuthology.orchestra.run.smithi053.stderr: 92.4% -- replaced with /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-25T14:54:26.369 INFO:teuthology.orchestra.run.smithi150.stderr: 92.7% -- replaced with /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-25T14:54:26.371 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-25T14:54:26.371 INFO:teuthology.orchestra.run.smithi150.stderr:real 0m0.807s 2024-04-25T14:54:26.371 INFO:teuthology.orchestra.run.smithi150.stderr:user 0m2.951s 2024-04-25T14:54:26.371 INFO:teuthology.orchestra.run.smithi150.stderr:sys 0m0.136s 2024-04-25T14:54:26.547 INFO:teuthology.orchestra.run.smithi053.stderr: 90.5% -- replaced with /var/log/ceph/fd30e6ce-030f-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-25T14:54:26.551 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-04-25T14:54:26.551 INFO:teuthology.orchestra.run.smithi053.stderr:real 0m0.973s 2024-04-25T14:54:26.552 INFO:teuthology.orchestra.run.smithi053.stderr:user 0m3.999s 2024-04-25T14:54:26.552 INFO:teuthology.orchestra.run.smithi053.stderr:sys 0m0.369s 2024-04-25T14:54:26.553 INFO:tasks.cephadm:Archiving logs... 2024-04-25T14:54:26.553 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673495/remote/smithi053/log 2024-04-25T14:54:26.554 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T14:54:27.118 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673495/remote/smithi150/log 2024-04-25T14:54:27.119 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T14:54:27.547 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673495/remote/smithi183/log 2024-04-25T14:54:27.548 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T14:54:27.938 INFO:tasks.cephadm:Removing cluster... 2024-04-25T14:54:27.938 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 --force 2024-04-25T14:54:28.259 INFO:teuthology.orchestra.run.smithi053.stdout:Deleting cluster with fsid: fd30e6ce-030f-11ef-bc93-c7b262605968 2024-04-25T14:54:28.889 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 --force 2024-04-25T14:54:29.082 INFO:teuthology.orchestra.run.smithi150.stdout:Deleting cluster with fsid: fd30e6ce-030f-11ef-bc93-c7b262605968 2024-04-25T14:54:29.346 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid fd30e6ce-030f-11ef-bc93-c7b262605968 --force 2024-04-25T14:54:29.536 INFO:teuthology.orchestra.run.smithi183.stdout:Deleting cluster with fsid: fd30e6ce-030f-11ef-bc93-c7b262605968 2024-04-25T14:54:29.786 INFO:tasks.cephadm:Removing cephadm ... 2024-04-25T14:54:29.787 DEBUG:teuthology.orchestra.run.smithi053:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T14:54:29.848 DEBUG:teuthology.orchestra.run.smithi150:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T14:54:29.865 DEBUG:teuthology.orchestra.run.smithi183:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T14:54:29.880 INFO:tasks.cephadm:Teardown complete 2024-04-25T14:54:29.880 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T14:54:29.891 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T14:54:29.891 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T14:54:29.899 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T14:54:29.908 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T14:54:30.049 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T14:54:30.059 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T14:54:30.059 DEBUG:teuthology.orchestra.run.smithi053:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T14:54:30.067 DEBUG:teuthology.orchestra.run.smithi150:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T14:54:30.069 DEBUG:teuthology.orchestra.run.smithi183:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T14:54:30.082 INFO:teuthology.orchestra.run.smithi150.stderr:bash: line 1: ntpq: command not found 2024-04-25T14:54:30.083 INFO:teuthology.orchestra.run.smithi183.stderr:bash: line 1: ntpq: command not found 2024-04-25T14:54:30.086 INFO:teuthology.orchestra.run.smithi150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T14:54:30.086 INFO:teuthology.orchestra.run.smithi150.stdout:=============================================================================== 2024-04-25T14:54:30.086 INFO:teuthology.orchestra.run.smithi150.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 34 +1072us[+1071us] +/- 46ms 2024-04-25T14:54:30.087 INFO:teuthology.orchestra.run.smithi150.stdout:^* hv02.front.sepia.ceph.com 3 6 377 32 -1419us[-1420us] +/- 64ms 2024-04-25T14:54:30.087 INFO:teuthology.orchestra.run.smithi150.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 38 +23us[ +23us] +/- 71ms 2024-04-25T14:54:30.087 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:54:30.088 INFO:teuthology.orchestra.run.smithi183.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T14:54:30.088 INFO:teuthology.orchestra.run.smithi183.stdout:=============================================================================== 2024-04-25T14:54:30.088 INFO:teuthology.orchestra.run.smithi183.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 36 +1083us[+1083us] +/- 46ms 2024-04-25T14:54:30.088 INFO:teuthology.orchestra.run.smithi183.stdout:^* hv02.front.sepia.ceph.com 3 6 377 38 -1427us[-1433us] +/- 64ms 2024-04-25T14:54:30.089 INFO:teuthology.orchestra.run.smithi183.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 41 +14us[+8962ns] +/- 71ms 2024-04-25T14:54:30.089 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:54:30.117 INFO:teuthology.orchestra.run.smithi053.stderr:bash: line 1: ntpq: command not found 2024-04-25T14:54:30.126 INFO:teuthology.orchestra.run.smithi053.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T14:54:30.126 INFO:teuthology.orchestra.run.smithi053.stdout:=============================================================================== 2024-04-25T14:54:30.126 INFO:teuthology.orchestra.run.smithi053.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 34 +1078us[+1072us] +/- 46ms 2024-04-25T14:54:30.126 INFO:teuthology.orchestra.run.smithi053.stdout:^* hv02.front.sepia.ceph.com 3 6 377 32 -1431us[-1437us] +/- 64ms 2024-04-25T14:54:30.126 INFO:teuthology.orchestra.run.smithi053.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 32 +33us[ +27us] +/- 71ms 2024-04-25T14:54:30.126 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:54:30.128 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T14:54:30.138 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T14:54:30.138 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T14:54:30.148 DEBUG:teuthology.orchestra.run.smithi053:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T14:54:30.366 DEBUG:teuthology.orchestra.run.smithi150:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T14:54:30.435 DEBUG:teuthology.orchestra.run.smithi183:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T14:54:30.502 DEBUG:teuthology.orchestra.run.smithi053:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T14:54:30.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:54:30.603 DEBUG:teuthology.orchestra.run.smithi150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T14:54:30.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:54:30.633 DEBUG:teuthology.orchestra.run.smithi183:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T14:54:30.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:54:30.660 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T14:54:30.672 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T14:54:30.694 INFO:teuthology.task.internal:Duration was 2037.680831 seconds 2024-04-25T14:54:30.694 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T14:54:30.704 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T14:54:30.704 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T14:54:30.712 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T14:54:30.715 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T14:54:30.751 INFO:teuthology.orchestra.run.smithi183.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T14:54:30.752 INFO:teuthology.orchestra.run.smithi150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T14:54:30.860 INFO:teuthology.orchestra.run.smithi053.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T14:54:31.260 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T14:54:31.260 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi053.front.sepia.ceph.com 2024-04-25T14:54:31.260 DEBUG:teuthology.orchestra.run.smithi053:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T14:54:31.341 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi150.front.sepia.ceph.com 2024-04-25T14:54:31.342 DEBUG:teuthology.orchestra.run.smithi150:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T14:54:31.374 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi183.front.sepia.ceph.com 2024-04-25T14:54:31.374 DEBUG:teuthology.orchestra.run.smithi183:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T14:54:31.398 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T14:54:31.398 DEBUG:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T14:54:31.407 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T14:54:31.417 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T14:54:31.497 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T14:54:31.497 DEBUG:teuthology.orchestra.run.smithi053:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T14:54:31.503 DEBUG:teuthology.orchestra.run.smithi150:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T14:54:31.505 DEBUG:teuthology.orchestra.run.smithi183:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T14:54:32.047 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T14:54:32.059 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T14:54:32.059 DEBUG:teuthology.orchestra.run.smithi053:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T14:54:32.147 DEBUG:teuthology.orchestra.run.smithi150:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T14:54:32.172 DEBUG:teuthology.orchestra.run.smithi183:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T14:54:32.196 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T14:54:32.208 DEBUG:teuthology.orchestra.run.smithi053:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T14:54:32.216 DEBUG:teuthology.orchestra.run.smithi150:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T14:54:32.218 DEBUG:teuthology.orchestra.run.smithi183:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T14:54:32.239 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = core 2024-04-25T14:54:32.259 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = core 2024-04-25T14:54:32.315 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = core 2024-04-25T14:54:32.361 DEBUG:teuthology.orchestra.run.smithi053:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T14:54:32.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:54:32.413 DEBUG:teuthology.orchestra.run.smithi150:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T14:54:32.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:54:32.427 DEBUG:teuthology.orchestra.run.smithi183:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T14:54:32.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:54:32.441 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T14:54:32.455 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T14:54:32.456 DEBUG:teuthology.misc:Transferring archived files from smithi053:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673495/remote/smithi053 2024-04-25T14:54:32.456 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T14:54:32.561 DEBUG:teuthology.misc:Transferring archived files from smithi150:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673495/remote/smithi150 2024-04-25T14:54:32.561 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T14:54:32.597 DEBUG:teuthology.misc:Transferring archived files from smithi183:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673495/remote/smithi183 2024-04-25T14:54:32.597 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T14:54:32.634 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T14:54:32.634 DEBUG:teuthology.orchestra.run.smithi053:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T14:54:32.642 DEBUG:teuthology.orchestra.run.smithi150:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T14:54:32.645 DEBUG:teuthology.orchestra.run.smithi183:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T14:54:32.696 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T14:54:32.709 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T14:54:32.709 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T14:54:32.720 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T14:54:32.720 DEBUG:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T14:54:32.726 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T14:54:32.729 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T14:54:32.741 INFO:teuthology.orchestra.run.smithi150.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 14:54 /home/ubuntu/cephtest 2024-04-25T14:54:32.742 INFO:teuthology.orchestra.run.smithi183.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 14:54 /home/ubuntu/cephtest 2024-04-25T14:54:32.777 INFO:teuthology.orchestra.run.smithi053.stdout: 393610 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 14:54 /home/ubuntu/cephtest 2024-04-25T14:54:32.780 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T14:54:32.792 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T14:54:32.815 INFO:teuthology.nuke:Checking targets against current locks 2024-04-25T14:54:32.842 DEBUG:teuthology.nuke:shortname: smithi053 2024-04-25T14:54:32.842 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T14:54:32.868 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi053.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673495', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 14:11:49.865066', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGJM6JH6Pk6xemw5/TdSJdi6Ak3+jy57DEtD2j58Q0Di+iceZYPyedDZQkbwn0ylfTB475BKPxEYjZfAolPaKfY='} 2024-04-25T14:54:32.872 DEBUG:teuthology.nuke:shortname: smithi150 2024-04-25T14:54:32.872 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T14:54:32.896 DEBUG:teuthology.nuke:shortname: smithi183 2024-04-25T14:54:32.896 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T14:54:32.912 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi183.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673495', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 14:11:49.863707', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHKX7tMdcjyqITsVtCbAL8eIzb/rVW9tORkvJa4nMcOylbzH29W+LZT/CrxO0igJkvWpXYzf8pbEkMGbgFG3m/0='} 2024-04-25T14:54:32.921 INFO:teuthology.orchestra.console.smithi053:Power off 2024-04-25T14:54:32.921 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T14:54:32.932 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi150.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673495', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 14:11:49.866364', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFa/eCBrB7n0Pjo8xu/HaUXE71rX4OFlV6TbzV1r4WlItzqxBEy9mstSRPk2c8CErEC68nRDrHxYu40nVK+j0Qc='} 2024-04-25T14:54:32.944 DEBUG:teuthology.orchestra.console.smithi053:power off output: Chassis Power Control: Down/Off 2024-04-25T14:54:32.944 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T14:54:32.967 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is on 2024-04-25T14:54:32.983 INFO:teuthology.orchestra.console.smithi183:Power off 2024-04-25T14:54:32.983 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T14:54:32.998 INFO:teuthology.orchestra.console.smithi150:Power off 2024-04-25T14:54:32.999 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T14:54:33.007 DEBUG:teuthology.orchestra.console.smithi183:power off output: Chassis Power Control: Down/Off 2024-04-25T14:54:33.008 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T14:54:33.020 DEBUG:teuthology.orchestra.console.smithi150:power off output: Chassis Power Control: Down/Off 2024-04-25T14:54:33.021 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T14:54:33.029 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is on 2024-04-25T14:54:33.044 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is on 2024-04-25T14:54:36.968 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T14:54:37.030 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T14:54:37.045 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T14:54:37.079 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is on 2024-04-25T14:54:37.139 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is on 2024-04-25T14:54:37.153 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is on 2024-04-25T14:54:41.084 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T14:54:41.141 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T14:54:41.154 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T14:54:41.194 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is on 2024-04-25T14:54:41.251 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is on 2024-04-25T14:54:41.263 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is on 2024-04-25T14:54:45.197 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T14:54:45.252 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T14:54:45.264 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T14:54:45.307 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is off 2024-04-25T14:54:45.361 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is off 2024-04-25T14:54:45.374 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is off 2024-04-25T14:54:45.408 INFO:teuthology.orchestra.console.smithi053:Power off completed 2024-04-25T14:54:45.462 INFO:teuthology.orchestra.console.smithi183:Power off completed 2024-04-25T14:54:45.475 INFO:teuthology.orchestra.console.smithi150:Power off completed 2024-04-25T14:54:45.581 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} duration: 2037.6808314323425 failure_reason: 'Command failed on smithi053 with status 22: ''sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get run_dir''' flavor: default owner: scheduled_leonidus@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=7b3a822e70ef45d2877e0db6610e3fba status: fail success: false 2024-04-25T14:54:45.581 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T14:54:45.662 INFO:teuthology.run:FAIL